Traffic element matching method, device and computer storage medium based on posture

Abstract

The present application relates to a method, device, and computer storage medium for posture-based traffic element matching. The method includes: obtaining the posture information of the acquisition vehicle and the position M of the target traffic element in the image coordinate system in any frame of multiple images; obtaining a first transformation matrix from the earth coordinate system to the vehicle coordinate system based on the posture information of the acquisition vehicle in any frame of the multiple images, wherein the first transformation matrix includes a rotation matrix R and a translation matrix t; calculating the UTM coordinates P of the target traffic element in any frame of the multiple images based on a second transformation matrix, the intrinsic parameter matrix K of the camera carried by the acquisition vehicle, the position M of the target traffic element in the image coordinate system, the rotation matrix R, and the translation matrix t; clustering the UTM coordinates of the target traffic element in the multiple frames to obtain matching results of the target traffic element in the multiple frames. The technical solution of the present application can accurately match traffic elements.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, device, and computer storage medium for posture-based traffic element matching. Background Art

[0002] An electronic map is a software suite used for navigation on GPS devices, primarily for route planning and navigation. Electronic maps consist of general elements such as roads, backgrounds, annotations, and points of interest. They can also include a variety of specialized features, such as specific traffic elements like traffic lights. The production of high-precision electronic maps requires not only the gradual enrichment and improvement of traffic signs, such as traffic lights on the road or in the air, but also the identification of methods to improve the accuracy of element locations. While the accuracy of single-image resolution is insufficient, the accuracy of multiple images can be improved, but a method is required to determine the matching of the same traffic element across multiple images. Related technologies typically match traffic elements on electronic maps based on the target object's feature points. However, because feature points are easily affected by factors such as lighting and / or object shape, this matching method suffers from low accuracy. Summary of the Invention

[0003] In order to solve or partially solve the problems existing in the related art, the present application provides a posture-based traffic element matching method, device and computer storage medium, which can accurately match traffic elements in an image.

[0004] In a first aspect, the present application provides a posture-based traffic element matching method, comprising:

[0005] Obtain the pose information of the acquisition vehicle and the position M of the target traffic element in the image coordinate system in any frame of multiple images;

[0006] Acquire a first transformation matrix from a geodetic coordinate system to a vehicle coordinate system according to the pose information of the vehicle captured in any one frame of image, wherein the first transformation matrix includes a rotation matrix R and a translation matrix t;

[0007] Calculate the UTM coordinates P of the target traffic element in any frame of the image based on the second transformation matrix, the intrinsic parameter matrix K of the camera carried by the collection vehicle, the position M of the target traffic element in the image coordinate system, the rotation matrix R, and the translation matrix t, where the second transformation matrix is ​​a transformation relationship from the earth coordinate system to the camera coordinate system;

[0008] Clustering the UTM coordinates of the target traffic elements in the multiple frames of images to obtain matching results of the target traffic elements in the multiple frames of images.

[0009] A second aspect of the present application provides a posture-based traffic element matching device, comprising:

[0010] The first acquisition module is used to obtain the posture information of the acquisition vehicle and the position M of the target traffic element in the image coordinate system in any frame of multiple frames;

[0011] A second acquisition module is configured to acquire a first transformation matrix from a geodetic coordinate system to a vehicle coordinate system based on the posture information of the vehicle captured in any one frame of image, wherein the first transformation matrix includes a rotation matrix R and a translation matrix t;

[0012] a calculation module, configured to calculate the UTM coordinates P of the target traffic element in any frame of the image based on a second transformation matrix, an intrinsic parameter matrix K of the camera carried by the collection vehicle, a position M of the target traffic element in the image coordinate system, the rotation matrix R, and the translation matrix t, wherein the second transformation matrix is ​​a transformation relationship from the earth coordinate system to the camera coordinate system;

[0013] The clustering module is used to cluster the UTM coordinates of the target traffic elements in the multiple frames of images to obtain matching results of the target traffic elements in the multiple frames of images.

[0014] A third aspect of the present application provides an electronic device, including:

[0015] processor; and

[0016] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method described above.

[0017] A fourth aspect of the present application provides a computer storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method described above.

[0018] The technical solution provided by the present application may include the following beneficial effects: since the posture information of the collection vehicle and the position M of the target traffic element in the image coordinate system in any frame of multiple images are not affected by factors such as ambient lighting, and the intrinsic parameter matrix K of the camera carried by the collection vehicle is a property of the camera and is also not affected by factors such as ambient lighting, therefore, after solving the UTM coordinates P of the target traffic element in any frame of the image based on the second transformation matrix, the intrinsic parameter matrix K of the camera carried by the collection vehicle, the position M of the target traffic element in the image coordinate system, the rotation matrix R and the translation matrix t, the UTM coordinates of the target traffic elements in the multiple frames of images are clustered, which can achieve accurate matching of the traffic elements in the electronic map.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0021] Figure 1 1 is a flow chart of a method for matching traffic elements based on posture according to an embodiment of the present application;

[0022] Figure 2 is a structural diagram of a posture-based traffic element matching device shown in an embodiment of the present application;

[0023] Figure 3 is a schematic structural diagram of an electronic device shown in an embodiment of the present application;

[0024] Figure 4 2 is a schematic diagram showing two image frames containing target traffic elements (traffic lights) according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although the accompanying drawings illustrate embodiments of the present application, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0026] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0027] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0028] An electronic map is a software suite used for navigation on GPS devices, primarily for route planning and navigation. Electronic maps consist of general elements such as roads, backgrounds, annotations, and points of interest. They can also include a variety of specialized features, such as specific traffic elements like traffic lights. The production of high-precision electronic maps requires not only the gradual enrichment and improvement of traffic signs, such as traffic lights on the road or in the air, but also the identification of methods to improve the accuracy of element locations. While the accuracy of single-image resolution is insufficient, the accuracy of multiple images can be improved, but a method is required to determine the matching of the same traffic element across multiple images. Related technologies typically match traffic elements on electronic maps based on the target object's feature points. However, because feature points are easily affected by factors such as lighting and / or object shape, this matching method suffers from low accuracy.

[0029] To address the above issues, an embodiment of the present application provides a posture-based traffic element matching method that can accurately match traffic elements in an image.

[0030] The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0031] See also Figure 1 , is a flow chart of a method for matching traffic elements based on posture according to an embodiment of the present application, which mainly includes steps S101 to S104, as described below:

[0032] Step S101: Acquire the posture information of the acquisition vehicle and the position M of the target traffic element in the image coordinate system in any frame of multiple frames.

[0033] In an embodiment of the present application, the position and posture information of the collection vehicle in any frame of multiple images includes the position and posture of the collection vehicle in the geodetic coordinate system (or world coordinate system). These position and posture information can be obtained by fusing data acquired by sensors such as GPS, inertial measurement unit, laser radar and camera carried on the collection vehicle, and the position M of the target traffic element, such as the traffic light in the image coordinate system, can also be obtained by scanning and / or shooting by the laser radar and / or camera carried by the collection vehicle and then fused based on the algorithm. Of course, the position M of the target traffic element in the image coordinate system can also be calculated by the roadside unit after uploading the image of the target traffic element to the backend server. The present application does not limit the method for obtaining the position and posture information of the collection vehicle and the position M of the target traffic element in the image coordinate system.

[0034] Step S102: acquiring a first transformation matrix from the earth coordinate system to the vehicle coordinate system based on the posture information of the vehicle captured in any one of the multiple frames of images, wherein the first transformation matrix includes a rotation matrix R and a translation matrix t.

[0035] As previously mentioned, the pose information of the collection vehicle is its pose information in the geodetic coordinate system. This pose information can be calculated from the data collected by the sensors onboard the collection vehicle. When the collection vehicle uses its onboard sensors to collect images, each frame of the image has a corresponding timestamp. Based on this timestamp, the pose information of the collection vehicle in the geodetic coordinate system and the pose information of the sensor in the device coordinate system at the same moment can be obtained. Based on these two pose information, the transformation matrix from the geodetic coordinate system to the device coordinate system can be obtained. Since the relative position of the sensor and the collection vehicle is fixed, this means that the conversion relationship between the coordinate system where the sensor is located, namely the device coordinate system, and the coordinate system where the collection vehicle is located, namely the vehicle coordinate system, is relatively fixed. Therefore, obtaining the transformation matrix from the geodetic coordinate system to the device coordinate system is equivalent to obtaining the transformation matrix from the geodetic coordinate system to the vehicle coordinate system, namely the first transformation matrix, which includes the rotation matrix R and the translation matrix t.

[0036] Step S103: Calculate the UTM coordinates P of the target traffic element in any frame of the multi-frame image based on the second transformation matrix, the intrinsic parameter matrix K of the camera carried by the collection vehicle, the position M of the target traffic element in the image coordinate system, the rotation matrix R, and the translation matrix t, where the second transformation matrix represents the transformation relationship from the geodetic coordinate system to the camera coordinate system.

[0037] Specifically, as an embodiment of the present application, step S103 may be implemented by steps S1031 to S1034, which are described in detail as follows:

[0038] Step S1031: Obtain an implicit expression of the coordinates P″ of the target traffic element in any frame of the multiple frames in the camera coordinate system according to the second transformation matrix, the rotation matrix R, and the translation matrix t.

[0039] Assume that the coordinates of the target traffic element in the geodetic coordinate system are P w , where P w is the unknown quantity and is recorded as:

[0040]

[0041] After step S102, the transformation matrix from the earth coordinate system to the vehicle coordinate system, i.e., the first transformation matrix, is obtained. Therefore, the coordinate P′ of the target traffic element in the vehicle coordinate system can be expressed as:

[0042]

[0043] In the above expression, These are all known quantities r1 obtained from the aforementioned examples.

[0044] In the embodiment of the present application, the transformation relationship from the earth coordinate system to the camera coordinate system, i.e., the second transformation matrix, can be obtained by calibration. Similar to the first transformation matrix mentioned above, the second transformation matrix also includes a rotation matrix and a translation matrix. The rotation matrix of the second transformation matrix is ​​denoted as R c , the translation matrix of the second transformation matrix is ​​t c , then the implicit expression of the coordinate P" of the target traffic element in any frame of image in the camera coordinate system is:

[0045] P″=R c P′+t c =R c (RP w +t)+t c ………………(Expression 1)

[0046] It should be noted that the reason why the expression of the coordinate P” of the target traffic element in any frame of image in the camera coordinate system is called an implicit expression is because the expression of the coordinate P” still contains some unknown quantities, causing the coordinate P” itself to become an unknown quantity.

[0047] Step S1032: Based on the camera's intrinsic parameter matrix K and the coordinate P' of the target traffic element in the camera coordinate system, the coordinate P' is mapped to the image coordinate system to obtain an expression of the position M of the target traffic element in the image coordinate system with respect to the intrinsic parameter matrix K and the coordinate P'.

[0048] In the embodiments of the present application, the camera's intrinsic parameter matrix K is provided by the camera manufacturer or calibrated by the user through other methods after delivery and use for a period of time. The specific calibration method will be described in the embodiments below and will not be discussed here for the time being. Specifically, based on the camera's intrinsic parameter matrix K and the coordinates P' of the target traffic feature in the camera coordinate system, the coordinates P' can be mapped to the image coordinate system according to the following expression:

[0049] M=P″………………(Expression 2)

[0050] Step S1033: Jointly solve the implicit expression of the coordinate P" of the target traffic element in any frame of the multi-frame image in the camera coordinate system and the expression of the position M of the target traffic element in the image coordinate system with respect to the camera's intrinsic parameter matrix K and the coordinate P", and obtain the coordinate P of the target traffic element in any frame of the multi-frame image in the earth coordinate system. w .

[0051] As mentioned above, since the position M of the target traffic element in the image coordinate system can be obtained in many ways, Expression 1 has only one unknown quantity. Therefore, by combining Expression 1 and Expression 2, we can easily obtain:

[0052] M=K[Rc (RP w +t)+t c ]………………(Expression 3)

[0053] The above expression 3 actually has only one unknown quantity, which is the coordinate P of the target traffic element in any frame of the multi-frame image in the geodetic coordinate system. w , it is not difficult to get:

[0054]

[0055] Among them, R -1 、 and K -1 They are R, R c and K are the inverse matrices of these three matrices, while R, R c , K, M, t c and t are both known quantities, so it is not difficult to calculate the coordinates P of the target traffic element in the geodetic coordinate system in any frame of the above image. w .

[0056] Step S1034: according to the coordinate P of the target traffic element in any frame of the multiple frames of images in the geodetic coordinate system w As well as the transformation matrix between the geodetic coordinate system and the UTM coordinate system, the UTM coordinates of the target traffic element in any frame of the multiple frames are obtained.

[0057] The UTM coordinate system is a plane rectangular coordinate system corresponding to the Universal Transverse Mercator Grid System (UTM) coordinate system. Since the UTM coordinate system and the geodetic coordinate system have a fixed transformation relationship, that is, the transformation matrix between the geodetic coordinate system and the UTM coordinate system, the coordinates P of the target traffic element in any frame of the multiple frames are obtained through the above steps S1031 to S1033. w Afterwards, according to the coordinate P of the target traffic element in any frame of the multi-frame image in the geodetic coordinate system w As well as the transformation matrix between the geodetic coordinate system and the UTM coordinate system, the UTM coordinates of the target traffic element in any frame of the multiple frames are obtained.

[0058] The above embodiment mentioned that the intrinsic parameter matrix K of the camera carried by the collection vehicle can be directly applied after being provided by the camera manufacturer, or can be obtained through calibration, and this application does not limit this. Taking into account that due to various reasons, the original intrinsic parameter matrix K of the camera carried by the collection vehicle may no longer be accurate after being used for a period of time. Therefore, in the embodiment of the present application, the intrinsic parameter matrix K of the camera carried by the collection vehicle can still be obtained through calibration. Specifically, as an embodiment of the present application, obtaining the intrinsic parameter matrix K of the camera carried by the collection vehicle through calibration can be: when the calibration target is located in the calibration domain, moving the camera carried by the collection vehicle to a position corresponding to the calibration domain; photographing the calibration target to obtain the posture image of the calibration target; determining the pixel coordinates of the calibration map in the calibration target based on the posture image of the calibration target in each calibration domain; and calculating the intrinsic parameter matrix K of the camera based on the pixel coordinates of each calibration map and the coordinates of the calibration map of the calibration target in the geodetic coordinates. In the above embodiment, the calibration domain is a pre-set area within the field of view of the camera so that the camera can photograph the calibration target image within the calibration domain. The calibration diagram is the pattern of the calibration target itself, which can be a square, circle, or other pattern that can be used for calibration. Each calibration domain can be pre-set with the corresponding camera position. When it is necessary to capture the pose image of the calibration target on the calibration domain, the camera can be moved to the position corresponding to the calibration domain. For example, the camera position is set to the position directly above the center of the calibration domain. When it is necessary to capture the calibration target in the calibration domain, the camera can be moved to the position directly above the center of the calibration domain so that the camera can capture the pose image of the calibration target vertically. Alternatively, the camera position is set to the left or right of the center of the calibration domain. When it is necessary to capture the calibration target in the calibration domain, the camera can be moved to the left or right of the center of the calibration domain so that the camera can capture the pose image of the calibration target at an angle, and so on. In addition, in order to prevent missing the shooting of the calibration target, the above embodiment also includes checking the integrity of the posture image of the calibration target when the camera shoots the calibration target; when the posture image does not pass the verification, the camera is controlled to re-shoot the calibration target to obtain the posture image of the calibration target again.

[0059] As another embodiment of the present application, obtaining the intrinsic parameter matrix K of the camera carried by the collection vehicle through calibration can also be: determining the camera calibratable area in the calibration image taken by the camera; if the camera calibratable area meets the preset conditions, determining the calibration image as a successfully calibrated image; calibrating the camera based on the successfully calibrated image to obtain the camera's intrinsic parameter matrix K. In the above embodiment, the judgment standard of the calibratable area refers to that the overlap between the calibration domain and the specified calibration area is not less than a preset threshold, and the method of determining the camera calibratable area in the calibration image taken by the camera can be to use the area covered by the calibration domain as the camera's calibratable area, or to determine the calibratable area based on a specific point, wherein, when the calibration plate is a circular array calibration plate, the specific point can be the center of the circle formed by the circular array, and when the calibration plate is a checkerboard calibration plate, the specific point can be a corner point of the checkerboard.

[0060] Taking into account that when the camera is used on the collection vehicle, due to reasons such as bumpy and rugged roads, the intrinsic parameter matrix K of the calibrated camera may become inaccurate, affecting the calculation result of the UTM coordinate P of the target traffic element in any frame image, therefore, in the embodiment of the present application, after obtaining the intrinsic parameter matrix K of the calibrated camera, the intrinsic parameter matrix K of the calibrated camera can also be corrected, specifically by: obtaining the characteristics of the current calibration field in the vehicle coordinate system in real time; calculating the third transformation matrix based on the characteristics of the calibration field and the characteristics of the current calibration field in the vehicle coordinate system; obtaining the vehicle during calibration; The camera-vehicle transformation matrix from the coordinate system to the camera coordinate system during calibration; the camera's intrinsic parameter matrix K is corrected in real time based on the third transformation matrix, the camera-vehicle transformation matrix, and the vehicle coordinate system during calibration. The third transformation matrix in the above embodiment is defined as the product of the characteristics of the current calibration field and the calibration field characteristics. The calibration field characteristics are the characteristics of the calibration field acquired by the camera during calibration. Real-time acquisition of the characteristics of the current calibration field in the vehicle coordinate system can be achieved by photographing the characteristic points of the calibration field with the camera to obtain their three-dimensional information, and then fitting this three-dimensional information to obtain the characteristics of the current calibration field in the vehicle coordinate system. As for the real-time correction of the camera's intrinsic parameter matrix K based on the third transformation matrix, the camera-vehicle transformation matrix, and the vehicle coordinate system during calibration, one implementation method is to calculate the product of the third transformation matrix and the camera-vehicle transformation matrix to perform real-time correction on the camera's intrinsic parameter matrix K.

[0061] Step S104: clustering the UTM coordinates of the target traffic elements in the multiple frames of images to obtain matching results of the target traffic elements in the multiple frames of images.

[0062] After step S101 to step S103, the UTM coordinates of each target traffic element in the multi-frame image are obtained. Through certain features of the UTM coordinates of each target traffic element, such as close values, it can be considered that these target traffic elements in the multi-frame image belong to the same traffic element. In an embodiment of the present application, the UTM coordinates of the target traffic elements in the multi-frame image can be clustered. If the UTM coordinates of the target traffic elements in the multi-frame image belong to the same cluster, then the target traffic elements in these multi-frame images are the same traffic element, and the image frames where these same traffic elements are located can be divided into the same group. As for the specific clustering algorithm, it can be any one or a combination of multiple clustering algorithms including K-MEANS clustering, mean shift clustering, DBSCAN clustering, expectation maximization clustering of Gaussian mixture model and hierarchical clustering algorithm, and this application does not limit this. The following is combined with Figure 4 Given two pictures (labeled by A and B respectively), it shows Figure 1 An example of a technical solution for a posture-based traffic element matching method.

[0063] Images A and B are scanned or photographed by laser radar and / or camera equipment carried by the collection vehicle. They contain the target traffic element, namely the traffic light framed by the rectangular frame in the two images (i.e., the area indicated by the arrow at the location of the letter C in the image). Since these collection vehicles are also equipped with positioning equipment such as GPS and inertial measurement units, when taking images A and B, the data obtained by the above-mentioned laser radar and positioning equipment carried by the collection vehicle are fused based on some algorithms to obtain the position information of the collection vehicle in image A and the position M of the traffic light in the image coordinate system. a As well as the pose information of the collected vehicle in picture B and the position M of the traffic light in the image coordinate system b .

[0064] When the collection vehicle uses its onboard sensors to capture images, each frame is timestamped. This timestamp allows us to determine the vehicle's pose in the geodetic coordinate system and the sensor's pose in the device coordinate system at the same moment. Based on these two poses, we can derive the transformation matrix from the geodetic coordinate system to the device coordinate system. Since the relative positions of the sensor and the collection vehicle are fixed, the transformation relationship between the sensor's coordinate system (the device coordinate system) and the vehicle's coordinate system (the vehicle coordinate system) is relatively fixed. Therefore, obtaining the transformation matrix from the geodetic coordinate system to the device coordinate system is equivalent to obtaining the transformation matrix from the geodetic coordinate system to the vehicle coordinate system, the first transformation matrix, which includes the rotation matrix R and the translation matrix t. It should be noted that since the transformation relationship from the geodetic coordinate system to the vehicle coordinate system is fixed, we can obtain the first transformation matrix from the geodetic coordinate system to the vehicle coordinate system based on either the pose information of the collection vehicle in Image A or the pose information of the collection vehicle in Image B.

[0065] Our goal is to obtain the UTM coordinates of the traffic lights enclosed by the rectangular boxes in pictures A and B. Here we obtain the UTM coordinates P of the traffic light enclosed by the rectangular box in picture A. a For example, the UTM coordinates P of the traffic light enclosed by the rectangular frame in picture B are b The calculation principle is similar. One implementation scheme is to obtain the implicit expression of the coordinate P" of the traffic light in picture A in the camera coordinate system based on the second transformation matrix, the rotation matrix R and the translation matrix t; according to the camera's intrinsic parameter matrix K and the coordinate P" of the traffic light in picture A in the camera coordinate system, map the coordinate P" to the image coordinate system to obtain the expression of the position M of the traffic light in picture A in the image coordinate system with respect to the intrinsic parameter matrix K and the coordinate P"; jointly solve the implicit expression of the coordinate P" of the traffic light in picture A in the camera coordinate system and the expression of the position M of the traffic light in picture A in the image coordinate system with respect to the intrinsic parameter matrix K and the coordinate P", and obtain the coordinate P" of the traffic light in picture A in the geodetic coordinate system. w ; According to the coordinates P of the traffic light in the geodetic coordinate system in picture A w And the transformation matrix between the geodetic coordinate system and the UTM coordinate system, we get the UTM coordinate P of the traffic light in picture A a Similarly, we can get the UTM coordinates P of the traffic light enclosed by the rectangular frame in picture B. b .

[0066] Get the UTM coordinate P of the traffic light in picture A a And the UTM coordinate P of the traffic light framed by the rectangular box in picture B bAfter that, the UTM coordinates P can be clustered based on any one or a combination of clustering algorithms such as K-MEANS clustering, mean shift clustering, DBSCAN clustering, Gaussian mixture model expectation maximization clustering and hierarchical clustering algorithm. a and UTM coordinates P b Perform clustering. If the UTM coordinate P a and UTM coordinates P b If the traffic lights in the rectangular box in picture A and picture B belong to the same cluster, then the traffic lights in the rectangular box in picture A and picture B are the same traffic lights. The pictures A and B containing the same traffic lights can be divided into the same group.

[0067] From the above Figure 1 From the example of the posture-based traffic element matching method, it can be seen that the posture information of the collection vehicle and the position M of the target traffic element in the image coordinate system in any frame of multiple frames are not affected by factors such as ambient lighting, and the intrinsic parameter matrix K of the camera carried by the collection vehicle is a property of the camera and is also not affected by factors such as ambient lighting. Therefore, after solving the UTM coordinates P of the target traffic element in any frame of the image based on the second transformation matrix, the intrinsic parameter matrix K of the camera carried by the collection vehicle, the position M of the target traffic element in the image coordinate system, the rotation matrix R and the translation matrix t, the UTM coordinates of the target traffic elements in multiple frames are clustered, which can achieve accurate matching of traffic elements in the electronic map.

[0068] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a posture-based traffic element matching device, electronic device and corresponding embodiments.

[0069] See also Figure 2 , is a schematic diagram of the structure of a posture-based traffic element matching device according to an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown. Figure 2 The exemplary posture-based traffic element matching device mainly includes a first acquisition module 201, a second acquisition module 202, a solution module 203 and a clustering module 204, wherein:

[0070] The first acquisition module 201 is used to obtain the pose information of the acquisition vehicle and the position M of the target traffic element in the image coordinate system in any frame of multiple frames;

[0071] A second acquisition module 202 is configured to acquire a first transformation matrix from a geodetic coordinate system to a vehicle coordinate system based on the posture information of the vehicle collected in any one of the multiple image frames, wherein the first transformation matrix includes a rotation matrix R and a translation matrix t;

[0072] A calculation module 203 is configured to calculate the UTM coordinates P of the target traffic element in any frame of the multiple images based on the second transformation matrix, the intrinsic parameter matrix K of the camera carried by the collection vehicle, the position M of the target traffic element in the image coordinate system, the rotation matrix R, and the translation matrix t, wherein the second transformation matrix represents the transformation relationship from the earth coordinate system to the camera coordinate system;

[0073] The clustering module 204 is configured to cluster the UTM coordinates of the target traffic elements in the multiple frames of images to obtain matching results of the target traffic elements in the multiple frames of images.

[0074] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0075] From the above Figure 2 It can be seen from the example of the posture-based traffic element matching device that the posture information of the collection vehicle and the position M of the target traffic element in the image coordinate system in any frame of multiple frames are not affected by factors such as ambient lighting, and the intrinsic parameter matrix K of the camera carried by the collection vehicle is a property of the camera and is also not affected by factors such as ambient lighting. Therefore, after solving the UTM coordinates P of the target traffic element in any frame of the image based on the second transformation matrix, the intrinsic parameter matrix K of the camera carried by the collection vehicle, the position M of the target traffic element in the image coordinate system, the rotation matrix R and the translation matrix t, the UTM coordinates of the target traffic elements in multiple frames of images are clustered, which can achieve accurate matching of traffic elements in the electronic map.

[0076] Figure 3 It is a structural diagram of an electronic device shown in an embodiment of the present application.

[0077] See also Figure 3 , the electronic device 300 includes a memory 310 and a processor 320.

[0078] The processor 320 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0079] The memory 310 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. ROM may store static data or instructions required by the processor 320 or other modules of the computer. The permanent storage may be a readable and writable storage device. The permanent storage may be a non-volatile storage device that retains stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (e.g., a magnetic or optical disk, flash memory) as the permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (e.g., a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. The system memory may store some or all instructions and data required by the processor during operation. In addition, the memory 310 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used. In some embodiments, the memory 310 may include a readable and / or writable removable storage device, such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0080] The memory 310 stores executable codes. When the executable codes are processed by the processor 320 , the processor 320 may execute part or all of the above-mentioned methods.

[0081] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0082] Alternatively, the present application can also be implemented as a computer storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0083] The embodiments of the present application have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein.

Claims

1. A traffic element matching method based on posture, characterized in that: The method comprises: Obtain the pose information of the acquisition vehicle and the position M of the target traffic element in the image coordinate system in any frame of multiple images; Acquire a first transformation matrix from a geodetic coordinate system to a vehicle coordinate system according to the pose information of the vehicle captured in any one frame of image, wherein the first transformation matrix includes a rotation matrix R and a translation matrix t; Calculate the UTM coordinates P of the target traffic element in any frame of the image based on the second transformation matrix, the intrinsic parameter matrix K of the camera carried by the collection vehicle, the position M of the target traffic element in the image coordinate system, the rotation matrix R, and the translation matrix t, wherein the second transformation matrix represents the transformation relationship from the geodetic coordinate system to the camera coordinate system; Clustering the UTM coordinates of the target traffic elements in the multiple frames of images to obtain matching results of the target traffic elements in the multiple frames of images.

2. The posture-based traffic element matching method according to claim 1, characterized in that: Calculating the UTM coordinates P of the target traffic element in any one frame of the image based on the second transformation matrix, the intrinsic parameter matrix K of the camera carried by the collection vehicle, the position M of the target traffic element in the image coordinate system, the rotation matrix R, and the translation matrix t includes: Obtaining an implicit expression for the coordinates P″ of the target traffic element in the arbitrary frame image in the camera coordinate system according to the second transformation matrix, the rotation matrix R, and the translation matrix t; According to the intrinsic parameter matrix K of the camera and the coordinate P' of the target traffic element in the camera coordinate system, the coordinate P' is mapped to the image coordinate system, and an expression of the position M of the target traffic element in the image coordinate system with respect to the intrinsic parameter matrix K and the coordinate P' is obtained; Jointly solve the implicit expression of the coordinate P" of the target traffic element in the camera coordinate system in any frame of the image and the expression of the position M of the target traffic element in the image coordinate system with respect to the intrinsic parameter matrix K and the coordinate P", and obtain the coordinate P of the target traffic element in the earth coordinate system in any frame of the image. w ; According to the coordinate P of the target traffic element in any frame of the multiple frames of images in the geodetic coordinate system w and the transformation matrix between the geodetic coordinate system and the UTM coordinate system, and obtain the UTM coordinates of the target traffic element in any frame of the multiple frames.

3. The posture-based traffic element matching method according to claim 1, characterized in that: The method further includes: obtaining an intrinsic parameter matrix K of the camera carried by the collection vehicle through calibration.

4. The method for matching traffic elements based on posture according to claim 3, characterized in that: The calibration step of obtaining the intrinsic parameter matrix K of the camera carried by the acquisition vehicle includes: When the calibration target is located in the calibration domain, moving the camera to a position corresponding to the calibration domain; photographing the calibration target to obtain a posture image of the calibration target; Determining pixel coordinates of a calibration image in the calibration target according to the pose image of the calibration target in each calibration domain; The intrinsic parameter matrix K of the camera is calculated according to the pixel coordinates of each calibration image and the coordinates of the calibration image of the calibration target in the geodetic coordinates.

5. The posture-based traffic element matching method according to claim 4, characterized in that: The method further comprises: When the camera captures the calibration target, performing integrity verification on the pose image of the calibration target; When the pose image fails the verification, the camera is controlled to re-photograph the calibration target to obtain the pose image of the calibration target again.

6. The method for matching traffic elements based on posture according to claim 3, characterized in that: The calibration step of obtaining the intrinsic parameter matrix K of the camera carried by the acquisition vehicle includes: Determining a camera calibratable area in a calibration image captured by the camera; If the camera calibrable area meets the preset conditions, determining that the calibrated image is a successfully calibrated image; The camera is calibrated based on the successfully calibrated image to obtain an intrinsic parameter matrix K of the camera.

7. The posture-based traffic element matching method according to claim 3, characterized in that: The method further comprises: Acquiring features of a current calibration field in the vehicle coordinate system in real time; calculating a third transformation matrix based on the features of the calibration field and features of the current calibration field in the vehicle coordinate system, wherein the features of the calibration field are features of the calibration field acquired by the camera during calibration; Obtaining a camera-vehicle transformation matrix from the vehicle coordinate system during calibration to the camera coordinate system during calibration; The camera's intrinsic parameter matrix K is corrected in real time according to the third transformation matrix, the camera-to-vehicle conversion matrix, and the vehicle coordinate system during calibration.

8. A traffic element matching device based on posture, characterized in that: The device comprises: The first acquisition module is used to obtain the posture information of the acquisition vehicle and the position M of the target traffic element in the image coordinate system in any frame of multiple frames; A second acquisition module is configured to acquire a first transformation matrix from a geodetic coordinate system to a vehicle coordinate system based on the posture information of the vehicle captured in any one frame of image, wherein the first transformation matrix includes a rotation matrix R and a translation matrix t; a solving module, configured to solve the UTM coordinate P of the target traffic element in any frame of the image based on a second transformation matrix, an intrinsic parameter matrix K of the camera carried by the collection vehicle, a position M of the target traffic element in the image coordinate system, the rotation matrix R, and the translation matrix t, wherein the second transformation matrix represents the transformation relationship from the earth coordinate system to the camera coordinate system; The clustering module is used to cluster the UTM coordinates of the target traffic elements in the multiple frames of images to obtain matching results of the target traffic elements in the multiple frames of images.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to perform the method according to any one of claims 1 to 7.

10. A computer storage medium having executable codes stored thereon, wherein when the executable codes are executed by a processor of an electronic device, the processor is caused to execute the method according to any one of claims 1 to 7.

Images