Method, device and electronic equipment for updating traffic signs in high-precision maps

Abstract

The embodiments of the present application provide a method, device and electronic device for updating traffic signs in a high-precision map, wherein an on-board controller can generate update information of traffic signs based on a driving video frame sequence collected by an on-board camera, and send the update information to a map provider, so that the map provider can update the traffic signs in a timely manner. During this process, the map provider is not required to perform on-site scanning of traffic signs in real time, which greatly reduces the consumption of manpower and resources.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving map technology, and in particular to a method, device and electronic equipment for updating traffic signs in a high-precision map. Background Art

[0002] High-precision maps are crucial for current car driving. They can provide users with road network information and recommend driving routes from the starting point to the destination, bringing convenience to driving.

[0003] Traffic landmarks in high-precision maps (for example, traffic signs) can not only provide important road information attributes, but also provide prior information for visual positioning, especially longitudinal positioning. Usually, map providers will use surveying and mapping equipment and auxiliary three-dimensional scanning equipment such as lidar to scan, extract, store and other production-related operations on traffic signs on both sides of the road to complete the production of high-precision maps and traffic signs.

[0004] However, facing a city or larger scale scenario, the continuous updating of high-precision maps after production has become a thorny issue. Currently, we continue to rely on surveying equipment and map collection vehicles to perform repeated scanning operations to ensure real-time updates. Although this is feasible in a small range, as the coverage area continues to expand and the map scale continues to grow, the manpower and resources consumed by this method of map updating are extremely huge, so that it becomes impossible to achieve real-time confirmation, updating and adding some road-related attributes. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method, device and electronic equipment for updating traffic signs in high-precision maps, which can send the update information of traffic signs to the map provider through the driver's vehicle, so that the map provider can update the traffic signs in time. In this process, the map provider does not need to perform on-site scanning of traffic signs in real time, which greatly reduces the consumption of manpower and resources.

[0006] In a first aspect, an embodiment of the present invention provides a method for updating traffic signs in a high-precision map, wherein the method is applied to a vehicle-mounted controller, and the vehicle-mounted controller is also communicatively connected to a vehicle-mounted camera and a vehicle-mounted positioning module, and the method comprises: obtaining a driving video frame sequence captured by the vehicle-mounted camera during vehicle driving; performing traffic sign detection and tracking on the driving video frame sequence based on the positioning information collected by the vehicle-mounted positioning module, and obtaining driving video frames marked with traffic sign detection frames and driving trajectory information of the same traffic sign; wherein the driving trajectory information is a driving video frame subsequence containing the same detection frame identifier; taking the traffic sign corresponding to each driving trajectory information as a first object, and executing The following operations: calculate a first map position of a first object in a high-precision map based on a driving video frame subsequence; adjust the position of the first map position to obtain a second map position; calculate an average projection error of a center point of the first object based on the second map position; when the average projection error is less than a preset projection error threshold and the number of frames of the driving video frame subsequence is greater than a preset number of video frames, search for a third map position of the first object in the high-precision map based on the second map position; if the distance between the third map position and the second map position is greater than a preset distance threshold, generate update information and send it to a map supplier of the high-precision map, so that the map supplier updates the position of the first object in the high-precision map according to the second map position carried in the update information.

[0007] The above-mentioned step of calculating the first map position of the first object in the high-precision map based on the driving video frame subsequence includes: calculating the central pixel of the center point of the first object based on the driving video frame subsequence; calculating the camera projection matrix of the vehicle camera based on the central pixel and the camera parameters of the vehicle camera; calculating the first map position based on the camera projection matrix and the central position information of the traffic sign detection box included in each video frame in the driving video frame subsequence.

[0008] The first map position is calculated as follows: Among them, [u N , v N ] represents the center position information of the traffic sign detection frame included in the Nth video frame in the driving video frame subsequence, P N (1), P N (2), P N (3) are the first row matrix value, the second row matrix value and the third row matrix value of the Nth camera projection matrix corresponding to the first object, respectively. i Represents the i-th first object.

[0009] The above-mentioned step of adjusting the position of the first map position to obtain the second map position includes: constructing an optimization function based on the center position information of the traffic sign detection frame included in each video frame in the driving video frame subsequence corresponding to the first object, the first map position corresponding to the first object, and the camera projection matrix; adjusting the first map position and the camera projection matrix until the optimization function reaches a preset function value, thereby obtaining the adjusted second map position.

[0010] The above-mentioned step of calculating the average projection error of the center point of the first object based on the second map position includes: calculating the average projection error based on the center position information of the traffic sign detection box included in each video frame in the driving video frame subsequence corresponding to the first object, the second map position corresponding to the first object and the camera projection matrix.

[0011] The above average projection error is calculated by the following formula: Where N represents the number of video frames in the driving video frame subsequence, X i represents the i-th first object, P j represents the camera projection matrix corresponding to the first object, u ij represents the pixel [u, v] of the center point of the i-th first object on the j-th video frame, π(P j , X i ) represents the camera projection matrix P j The first two pixels are taken out under the transformation.

[0012] The above-mentioned step of searching for the third map position of the first object in the high-precision map based on the second map position includes: searching for identification information within a radius with the second map position as the center and a preset search distance in the high-precision map; searching for target identification information that matches the object identification information of the first object from multiple identification information; and determining the map position corresponding to the target identification information as the third map position.

[0013] In a second aspect, an embodiment of the present invention further provides a device for updating traffic signs in a high-precision map, wherein the device is applied to a vehicle-mounted controller, and the vehicle-mounted controller is also communicatively connected to the vehicle-mounted camera and the vehicle-mounted positioning module, and the above-mentioned device includes: an acquisition module, used to acquire a driving video frame sequence captured by the vehicle-mounted camera during vehicle driving; a detection and tracking module, used to perform traffic sign detection and tracking on the driving video frame sequence based on the positioning information captured by the vehicle-mounted positioning module, and obtain driving video frames marked with traffic sign detection frames and driving trajectory information of the same traffic sign; wherein the driving trajectory information is a subsequence of driving video frames containing the same detection frame identifier; an execution module, used to take the traffic sign corresponding to each driving trajectory information as a first object, and perform the following operations for each first object: a first calculation A module for calculating a first map position of a first object in a high-precision map based on a driving video frame subsequence; a position adjustment module for adjusting the first map position to obtain a second map position; a second calculation module for calculating an average projection error of a center point of the first object based on the second map position; a search module for searching for a third map position of the first object in the high-precision map based on the second map position when the average projection error is less than a preset projection error threshold and the number of frames of the driving video frame subsequence is greater than a preset number of video frames; and a generation module for generating update information and sending it to a map supplier of the high-precision map if the distance between the third map position and the second map position is greater than a preset distance threshold, so that the map supplier updates the position of the first object in the high-precision map according to the second map position carried in the update information.

[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, which includes a processor and a memory, the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above method.

[0015] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above method.

[0016] The embodiments of the present invention bring the following beneficial effects:

[0017] The embodiments of the present application provide a method, device and electronic device for updating traffic signs in a high-precision map, wherein a vehicle-mounted camera can be used to detect and track traffic signs in a collected driving video frame sequence, and a driving video frame marked with a traffic sign detection frame and driving trajectory information of the same traffic sign are obtained. The traffic sign corresponding to each driving trajectory information is respectively used as a first object, and a first map position of the first object in the high-precision map is calculated based on a driving video frame subsequence; the first map position is adjusted to obtain a second map position; the average projection error of the center point of the first object is calculated based on the second map position; when the average projection error is less than a preset projection error threshold and the number of frames of the driving video frame subsequence is greater than a preset number of video frames, the third map position of the first object in the high-precision map is searched based on the second map position; if the distance between the third map position and the second map position is greater than the preset distance threshold, update information is generated and sent to the map supplier of the high-precision map, so that the map supplier updates the position of the first object in the high-precision map according to the second map position carried in the update information. The vehicle-mounted controller of the present application can generate update information of traffic signs based on the driving video frame sequence collected by the vehicle-mounted camera, and send the update information to the map provider so that the map provider can update the traffic signs in a timely manner. In this process, the map provider does not need to perform on-site scanning of traffic signs in real time, which greatly reduces the consumption of manpower and resources.

[0018] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flow chart of a method for updating traffic signs in a high-precision map provided by an embodiment of the present invention;

[0022] Figure 2 A flowchart of another method for updating traffic signs in a high-precision map provided by an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of the structure of a device for updating traffic signs in a high-precision map provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0026] Considering that the update of traffic signs in existing high-precision maps still relies on surveying and mapping equipment and map collection vehicles to perform repeated scanning operations for updating, the manpower and resources consumed by this method of map updating are extremely huge, so that it becomes impossible to instantly confirm, update and add some road-related attributes. Based on this, an embodiment of the present invention provides a method, device and electronic device for updating traffic signs in a high-precision map, which generates update information of traffic signs based on a driving video frame sequence collected by a vehicle-mounted camera, and sends the update information to a map supplier, so that the map supplier can update the traffic signs in a timely manner. In this process, the map supplier does not need to perform on-site scanning of traffic signs in real time, which greatly reduces the consumption of manpower and resources.

[0027] This embodiment provides a method for updating traffic signs in a high-precision map, wherein the method is applied to a vehicle controller, which is also connected to the vehicle camera and the vehicle positioning module; see Figure 1 A flow chart of a method for updating traffic signs in a high-precision map is shown, and the method specifically comprises the following steps:

[0028] Step S102, obtaining a driving video frame sequence captured by a vehicle-mounted camera during the driving process of the vehicle;

[0029] Usually, the vehicle-mounted camera is installed in front of the windshield of the vehicle to collect the driving video frame sequence.

[0030] Step S104, performing traffic sign detection and tracking on the driving video frame sequence based on the positioning information collected by the vehicle positioning module, and obtaining driving video frames marked with traffic sign detection frames and driving trajectory information of the same traffic sign; wherein the driving trajectory information is a subsequence of driving video frames containing the same detection frame identifier;

[0031] In this embodiment, the vehicle-mounted positioning module can select a GNSS (Global Navigation Satellite System) / IMU (Inertial Measurement Unit) combined positioning device, or a GNSS satellite positioning device to perform real-time positioning of the vehicle to obtain positioning information.

[0032] When detecting and tracking traffic signs in a driving video frame sequence, a traffic sign detection frame of each traffic sign can be marked on the driving video frame including the traffic sign, and driving track information corresponding to each traffic sign can be obtained. It is understandable that traffic sign detection frames confirmed as corresponding to the same traffic sign by tracking the traffic sign in different video frames have the same detection frame identifier.

[0033] In this embodiment, a neural network detection algorithm can be used to detect traffic signs. For ease of understanding, the traffic sign is taken as an example. When annotating data in the neural network training stage, the annotation box is as close to the boundary of the traffic sign as possible, and two traffic signs of different shapes should be labeled separately. Regardless of whether the traffic sign is rectangular, circular, triangular or other polygonal shapes, the detection result only gives the most fitting rectangular enclosing box to mark the traffic sign from the driving video frame.

[0034] In order to facilitate the tracking of the same traffic sign in a driving video frame sequence, it can be achieved through a filtering tracking method, or other tracking methods can be used, which are not limited here; in the process of using the tracking method to track the trajectory of the same traffic sign, the positioning information collected by the positioning module is required to complete the tracking. The tracking process of this application is the same as the existing tracking process, and will not be repeated here.

[0035] The above driving trajectory information can be represented by the frame identifier of the driving video frame and the coordinate information of the traffic sign detection frame. For example, there are 5 video frames containing the traffic sign detection frame of the traffic sign 1 in the above driving video frame sequence, and the frame identifiers of the corresponding driving video frames are 1, 2, 3, 4, and 5. In order to accurately find the position of each of the above traffic sign detection frames in the video frame, in this embodiment, the position of the center point of the traffic sign detection frame can be selected as the position information of the traffic sign.

[0036] Step S106: Take the traffic signs corresponding to each piece of driving trajectory information as first objects, and perform the operations of steps S108 to S116 for each first object:

[0037] Step S108, calculating a first map position of the first object in the high-precision map based on the driving video frame subsequence;

[0038] The first map position in the present application refers to the location information of the traffic sign in the high-precision map. Since the process of acquiring video by the vehicle-mounted camera is a process from a 3D world to a 2D image, the calculation process of the first map position in the present application is a process of restoring from a 2D image to 3D.

[0039] Step S110, adjusting the first map position to obtain a second map position;

[0040] Since the positioning accuracy of the vehicle-mounted positioning module of the present application is low, the transformation from the high-precision map coordinate system to the vehicle coordinate system provided is not very accurate, so the first map position calculated above is relatively rough. Therefore, in order to accurately determine the position information of the traffic sign in the high-precision map, it is necessary to accurately adjust the first map position to obtain the precise position of the traffic sign in the high-precision map.

[0041] Step S112, calculating an average projection error of the center point of the first object based on the second map position;

[0042] The center point is the center point of the traffic sign detection frame corresponding to the first object mentioned above. In this embodiment, the average projection error is an important judgment standard indicator for entering the next step of high-precision map update. In actual use, other judgment standard indicators can also be calculated as needed, which are not limited here.

[0043] Step S114, when the average projection error is less than a preset projection error threshold and the number of frames in the driving video frame subsequence is greater than a preset number of video frames, searching for a third map position of the first object in the high-precision map based on the second map position;

[0044] Only when the average projection error is less than the preset projection error threshold and the number of frames in the driving video frame subsequence is greater than the preset number of video frames, does it indicate that the second map position of the first object has been reconstructed successfully, and the update step of the traffic sign in the high-precision map can be entered, that is, the search process for the third map position can be executed. If the average projection error is not less than the preset projection error threshold and / or the number of frames in the driving video frame subsequence is not greater than the preset number of video frames, it indicates that the reconstruction of the second map position of the first object has failed, and the update step of the traffic sign in the high-precision map cannot be entered.

[0045] The preset projection error threshold and the preset number of video frames can be set according to actual needs and are not limited here.

[0046] Step S116, if the distance between the third map location and the second map location is greater than a preset distance threshold, generate update information and send it to the map provider of the high-precision map, so that the map provider updates the location of the first object in the high-precision map according to the second map location carried in the update information.

[0047] In actual use, if the distance between the third map location and the second map location is not greater than the preset distance threshold, it means that the traffic sign of the high-precision map at this location does not need to be updated. If no traffic sign is found at this location, or the distance between the third map location and the second map location is greater than the preset distance threshold, an update message reporting a problem with the traffic sign at that location on the high-precision map can be generated and sent to the map provider, so that the map provider can perform a series of update operations such as adding, changing the location or changing the attributes of the traffic sign at that location on the high-precision map according to the second map location carried in the update message, or use a more accurate map collection and production vehicle at this location to perform supplementary operations to update the traffic sign in the high-precision map.

[0048] The method for updating traffic signs in high-precision maps provided in the embodiments of the present application does not rely on or reduces the frequency of use of laser radars of surveying and mapping equipment and scanning equipment, and realizes the detection and tracking of traffic signs in large-scale high-precision maps based on the hardware of general intelligent vehicles, and calculates the second map position of the traffic sign on the high-precision map based on the detected and tracked driving trajectory information, and compares the second map position with the third map position corresponding to the position found on the existing high-precision map. Through the established strategy, it is determined whether the high-precision map needs to be updated or supplemented, which ultimately ensures the immediacy of updating traffic signs in the high-precision map and greatly saves labor costs.

[0049] This embodiment provides another method for updating traffic signs in a high-precision map, which is implemented on the basis of the above embodiment; this embodiment focuses on describing the specific implementation method of calculating the first map position. Figure 2 Another method for updating traffic signs in a high-precision map is shown as a flow chart. The method for updating traffic signs in a high-precision map in this embodiment includes the following steps:

[0050] Step S202, obtaining a driving video frame sequence captured by a vehicle-mounted camera during the driving process of the vehicle;

[0051] Step S204, based on the positioning information collected by the vehicle positioning module, traffic sign detection and tracking are performed on the driving video frame sequence to obtain driving video frames marked with traffic sign detection frames and driving trajectory information of the same traffic sign; wherein the driving trajectory information is a driving video frame subsequence containing the same detection frame identifier;

[0052] Step S206: Take the traffic signs corresponding to each piece of driving trajectory information as first objects, and perform the operations of steps S208 to S220 for each first object:

[0053] Step S208, calculating the central pixel of the central point of the first object based on the driving video frame subsequence;

[0054] In this embodiment, the center position information of the traffic sign detection frame included in each video frame in the driving video frame subsequence is first determined, and then the average of multiple center position information is calculated to obtain the center pixel.

[0055] Step S210, calculating a camera projection matrix of the vehicle-mounted camera based on the center pixel and the camera parameters of the vehicle-mounted camera;

[0056] The camera projection matrix is ​​calculated by the following formula:

[0057] Among them, the vector x represents the normalized coordinates of the 2D pixel, the coordinates of the center pixels u and v of the center point of the traffic sign are determined by the above step S208, λ is the scaling factor, K is a 3*3 matrix, representing the internal parameters of the camera parameters, which can be obtained in advance according to the vehicle camera manufacturer or related calibration algorithms, and [R|t] is a 3*4 pose transformation matrix, representing the rotation R from the high-precision map coordinate system to the camera coordinate system C. CW and displacement t CW , [R|t] can be obtained by the following formula:

[0058] Among them, V coordinate system represents the vehicle coordinate system, R CW and t Cw represents the transformation from the HD map coordinate system to the vehicle coordinate system, which can be obtained through the vehicle positioning module in the vehicle, and R CW and t CW It represents the transformation from the vehicle coordinate system to the camera coordinate system, which is determined by the installation position of the vehicle camera, or can be provided by the vehicle manufacturer, or obtained through the external parameter calibration algorithm of the vehicle camera. Therefore, the projection matrix P = K[R|t] becomes known at this time and can be obtained.

[0059] Step S212, calculating a first map position based on the camera projection matrix and the center position information of the traffic sign detection frame included in each video frame in the driving video frame subsequence;

[0060] On the locomotive of the projection equation, the first map position can be calculated by cross transformation, that is, the first map position can be calculated by the following formula: Among them, [u N , v N ] represents the center position information of the traffic sign detection frame included in the Nth video frame in the driving video frame subsequence, P N (1), P N (2), P N (3) are the first row matrix value, the second row matrix value and the third row matrix value of the Nth camera projection matrix corresponding to the first object, respectively. i Represents the i-th first object.

[0061] Step S214, adjusting the first map position to obtain a second map position;

[0062] The process of step S214 can be implemented by steps A1 to A2:

[0063] Step A1, constructing an optimization function based on the center position information of the traffic sign detection frame included in each video frame in the driving video frame subsequence corresponding to the first object, the first map position corresponding to the first object, and the camera projection matrix;

[0064] If there are M different traffic signs observed by N video frames, the constructed optimization function f is: Among them, h(x) is the robust kernel function, u ij represents the pixel [u, v] of the center point of the i-th first object on the j-th video frame, π(P j , X i ) represents the camera projection matrix P j The first two pixels are taken out under the transformation.

[0065] Robust Kernel Function δ is a preset value. The robust kernel function is added to the optimization function to prevent the influence of some abnormal values ​​on the function when optimizing the optimization function.

[0066] Step A2, adjusting the first map position and the camera projection matrix until the optimization function reaches a preset function value, to obtain an adjusted second map position.

[0067] In this embodiment, the preset function value is the minimum value of f. The process of minimizing f by adjusting the P and X values ​​is called bundle adjustment. The adjusted X value is the second map position.

[0068] Step S216, calculating an average projection error of the center point of the first object based on the second map position;

[0069] In this embodiment, the average projection error can be calculated by the following formula: Where N represents the number of video frames in the driving video frame subsequence, X i represents the i-th first object, P j represents the camera projection matrix corresponding to the first object, u ij represents the pixel [u, v] of the center point of the i-th first object on the j-th video frame, π(P j , X i ) represents the camera projection matrix P j The first two pixels are taken out under the transformation.

[0070] Step S218, when the average projection error is less than a preset projection error threshold and the number of frames in the driving video frame subsequence is greater than a preset number of video frames, searching for a third map position of the first object in the high-precision map based on the second map position;

[0071] The process of searching for the third map location is as follows: searching in the high-precision map for identification information within a radius with the second map location as the center and a preset search distance; searching for target identification information that matches the object identification information of the first object from multiple identification information; and determining the map location corresponding to the target identification information as the third map location.

[0072] In order to facilitate the distinction of traffic signs, each traffic sign is assigned identification information with unique identification function. In this embodiment, the identification information of the same traffic sign on the high-precision map is the same as the identification information of the traffic sign in the video frame captured by the vehicle-mounted camera, where the identification information can be represented by numbers, words, etc., which is not limited here.

[0073] In specific implementation, if there is target identification information that is the same as the identification information of the first object among the identification information found within the search range, the map position corresponding to the target identification information can be determined as the third map position; if no identification information that is the same as the identification information of the first object is found, it indicates that the first object does not exist in the area.

[0074] Step S220, if the distance between the third map location and the second map location is greater than a preset distance threshold, generate update information and send it to the map provider of the high-precision map, so that the map provider updates the location of the first object in the high-precision map according to the second map location carried in the update information.

[0075] When the distance between the third map location and the second map location is greater than a preset distance threshold, or when the first object does not exist, update information needs to be generated and sent to the map provider so that the map provider can update the first object in the high-precision map.

[0076] The embodiment of the present invention also provides a device for updating traffic signs in a high-precision map, wherein the device is applied to a vehicle-mounted controller, and the vehicle-mounted controller is also communicatively connected with a vehicle-mounted camera and a vehicle-mounted positioning module. Figure 3 A schematic diagram of the structure of a device for updating traffic signs in a high-precision map is shown, the device comprising:

[0077] The acquisition module 302 is used to acquire a driving video frame sequence captured by the vehicle-mounted camera during the driving process of the vehicle;

[0078] The detection and tracking module 304 is used to detect and track traffic signs on the driving video frame sequence based on the positioning information collected by the vehicle positioning module, and obtain driving video frames marked with traffic sign detection frames and driving trajectory information of the same traffic sign; wherein the driving trajectory information is a subsequence of driving video frames containing the same detection frame identifier;

[0079] The execution module 306 is used to take the traffic sign corresponding to each piece of driving trajectory information as a first object, and perform the following operations on each first object:

[0080] A first calculation module 308, configured to calculate a first map position of a first object in a high-precision map based on a driving video frame subsequence;

[0081] A position adjustment module 310, configured to adjust the first map position to obtain a second map position;

[0082] A second calculation module 312, configured to calculate an average projection error of a center point of the first object based on the second map position;

[0083] A search module 314 is configured to search for a third map position of the first object in the high-precision map based on the second map position when the average projection error is less than a preset projection error threshold and the number of frames in the driving video frame subsequence is greater than a preset number of video frames;

[0084] The generation module 316 is used to generate update information and send it to the map provider of the high-precision map if the distance between the third map location and the second map location is greater than a preset distance threshold, so that the map provider updates the position of the first object in the high-precision map according to the second map location carried in the update information.

[0085] An embodiment of the present application provides a device for updating traffic signs in a high-precision map, wherein a vehicle-mounted camera can be used to detect and track traffic signs in a collected driving video frame sequence, and driving video frames marked with traffic sign detection frames and driving trajectory information of the same traffic sign are obtained. The traffic signs corresponding to each driving trajectory information are respectively used as first objects, and a first map position of the first object in the high-precision map is calculated based on a driving video frame subsequence; the first map position is adjusted to obtain a second map position; the average projection error of the center point of the first object is calculated based on the second map position; when the average projection error is less than a preset projection error threshold and the number of frames in the driving video frame subsequence is greater than a preset number of video frames, the third map position of the first object in the high-precision map is searched based on the second map position; if the distance between the third map position and the second map position is greater than the preset distance threshold, update information is generated and sent to a map supplier of the high-precision map, so that the map supplier updates the position of the first object in the high-precision map according to the second map position carried in the update information. The vehicle-mounted controller of the present application can generate update information of traffic signs based on the driving video frame sequence collected by the vehicle-mounted camera, and send the update information to the map provider so that the map provider can update the traffic signs in a timely manner. In this process, the map provider does not need to perform on-site scanning of traffic signs in real time, which greatly reduces the consumption of manpower and resources.

[0086] The device for updating traffic signs in a high-precision map provided in an embodiment of the present invention has the same technical features as the method for updating traffic signs in a high-precision map provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0087] The present application also provides an electronic device, such as Figure 4 As shown, it is a schematic diagram of the structure of the electronic device, wherein the electronic device includes a processor 121 and a memory 120, the memory 120 stores computer executable instructions that can be executed by the processor 121, and the processor 121 executes the computer executable instructions to implement the method for updating traffic signs in the above-mentioned high-precision map.

[0088] exist Figure 4 In the illustrated embodiment, the electronic device further includes a bus 122 and a communication interface 123 , wherein the processor 121 , the communication interface 123 and the memory 120 are connected via the bus 122 .

[0089] Among them, the memory 120 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 123 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 122 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 122 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0090] The processor 121 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 121. The above processor 121 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor 121 reads the information in the memory and combines its hardware to complete the steps of the method for updating traffic signs in the high-precision map of the aforementioned embodiment.

[0091] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method for updating traffic signs in the above-mentioned high-precision map. The specific implementation can be found in the aforementioned method embodiment, which will not be repeated here.

[0092] The computer program product of the method, device and electronic device for updating traffic signs in a high-precision map provided in the embodiments of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0093] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0094] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0095] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", and "secondary" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0096] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for updating traffic signs in a high-precision map, characterized in that: The method is applied to a vehicle-mounted controller, which is also in communication connection with a vehicle-mounted camera and a vehicle-mounted positioning module, and the method includes: Acquire a driving video frame sequence captured by the vehicle-mounted camera during the driving of the vehicle; Based on the positioning information collected by the vehicle-mounted positioning module, the driving video frame sequence is subjected to traffic sign detection and tracking to obtain driving video frames marked with traffic sign detection frames and driving trajectory information of the same traffic sign; wherein the driving trajectory information is a driving video frame subsequence containing the same detection frame identifier; The traffic signs corresponding to each piece of driving trajectory information are respectively taken as first objects, and the following operations are performed for each of the first objects: Calculating a first map position of the first object in the high-precision map based on the driving video frame subsequence; Adjusting the first map position to obtain a second map position; calculating an average projection error of a center point of the first object based on the second map position; When the average projection error is less than a preset projection error threshold and the number of frames of the driving video frame subsequence is greater than a preset number of video frames, searching for a third map position of the first object in the high-precision map based on the second map position; If the distance between the third map position and the second map position is greater than a preset distance threshold, generating update information and sending it to a map provider of the high-precision map, so that the map provider updates the position of the first object on the high-precision map according to the second map position carried in the update information; The step of adjusting the first map position to obtain a second map position includes: constructing an optimization function based on the center position information of the traffic sign detection frame included in each video frame in the driving video frame subsequence corresponding to the first object, the first map position corresponding to the first object, and the camera projection matrix; Adjusting the first map position and the camera projection matrix until the optimization function reaches a preset function value, to obtain an adjusted second map position; The step of searching for a third map position of the first object in the high-precision map based on the second map position includes: Searching the high-precision map for identification information within a radius with a preset search distance and the second map location as the center; Searching for target identification information matching the object identification information of the first object from the plurality of identification information; The map position corresponding to the target identification information is determined as the third map position.

2. The method according to claim 1, characterized in that The step of calculating a first map position of the first object in the high-precision map based on the driving video frame subsequence comprises: Calculate a central pixel of a central point of the first object based on the driving video frame subsequence; Calculate a camera projection matrix of the vehicle-mounted camera based on the central pixel and the camera parameters of the vehicle-mounted camera; A first map position is calculated based on the camera projection matrix and the center position information of the traffic sign detection frame included in each video frame in the driving video frame subsequence.

3. The method according to claim 2, characterized in that The first map position is calculated by the following formula: Among them, [u N , v N ] represents the center position information of the traffic sign detection frame included in the Nth video frame in the driving video frame subsequence, P N (1), P N (2), P N (3) are the first row matrix value, the second row matrix value and the third row matrix value of the Nth camera projection matrix corresponding to the first object, respectively, X i Represents the i-th first object.

4. The method according to claim 2, characterized in that: The step of calculating the average projection error of the center point of the first object based on the second map position comprises: An average projection error is calculated based on the center position information of the traffic sign detection frame included in each video frame in the driving video frame subsequence corresponding to the first object, the second map position corresponding to the first object, and the camera projection matrix.

5. The method according to claim 4, characterized in that The average projection error is calculated by the following formula: Wherein, N represents the number of video frames in the driving video frame subsequence, X i represents the i-th first object, P i represents the camera projection matrix corresponding to the first object, u ij represents the pixel [u,v] of the center point of the i-th first object on the j-th video frame, π(P j , X i ) represents the camera projection matrix P j The first two pixels are taken out under the transformation.

6. A device for updating traffic signs in a high-precision map, characterized in that: The device is applied to a vehicle-mounted controller, and the vehicle-mounted controller is also communicatively connected with a vehicle-mounted camera and a vehicle-mounted positioning module. The device includes: An acquisition module is used to acquire a driving video frame sequence captured by the vehicle-mounted camera during the driving process of the vehicle; A detection and tracking module, configured to detect and track traffic signs on the driving video frame sequence based on the positioning information collected by the vehicle-mounted positioning module, and obtain driving video frames marked with traffic sign detection frames and driving trajectory information of the same traffic sign; wherein the driving trajectory information is a subsequence of driving video frames containing the same detection frame identifier; An execution module is used to take the traffic sign corresponding to each piece of the driving trajectory information as a first object, and perform the following operations on each of the first objects: A first calculation module, configured to calculate a first map position of the first object in the high-precision map based on the driving video frame subsequence; A position adjustment module, used for adjusting the first map position to obtain a second map position; a second calculation module, configured to calculate an average projection error of a center point of the first object based on the second map position; a search module, configured to search for a third map position of the first object in the high-precision map based on the second map position when the average projection error is less than a preset projection error threshold and the number of frames of the driving video frame subsequence is greater than a preset number of video frames; A generating module, configured to generate, if the distance between the third map position and the second map position is greater than a preset distance threshold, an update message to be sent to a map provider of the high-precision map, so that the map provider updates the position of the first object on the high-precision map according to the second map position carried in the update message; The position adjustment module is further used to construct an optimization function based on the center position information of the traffic sign detection frame included in each video frame in the driving video frame subsequence corresponding to the first object, the first map position corresponding to the first object, and the camera projection matrix; adjust the first map position and the camera projection matrix until the optimization function reaches a preset function value, and obtain an adjusted second map position; The search module is also used to search in the high-precision map for identification information within a radius with the second map position as the center and a preset search distance; search for target identification information matching the object identification information of the first object from multiple identification information; and determine the map position corresponding to the target identification information as the third map position.

7. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 5.

Images