Camera data hitching verification method and device, electronic equipment, storage medium and computer program product

Abstract

The invention provides a camera data hitching verification method and device, electronic equipment, a storage medium and a computer program product. The method comprises the steps that a three-dimensional model semantization data set of first camera precision and a service data set of second camera precision which are connected with a camera cluster in a hanging mode are obtained, the first camera precision is larger than the second camera precision, and the camera cluster comprises a plurality of single cameras; performing spatial clustering processing on the three-dimensional model semantic data set to obtain a model data spatial clustering cluster, and performing spatial clustering processing on the business data set to obtain a business data spatial clustering cluster; determining the sample number of the model data space clustering cluster and the service data space clustering cluster; and determining to perform hitching accuracy verification on the single camera or the camera cluster based on the sample number. According to the invention, the verification efficiency and accuracy of the calibration data of the camera can be improved.

Description

Technical Field

[0001] The present application relates to computer technology, and in particular to a camera data connection verification method, device, electronic device, storage medium and computer program product. Background Art

[0002] In order to improve road safety, management efficiency and respond to emergencies, a large number of physical cameras are installed on highways, including roadside cameras, gantry cameras, toll lane cameras, etc. Cameras are important equipment assets of highway companies, and the view data collected by cameras plays a key role in command and dispatch, obstacle removal and rescue, gantry vehicle detection, toll collection and other businesses.

[0003] In traditional business systems, camera asset information and its collected data management applications are mostly based on two-dimensional standard maps, which have low requirements for the accuracy of camera positioning coordinates and only need to meet cross-section management needs. When high-precision map technology is applied to the field of highway operation and management, the accuracy requirements of basic data of various types of facility assets are also significantly improved; in the process of map data collection and processing, the location and basic attributes of various types of facilities and equipment including cameras will be extracted, and the absolute accuracy of the location data can reach at least 1m, and the relative accuracy can be 15-25cm. Therefore, for the same device, the standard location coordinates stored in the business system database must be different from the location coordinates of the high-precision map.

[0004] In the process of upgrading the map from standard to high-precision, the business system needs to map the basic camera data in the business system database with the high-precision map vector data and 3D model semantic data in order to realize the highway camera management and view data application based on high-precision maps. Therefore, data attachment is a necessary data processing link. Its technical idea is to match data based on the similarity of positioning coordinates. The quality of data attachment directly affects the business application based on high-precision maps. However, there is still a lack of batch data quality inspection methods, and the efficiency and accuracy of quality inspection cannot be guaranteed. Summary of the invention

[0005] The embodiments of the present application provide a camera data mounting verification method, device, electronic device and computer-readable storage medium, which can improve the efficiency and accuracy of camera calibration data verification.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] The present application embodiment provides a camera data attachment verification method, including:

[0008] Obtaining a 3D model semantic data set of a first camera precision and a business data set of a second camera precision connected to a camera cluster, wherein the first camera precision is greater than the second camera precision, and the camera cluster includes a plurality of single cameras;

[0009] Performing spatial clustering processing on the semantic data set of the three-dimensional model to obtain spatial clustering clusters of model data, and performing spatial clustering processing on the business data set to obtain spatial clustering clusters of business data;

[0010] Determining the number of samples of the model data space clustering cluster and the business data space clustering cluster;

[0011] Based on the number of samples, the connection accuracy verification of a single camera or a camera cluster is determined.

[0012] In the above solution, the step of obtaining a semantic data set of a three-dimensional model with a first camera precision and a business data set with a second camera precision connected to the camera cluster includes:

[0013] Obtain a camera high-precision map dataset, a business dataset and a first mapping relationship connected to the camera cluster, wherein the first mapping relationship is a mapping relationship between the camera high-precision map dataset and the business dataset, and the camera high-precision map dataset includes a high-precision map vector dataset and a three-dimensional model semantic dataset;

[0014] The high-precision map vector dataset, the three-dimensional model semantic dataset and the business dataset are subjected to coordinate unification processing to obtain a high-precision map vector dataset, the three-dimensional model semantic dataset and the business dataset with unified coordinates.

[0015] In the above scheme, the method further includes:

[0016] Perform consistency check on the longitude and latitude of the high-precision map vector dataset and the three-dimensional model semantic dataset, and determine model data from the three-dimensional model semantic dataset that is inconsistent with the longitude and latitude of the high-precision map vector dataset;

[0017] The longitude and latitude of the model data are adjusted to be consistent with the corresponding vector data to obtain a three-dimensional model semantic data set that is consistent with the longitude and latitude of the high-precision map vector data set.

[0018] In the above solution, the verification of the accuracy of mounting a single camera or a camera cluster based on the number of samples includes:

[0019] If the number of samples of the business data space cluster is 1, it is determined to verify the accuracy of the mounting of the single camera;

[0020] If the number of samples of the business data space cluster is not 1, it is determined to verify the connection accuracy of the camera cluster.

[0021] In the above solution, if it is determined to verify the accuracy of the mounting of the single camera, then the verification of the accuracy of the mounting of the single camera includes:

[0022] Extract the spatial clustering of any single camera business data;

[0023] Searching within the search region whether there is a model data space clustering cluster set whose cluster attribute is a target value, wherein the model data space clustering cluster set includes one or more model data space clustering clusters;

[0024] If so, associating the single camera business data space clustering cluster with the model data space clustering cluster set;

[0025] Extracting the first camera number in the spatial cluster of the single camera service data;

[0026] Determine whether there is a model number corresponding to the first camera number in the first mapping relationship;

[0027] If so, extract the second camera number of the model data space cluster set associated with the single camera business data space cluster cluster;

[0028] If the first camera number is consistent with the second camera number, it is determined that the data connection of the corresponding camera is normal.

[0029] In the above scheme, if it is determined to verify the connection accuracy of the camera cluster, then the verification of the connection accuracy of the camera cluster includes:

[0030] Extracting camera numbers from the business data space clusters one by one to obtain multiple third camera numbers;

[0031] Extracting image sample data and camera parameters of cameras corresponding to a plurality of third camera numbers;

[0032] Extracting corresponding model position parameters from the three-dimensional model semantic data set according to the model number corresponding to the third camera number;

[0033] Based on the map engine, according to the camera parameters, determine the camera model perspective image;

[0034] Calculating the similarity between the image sample data and the camera model perspective image;

[0035] If the similarity reaches the similarity threshold, it is determined that the data connection of the camera cluster is normal.

[0036] The present application embodiment provides a camera data attachment verification device, including:

[0037] An acquisition module, used to obtain a semantic data set of a three-dimensional model with a first camera precision and a business data set with a second camera precision connected to a camera cluster, wherein the first camera precision is greater than the second camera precision, and the camera cluster includes a plurality of single cameras;

[0038] A clustering module, used to perform spatial clustering processing on the semantic data set of the three-dimensional model to obtain spatial clustering clusters of model data, and to perform spatial clustering processing on the business data set to obtain spatial clustering clusters of business data;

[0039] A determination module, used to determine the number of samples of the model data space clustering cluster and the business data space clustering cluster;

[0040] A verification module is used to determine the connection accuracy verification of a single camera or a camera cluster based on the number of samples.

[0041] An embodiment of the present application provides an electronic device, including:

[0042] A memory for storing executable instructions;

[0043] The processor is used to implement the camera data mounting verification method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0044] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the camera data attachment verification method provided in the embodiment of the present application.

[0045] An embodiment of the present application provides a computer program product storing a computer program for implementing the camera data attachment verification method provided in the embodiment of the present application when executed by a processor.

[0046] The embodiment of the present application obtains a three-dimensional model semantic data set of the first camera precision and a business data set of the second camera precision connected to the camera cluster, performs spatial clustering processing on the three-dimensional model semantic data set to obtain a model data spatial clustering cluster, and performs spatial clustering processing on the business data set to obtain a business data spatial clustering cluster; determines the number of samples of the model data spatial clustering cluster and the business data spatial clustering cluster; determines the verification of the connection accuracy of a single camera or a camera cluster based on the number of samples, thereby improving the efficiency and accuracy of camera calibration data verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1It is an optional structural diagram of the camera data hookup verification system 100 provided in an embodiment of the present application;

[0048] Figure 2 is an optional structural diagram of an electronic device 200 provided in an embodiment of the present application;

[0049] Figure 3 It is an optional flow chart of the camera data hookup verification method provided in the embodiment of the present application;

[0050] Figure 4 This is an optional flow chart of verifying the accuracy of mounting a single camera provided in an embodiment of the present application;

[0051] Figure 5 It is an optional flow chart of verifying the accuracy of camera cluster attachment provided by an embodiment of the present application;

[0052] Figure 6 It is an optional flowchart of abnormal data processing provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0054] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0055] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0057] The embodiments of the present application provide a camera data mounting verification method, device, electronic device and computer-readable storage medium, which can improve the efficiency and accuracy of camera calibration data verification.

[0058] First, the camera data connection verification system provided in the embodiment of the present application is described. Figure 1 , Figure 1 It is an optional structural diagram of the verification system 100 for camera data attachment provided in the embodiment of the present application. To realize the verification application supporting a camera data attachment, the electronic device 101 is connected to the business system database 103 through the network 102. Here, the electronic device 101 can be a terminal or a server. The business system database 103 can be a server. In some embodiments, the terminal can be a laptop, a tablet computer, a desktop computer, a smart phone, a dedicated messaging device, a portable gaming device, a smart speaker, a smart watch, etc., but is not limited thereto. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network) services, and big data and artificial intelligence platforms. The network 102 can be a wide area network or a local area network, or a combination of the two. The electronic device 101 and the business system database 103 can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiment of the present application.

[0059] Next, the electronic device for implementing the above-mentioned camera data hookup verification method provided in the embodiment of the present application is described. Figure 2 , Figure 2 is an optional structural diagram of an electronic device 200 provided in an embodiment of the present application. In practical applications, the electronic device 200 can be implemented as Figure 1 The terminal 103 or server 101 in the embodiment of the present invention is an electronic device. Figure 1 Taking the terminal 103 shown as an example, an electronic device for implementing the camera data attachment verification method of the embodiment of the present application is described.

[0060] Figure 2The electronic device 200 shown includes: at least one processor 201, a memory 205, at least one network interface 202 and a user interface 203. The various components in the electronic device 200 are coupled together via a bus system 204. It is understood that the bus system 204 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 204 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 204 is not described in detail. Figure 2 Various buses are labeled as bus system 204 .

[0061] The processor 201 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0062] The user interface 203 includes one or more output devices 2031 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 203 also includes one or more input devices 2032, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0063] The memory 205 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 205 may optionally include one or more storage devices that are physically remote from the processor 201.

[0064] The memory 205 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 205 described in the embodiment of the present application is intended to include any suitable type of memory.

[0065] In some embodiments, the memory 205 can store data to support various operations. Examples of these data include programs, modules, and data structures or subsets or supersets thereof. In the embodiment of the present application, the memory 205 stores an operating system 2051, a network communication module 2052, a presentation module 2053, an input processing module 2054, and a verification device 2055 for camera data attachment; specifically,

[0066] Operating system 2051, including system programs for processing various basic system services and performing hardware-related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0067] A network communication module 2052, for reaching other computing devices via one or more (wired or wireless) network interfaces 202, exemplary network interfaces 202 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB);

[0068] a presentation module 2053 for enabling presentation of information (e.g., a user interface for operating peripherals and displaying content and information) via one or more output devices 2031 (e.g., a display screen, a speaker, etc.) associated with the user interface 203;

[0069] The input processing module 2054 is used to detect one or more user inputs or interactions from one of the one or more input devices 2032 and translate the detected inputs or interactions.

[0070] In some embodiments, the camera data connection verification device provided in the embodiments of the present application can be implemented in software. Figure 2 The verification device 2055 of the camera data stored in the memory 205 is shown, which can be software in the form of a program and a plug-in, including the following software modules: an acquisition module 20551, a clustering module 20552, a determination module 20553 and a verification module 20554. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0071] In other embodiments, the camera data mounting verification device provided in the embodiment of the present application can be implemented in hardware. As an example, the camera data mounting verification device provided in the embodiment of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the camera data mounting verification method provided in the embodiment of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (ASIC, Application Specific Integrated Circuit), DSP, programmable logic device (PLD, Programmable Logic Device), complex programmable logic device (CPLD, Complex Programmable Logic Device), field programmable gate array (FPGA, Field-Programmable Gate Array) or other electronic components.

[0072] The following will describe the camera data attachment verification method provided in the embodiment of the present application in combination with the exemplary application and implementation of the terminal provided in the embodiment of the present application.

[0073] See also Figure 3 , Figure 3 This is an optional flow chart of the camera data hookup verification method provided in the embodiment of the present application, which will be combined with Figure 3 The steps shown are explained.

[0074] Step 301, obtaining a 3D model semantic data set of a first camera precision and a business data set of a second camera precision connected to a camera cluster, wherein the first camera precision is greater than the second camera precision, and the camera cluster includes a plurality of single cameras;

[0075] Step 302, performing spatial clustering processing on the semantic data set of the three-dimensional model to obtain spatial clustering clusters of model data, and performing spatial clustering processing on the business data set to obtain spatial clustering clusters of business data;

[0076] Step 303, determining the number of samples of the model data space clustering cluster and the business data space clustering cluster;

[0077] Step 304 , based on the number of samples, determining whether to verify the connection accuracy of a single camera or a camera cluster.

[0078] In actual implementation, obtain the camera high-precision map data set M, the business data set S, and the first mapping relationship between M and S; wherein M is the road network high-precision map camera data set, including the high-precision map vector data set VD and the three-dimensional model semantic data set MD; wherein VD is composed of multiple vector data, and its data attributes include the map element number V_ID, vector longitude, vector latitude, and direction; MD is composed of multiple three-dimensional model semantic data, and its data attributes include model number ID, corresponding map element number, model longitude, model latitude, model elevation, relative height, and direction; wherein the model elevation includes the camera model elevation and the road surface model elevation of the corresponding point, and the relative height is calculated by the difference between the two; S is the camera space data set in the business system database, and its data attributes include camera number N, standard high-precision map longitude, standard high-precision map latitude, direction, and stake number; the mapping relationship table collects the corresponding relationship between camera number N and model number ID; in the embodiment, the amount of MD data is greater than S, and for the data in M ​​that have a corresponding relationship between model number ID and camera number N, it means that the model has completed the business data attachment.

[0079] In actual implementation, the camera high-precision map data set comes from the high-precision map production database, the business data set comes from business systems or data middle platforms such as the video middle platform, charging system, and electromechanical operation and maintenance system, and the mapping relationship table comes from the high-precision map service middle platform or the system database that calls the high-precision map service.

[0080] In actual implementation, the business data set may have missing or significantly abnormal longitude and latitude of the precision map. The stake number (including kilometer-level and hundred-meter-level stake numbers) can be retrieved and converted into longitude and latitude coordinates through the map engine's stake number coordinate conversion tool.

[0081] In actual implementation, the location of high-precision map vector data and three-dimensional model semantic data usually includes longitude, latitude, and elevation. When extracting the data set, only longitude and latitude are extracted.

[0082] Next, according to the camera number in S, the image sample data of the corresponding camera is extracted from the business system database; then, according to the camera number in S, the camera focal length, principal point, and distortion coefficient are extracted from the business system database. Finally, the high-precision road vector data is obtained.

[0083] In some embodiments, the method of obtaining a three-dimensional model semantic dataset of the first camera precision and a business dataset of the second camera precision connected to the camera cluster includes: obtaining a high-precision map dataset, a business dataset and a first mapping relationship of the camera connected to the camera cluster, the first mapping relationship being a mapping relationship between the camera high-precision map dataset and the business dataset, the camera high-precision map dataset including a high-precision map vector dataset and a three-dimensional model semantic dataset; performing coordinate unification processing on the high-precision map vector dataset, the three-dimensional model semantic dataset and the business dataset to obtain a high-precision map vector dataset, the three-dimensional model semantic dataset and the business dataset with unified coordinates.

[0084] In actual implementation, the geographic coordinate systems of VD, MD, and S are unified. Usually, the geographic coordinate systems include WGS84, CGC2000, and GCJ02. The coordinate systems of VD and S are transformed based on the coordinate system in the semantic data of the three-dimensional model. The transformed vector longitude, vector latitude, and precision map longitude and latitude are updated to the VD and S data sets.

[0085] In some embodiments, the method also includes: performing consistency checking on the longitude and latitude of the high-precision map vector dataset and the three-dimensional model semantic dataset, determining model data from the three-dimensional model semantic dataset that is inconsistent with the longitude and latitude of the high-precision map vector dataset; adjusting the longitude and latitude of the model data to be consistent with the corresponding vector data to obtain a three-dimensional model semantic dataset that is consistent with the longitude and latitude of the high-precision map vector dataset.

[0086] In actual implementation, according to the "corresponding map element number" field in MD, the coordinate attributes of the high-precision map vector data with the corresponding number after unified processing are extracted from VD, and the consistency of the longitude and latitude of the vector and the model is checked; if they are consistent, enter S5; otherwise, the longitude and latitude of the model in MD are modified to make them consistent with the longitude and latitude of the vector.

[0087] In the embodiment of the present application, the data is also cleaned. Specifically, high-precision vector data of the road is extracted, and the detection area is demarcated by expanding 50m outward from the edge of the road. The data position is detected according to the longitude and latitude coordinates of the precision map of S, and the points beyond the detection interval are marked as deviation points; the pile number corresponding to the deviation point is extracted, and the pile number is converted into geographic coordinates through the pile number standard conversion tool of the map engine, and the position detection is performed again; if it is within the detection area, the longitude and latitude corresponding to the converted geographic coordinates are updated to S; if it exceeds the detection area, the deviation point is deleted from S; in the embodiment, the deleted deviation point needs to be manually verified for the actual installation position of its equipment. In addition, the identification of unmounted data is also performed, and the data without the corresponding model number ID for the camera number N in the mapping relationship table is marked as abnormal.

[0088] In some embodiments, the determination of connection accuracy verification for a single camera or a camera cluster based on the sample number includes: if the sample number of the business data space clustering cluster is 1, determining to verify the connection accuracy for the single camera; if the sample number of the business data space clustering cluster is not 1, determining to verify the connection accuracy for the camera cluster.

[0089] In actual implementation, first, the neighborhood radius Eps and the minimum density MinPts are set, and DBSCAN (density-based clustering method with noise) is used to perform density-based spatial clustering on MD and S, respectively, to form a model data spatial clustering cluster set M_Clu and a business data spatial clustering cluster set S_Clu, respectively, where M_Clu and S_Clu are composed of model data spatial clustering clusters and business data spatial clustering clusters, respectively; wherein the neighborhood radius Eps is set according to the width of the highway gantry or the installation spacing of the roadside monitoring equipment, and is generally set to 20m in the embodiment; the minimum density is set to 1, that is, there is at least one device in the spatial clustering cluster;

[0090] Then, the number of samples of each model data space cluster and business data space cluster is counted. If the number of samples is 1, the cluster attribute φ is defined as 1, indicating a single camera, otherwise φ = 0, indicating a camera cluster;

[0091] Furthermore, for any business data spatial clustering cluster, a search area with a matching radius R and a cluster center point as the center is constructed; in the embodiment, the matching radius R is generally set to 50m.

[0092] In actual implementation, if φ of the business data space clustering cluster is 1, the accuracy of the single camera mounting is verified; otherwise, the accuracy of the camera cluster mounting is verified.

[0093] In some embodiments, if it is determined to verify the accuracy of the attachment of a single camera, the verification of the accuracy of the attachment of the single camera includes: extracting any single camera business data space clustering cluster; searching in the retrieval area whether there is a model data space clustering cluster set whose cluster attribute is the target value, the model data space clustering cluster set including one or more model data space clustering clusters; if so, associating the single camera business data space clustering cluster with the model data space clustering cluster set; extracting the first camera number in the single camera business data space clustering cluster; determining whether there is a model number corresponding to the first camera number in the first mapping relationship; if so, extracting the second camera number of the model data space clustering cluster set associated with the single camera business data space clustering cluster; if the first camera number is consistent with the second camera number, determining that the data attachment of the corresponding camera is normal.

[0094] See also Figure 4 , Figure 4 It is an optional flow chart for verifying the accuracy of mounting a single camera provided in an embodiment of the present application.

[0095] Step 401 , search in the search area of ​​the business data space cluster cluster whether there is a model data space cluster cluster set with cluster attribute φ=1, if so, execute standard 401 .

[0096] In actual implementation, when verifying the accuracy of the single camera connection, extract the spatial clustering cluster sc of any single camera business data j , find out whether there is a model data space clustering cluster set mc with cluster attribute φ=1 in its search area j , which consists of one or more model data space clusters mc i i and j are the serial numbers of the two clusters respectively; if they do not exist, the camera cluster connection verification is performed.

[0097] Step 402: Building an association relationship.

[0098] In actual implementation, if there is a cluster set of model data space clusters with cluster attribute φ = 1 in the retrieval area, sc j With mc j Associate and proceed to step 403.

[0099] Step 403 , determine whether there is a corresponding model number in the mapping relationship table, if so, execute step 404 .

[0100] In actual implementation, extracting sc j Camera number N in j , check whether there is a corresponding model number ID in the mapping relationship tablej ; If it exists, execute step 404.

[0101] Step 404: determine whether the associated model space clusters have a consistent model number.

[0102] In actual implementation, extraction and sc j Associated mc j All clusters in mc i Camera number N_mc i , determine whether N j With N_mc i If it exists, set the camera number to N j The data is marked as normal; otherwise, it is marked as abnormal; the abnormal number is processed.

[0103] In some embodiments, if it is determined to verify the accuracy of the camera cluster, the verification of the accuracy of the camera cluster includes: extracting camera numbers from the business data space clustering cluster one by one to obtain multiple third camera numbers; extracting image sample data and camera parameters of the cameras corresponding to the multiple third camera numbers; extracting corresponding model position parameters from the three-dimensional model semantic data set according to the model number corresponding to the third camera number; based on the map engine, determining the camera model perspective image according to the camera parameters; calculating the similarity between the image sample data and the camera model perspective image; if the similarity reaches the similarity threshold, it is determined that the data mounting of the camera cluster is normal.

[0104] See also Figure 5 , Figure 5 It is an optional flow chart for verifying the accuracy of camera cluster attachment provided in an embodiment of the present application.

[0105] Step 501: extract the camera numbers in the business data space cluster.

[0106] In actual implementation, the business data space cluster SC of φ=1 is u Process them one by one, u is the spatial cluster number of the camera cluster, and extract all the camera numbers N_SC (u.k) , where k is the model number in the cluster, and its value is [1, K], K is the number of models in the cluster, and its value is not less than 1; for any k value.

[0107] Step 502: Check whether the corresponding model number already exists in the mapping relationship table.

[0108] In actual implementation, whether there is a mapping relationship table with N_SC (u.k) Corresponding model number ID (u.k); If it exists, go to step 2; otherwise, mark it as abnormal and perform abnormal data processing;

[0109] Step 503: extract the sample image P1 and camera parameters.

[0110] In actual implementation, extract N_SC (u.k) The image sample data P1, as well as the camera focal length, principal point, and distortion coefficient of the camera with the same number;

[0111] Step 504: determine the field of view of the camera model.

[0112] In actual implementation, according to ID (u.k) Extract the corresponding model longitude, model latitude, model relative height, and direction from the MD data set; based on the perspective projection function of the map engine, obtain the camera's field of view height, field of view width, and field of view depth according to the camera's focal length, principal point, and distortion coefficient, and determine the field of view range of the camera model;

[0113] Step 505: Render the visual field elements to the frame buffer to form the model visual field screen P2

[0114] In actual implementation, the map elements within the field of view are rendered into a frame buffer to form a high-precision map camera model perspective screen P2;

[0115] Step 506: Use Hough transform to extract features from P1 and P2.

[0116] Step 507, calculate the quantity similarity, angular similarity, and geometric similarity.

[0117] In actual implementation, Hough transform is used to extract features from P1 and P2 to evaluate the similarity between P1 and P2. The specific steps are as follows:

[0118] 1. Image preprocessing: grayscale P1 and P2, and perform edge detection to extract contours to form processed images P1' and P2';

[0119] 2. Hough transform: Perform Hough transform on P1' and P2' to obtain the straight line features in the image. P1(ρ x ,θ x )、P2(ρ y ,θ y ), where (ρ x ,θ x ), (ρ y ,θ y ) are straight line polar coordinates, x and y are the straight line numbers in the two figures, x∈[1,X], y∈[1,Y], X and Y are the numbers of straight lines in the two figures;

[0120] 3. Calculation of geometric similarity: Calculate Sim(P1,P2)=Num(P1,P2)*0.2+Sim KL (P1,P2)*0.8;

[0121] Where Num(P1,P2) is the quantity similarity, Among them, max(X,Y) is the maximum value of the two;

[0122] Sim KL (P1, P2) is the angle similarity, which is calculated as follows: Among them, D KL (P1||P2) is the data set divergence, calculated as

[0123] Step 508, determine whether the geometric similarity exceeds a threshold, if so, determine that the attachment is normal, otherwise mark the attachment as abnormal.

[0124] In actual implementation, if Sim(P1,P2) exceeds the threshold, it is considered that N_SC (u.k) With ID (u.k) If the mapping association is established, the data in the mapping table is marked as normal; otherwise, it is marked as abnormal and abnormal data processing is performed; wherein the threshold can be set to 0.9.

[0125] In some embodiments, abnormal data processing is also performed. Figure 6 , Figure 6 It is an optional flowchart of abnormal data processing provided in an embodiment of the present application.

[0126] Step 601, extracting the image sample p1 and camera parameters corresponding to the abnormal data camera number.

[0127] For data marked as abnormal, extract the image sample data p1 corresponding to its camera number N_abn, as well as the camera focal length, principal point, and distortion coefficient.

[0128] Step 602: extract the model data space clusters in the business data space cluster search area.

[0129] Step 603: extract the field of view to obtain the camera model perspective image dataset p2.

[0130] Extract each model data spatial clustering cluster within the retrieval area of ​​the business data spatial clustering cluster to which it belongs; extract the field of view range of all model data in each spatial clustering cluster according to the camera focal length, principal point, and distortion coefficient of the camera number N_abn according to the method of step 504, and obtain the high-precision map camera model perspective picture image dataset p2 corresponding to the camera number N_abn.

[0131] Step 604, use YOLO to detect the target area of ​​the markings, signs, and signs of all images of p1 and p2.

[0132] Use the YOLO algorithm to detect the markings, signs, and signs in all image data in p1 and p2 to form the target area;

[0133] Step 605: Use SIFT algorithm to process the target area to obtain feature points and descriptors, feature point quantity, and feature point matching relationship.

[0134] All the image target areas in p1 and p2 formed by standard 604 are processed by SIFT (Scale Invariant Feature Transform) algorithm to obtain the target area feature points and descriptors, the number N1 and N2 of the feature points of the p1 image and the feature points of each picture in p2, and the matching relationship between the feature points of p1 and p2;

[0135] Step 606, verifying the feature point matching relationship based on RANSAC (Random Sampling Consensus Algorithm), and deleting the image feature points that are incorrectly matched;

[0136] Step 607, calculating feature similarity.

[0137] Calculate the number of matches N between the feature points of the p1 image and the feature points of each image in p2 match , calculate the feature similarity of each image in p1 and p2 Among them, min(N1,N2) is the minimum value of the two;

[0138] Step 608: Link the model number and camera number corresponding to the image with the highest feature similarity, and update the mapping relationship table.

[0139] The camera number N_abn corresponding to p1 is linked to the model number corresponding to the camera model with the highest feature similarity in p2, and the mapping relationship is updated in the mapping relationship table.

[0140] The embodiment of the present application constructs a set of verification strategies for highway scenarios, combining spatial position matching, computer vision, and image feature extraction, which can quickly and accurately check the matching relationship between non-high-precision positioning data and high-precision map models, and provide reliable feedback and correction mechanisms. For roadside monitoring cameras, a single-device verification method can be used to quickly check the quality of the mounting data based on spatial position matching; for scenes with high equipment density such as ETC gantries, the Hough transform method is used to judge the similarity between the business system camera image and the model simulation field of view range image. This method can be targeted to check the mounting accuracy of cameras that have established a mounting relationship, thereby improving computational efficiency. In the data governance process, a data processing method combining the YOLO algorithm and the SIFT algorithm is used. The real-time and high efficiency of YOLO enables it to quickly detect the target area in the image, reducing the amount of SIFT calculations and improving the overall processing efficiency. At the same time, using YOLO to locate the target area first and then applying SIFT to calculate the similarity can avoid background interference and improve the accuracy and robustness of feature matching.

[0141] The following is a description of an exemplary structure of the camera data hookup verification device 2055 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the verification device 2055 of the camera data hookup stored in the memory 205 may include:

[0142] An acquisition module 20551 is used to obtain a semantic data set of a three-dimensional model with a first camera precision and a business data set with a second camera precision connected to a camera cluster, wherein the first camera precision is greater than the second camera precision, and the camera cluster includes a plurality of single cameras;

[0143] The clustering module 20552 is used to perform spatial clustering processing on the semantic data set of the three-dimensional model to obtain spatial clustering clusters of model data, and to perform spatial clustering processing on the business data set to obtain spatial clustering clusters of business data;

[0144] A determination module 20553 is used to determine the number of samples of the model data space clustering cluster and the business data space clustering cluster;

[0145] The verification module 20554 is used to determine the connection accuracy verification of a single camera or a camera cluster based on the number of samples.

[0146] In some embodiments, the acquisition module 20551 is also used to obtain a high-precision map dataset, a business dataset and a first mapping relationship of the camera connected to the camera cluster, wherein the first mapping relationship is a mapping relationship between the camera high-precision map dataset and the business dataset, and the camera high-precision map dataset includes a high-precision map vector dataset and a three-dimensional model semantic dataset; coordinate unification processing is performed on the high-precision map vector dataset, the three-dimensional model semantic dataset and the business dataset to obtain a high-precision map vector dataset, the three-dimensional model semantic dataset and the business dataset with unified coordinates.

[0147] In some embodiments, the device also includes a coordinate unification module, which is used to perform consistency detection on the longitude and latitude of the high-precision map vector dataset and the three-dimensional model semantic dataset, and determine the model data from the three-dimensional model semantic dataset that is inconsistent with the longitude and latitude of the high-precision map vector dataset; adjust the longitude and latitude of the model data to be consistent with the corresponding vector data to obtain a three-dimensional model semantic dataset that is consistent with the longitude and latitude of the high-precision map vector dataset.

[0148] In some embodiments, the verification module 20554 is also used for a single camera camera cluster. If the number of samples of the business data space clustering cluster is 1, it is determined to verify the accuracy of the connection of the single camera; if the number of samples of the business data space clustering cluster is not 1, it is determined to verify the accuracy of the connection of the camera cluster.

[0149] In some embodiments, if it is determined to verify the accuracy of the attachment of a single camera, the verification module 20554 is also used for the single camera to extract any single camera business data space clustering cluster; to search in the retrieval area whether there is a model data space clustering cluster set whose cluster attribute is the target value, and the model data space clustering cluster set includes one or more model data space clustering clusters; if so, to associate the single camera business data space clustering cluster with the model data space clustering cluster set; to extract the first camera number in the single camera business data space clustering cluster; to determine whether there is a model number corresponding to the first camera number in the first mapping relationship; if so, to extract the second camera number of the model data space clustering cluster set associated with the single camera business data space clustering cluster; if the first camera number is consistent with the second camera number, it is determined that the data attachment of the corresponding camera is normal.

[0150] In some embodiments, if it is determined to verify the accuracy of the camera cluster, the verification module 20554 is also used for the camera cluster to extract camera numbers from the business data space cluster cluster one by one to obtain multiple third camera numbers; extract image sample data and camera parameters of the cameras corresponding to the multiple third camera numbers; extract the corresponding model position parameters from the three-dimensional model semantic data set according to the model number corresponding to the third camera number; based on the map engine, determine the camera model perspective image according to the camera parameters; calculate the similarity between the image sample data and the camera model perspective image; if the similarity reaches the similarity threshold, it is determined that the data mounting of the camera cluster is normal.

[0151] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above-mentioned method embodiment, and has similar beneficial effects as the method embodiment, so it will not be repeated.

[0152] The embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the camera data attachment verification method described in the embodiment of the present application.

[0153] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the camera data mounting verification method provided in the embodiment of the present application.

[0154] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0155] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0156] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0157] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0158] In summary, the embodiments of the present application can improve the efficiency and accuracy of camera calibration data verification.

[0159] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A camera data hookup verification method, characterized in that: include: Obtaining a semantic data set of a three-dimensional model with a first camera precision and a business data set with a second camera precision connected to a camera cluster, wherein the first camera precision is greater than the second camera precision, and the camera cluster includes a plurality of single cameras; Performing spatial clustering processing on the semantic data set of the three-dimensional model to obtain spatial clustering clusters of model data, and performing spatial clustering processing on the business data set to obtain spatial clustering clusters of business data; Determining the number of samples of the model data space clustering cluster and the business data space clustering cluster; Based on the number of samples, the connection accuracy verification of a single camera or a camera cluster is determined.

2. The method according to claim 1, characterized in that The step of obtaining a semantic data set of a three-dimensional model with a first camera precision and a business data set with a second camera precision connected to the camera cluster includes: Obtain a camera high-precision map dataset, a business dataset and a first mapping relationship connected to the camera cluster, wherein the first mapping relationship is a mapping relationship between the camera high-precision map dataset and the business dataset, and the camera high-precision map dataset includes a high-precision map vector dataset and a three-dimensional model semantic dataset; The high-precision map vector dataset, the three-dimensional model semantic dataset and the business dataset are subjected to coordinate unification processing to obtain a high-precision map vector dataset, the three-dimensional model semantic dataset and the business dataset with unified coordinates.

3. The method according to claim 2, characterized in that The method further comprises: Perform consistency check on the longitude and latitude of the high-precision map vector dataset and the three-dimensional model semantic dataset, and determine model data from the three-dimensional model semantic dataset that is inconsistent with the longitude and latitude of the high-precision map vector dataset; The longitude and latitude of the model data are adjusted to be consistent with the corresponding vector data to obtain a three-dimensional model semantic data set that is consistent with the longitude and latitude of the high-precision map vector data set.

4. The method according to claim 1, characterized in that The determining, based on the number of samples, to verify the accuracy of mounting a single camera or a camera cluster includes: If the number of samples of the business data space cluster is 1, it is determined to verify the accuracy of the mounting of the single camera; If the number of samples of the business data space cluster is not 1, it is determined to verify the connection accuracy of the camera cluster.

5. The method according to claim 1, characterized in that: If it is determined to verify the accuracy of the mounting of the single camera, then the verification of the accuracy of the mounting of the single camera includes: Extract the spatial clustering of any single camera business data; Searching within the search region whether there is a model data space clustering cluster set whose cluster attribute is a target value, wherein the model data space clustering cluster set includes one or more model data space clustering clusters; If so, associating the single camera business data space clustering cluster with the model data space clustering cluster set; Extracting the first camera number in the spatial cluster of the single camera service data; Determine whether there is a model number corresponding to the first camera number in the first mapping relationship; If so, extract the second camera number of the model data space cluster set associated with the single camera business data space cluster cluster; If the first camera number is consistent with the second camera number, it is determined that the data connection of the corresponding camera is normal.

6. The method according to claim 1, characterized in that If it is determined to verify the connection accuracy of the camera cluster, then the verification of the connection accuracy of the camera cluster includes: Extracting camera numbers from the business data space clusters one by one to obtain multiple third camera numbers; Extracting image sample data and camera parameters of cameras corresponding to a plurality of third camera numbers; Extracting corresponding model position parameters from the three-dimensional model semantic data set according to the model number corresponding to the third camera number; Based on the map engine, according to the camera parameters, determine the camera model perspective image; Calculating the similarity between the image sample data and the camera model perspective image; If the similarity reaches the similarity threshold, it is determined that the data connection of the camera cluster is normal.

7. A camera data hookup verification device, characterized in that: include: An acquisition module, used to obtain a semantic data set of a three-dimensional model with a first camera precision and a business data set with a second camera precision connected to a camera cluster, wherein the first camera precision is greater than the second camera precision, and the camera cluster includes a plurality of single cameras; A clustering module, used to perform spatial clustering processing on the semantic data set of the three-dimensional model to obtain spatial clustering clusters of model data, and to perform spatial clustering processing on the business data set to obtain spatial clustering clusters of business data; A determination module, used to determine the number of samples of the model data space clustering cluster and the business data space clustering cluster; A verification module is used to determine the connection accuracy verification of a single camera or a camera cluster based on the number of samples.

8. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor is used to implement the camera data attachment verification method described in any one of claims 1 to 6 when executing the executable instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to implement the camera data connection verification method described in any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that A computer program is stored, which is used to implement the camera data connection verification method described in any one of claims 1 to 6 when executed by a processor.

Images