A data set construction method, device, equipment, program product and medium

Abstract

The present invention provides a data set construction method, device, equipment, program product and medium, which relates to the field of data processing technology, wherein the method comprises: obtaining latitude and longitude information corresponding to a target road and at least one video frame, wherein the latitude and longitude information is acquired by an on-board device installed on a target vehicle, and at least one video frame is acquired by a roadside device; for each video frame, based on an angle influence factor, obtaining the pixel coordinates corresponding to a feature point of the target vehicle in the video frame, wherein the feature point is used to identify the position of the on-board device on the target vehicle, and the angle influence factor is used to characterize the degree of influence of the shooting angle on the position of the feature point in the video frame when the roadside device acquires the video frame; obtaining the target data set according to the latitude and longitude information and the pixel coordinates corresponding to the feature point in the video frame. The solution of the present invention solves the problem that the existing data set construction method has safety hazards and the quality of the obtained data set is poor.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data set construction method, device, equipment, program product and medium. Background Art

[0002] In the construction of intelligent transportation systems, it is necessary to solve the problem of visualization of traffic participants at smart intersections. Among them, the collection and construction of smart intersection data sets is the basis of the entire longitude and latitude mapping in the visualization problem.

[0003] At present, the collection and construction scheme of data sets usually requires test personnel to enter the road and use real-time kinematic (RTK) tools to collect longitude and latitude information on the road surface one by one. The test personnel then mark the longitude and latitude information on the high-precision map and obtain the pixel values ​​corresponding to the longitude and latitude through the characteristic markers in the video to construct a mapping data set. However, this method of manually obtaining data and labeling data has certain safety hazards, and the quality of the obtained data set is poor, resulting in poor mapping effect of latitude and longitude mapping based on the data set. Summary of the invention

[0004] The present invention provides a data set construction method, device, equipment, program product and medium, which solve the problems that the existing data set construction method has potential safety hazards and the quality of the obtained data set is poor.

[0005] In a first aspect, an embodiment of the present invention provides a method for constructing a data set, comprising:

[0006] Acquire latitude and longitude information and at least one video frame corresponding to the target road, wherein the latitude and longitude information is acquired by an on-board device installed on the target vehicle, and the at least one video frame is acquired by a roadside device;

[0007] For each of the video frames, based on the angle influence factor, the pixel coordinates corresponding to the feature points of the target vehicle in the video frame are obtained, the feature points are used to identify the position of the vehicle-mounted device on the target vehicle, and the angle influence factor is used to characterize the degree of influence of the shooting angle on the position of the feature points in the video frame when the roadside device collects the video frame;

[0008] A target data set is obtained according to the longitude and latitude information and the pixel coordinates corresponding to the feature points in the video frame.

[0009] In a second aspect, an embodiment of the present invention provides a data set construction device, including:

[0010] A first acquisition module is used to acquire the latitude and longitude information corresponding to the target road and at least one video frame, wherein the latitude and longitude information is acquired by an on-board device installed on the target vehicle, and the at least one video frame is acquired by a roadside device;

[0011] A second acquisition module is used to acquire, for each of the video frames, pixel coordinates corresponding to feature points of the target vehicle in the video frame based on an angle influence factor, wherein the feature point is used to identify the position of the vehicle-mounted device on the target vehicle, and the angle influence factor is used to characterize the degree of influence of the shooting angle on the position of the feature point in the video frame when the roadside device collects the video frame;

[0012] The first processing module is used to obtain a target data set according to the longitude and latitude information and the pixel coordinates corresponding to the feature points in the video frame.

[0013] In a third aspect, an embodiment of the present invention provides a device, comprising: a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the data set construction method described in the first aspect are implemented.

[0014] In a fourth aspect, an embodiment of the present invention provides a computer program product, comprising computer instructions, which implement the steps of the method described above when executed by a processor.

[0015] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the data set construction method as described in the first aspect.

[0016] The beneficial effects of the above technical solution of the present invention are:

[0017] In the embodiment of the present invention, for the video frames collected by the roadside equipment, the pixel coordinates corresponding to the feature points of the target vehicle in the video frame can be obtained based on the angle influence factor. In this way, by introducing the angle influence factor, the influence of the shooting angle on the location of the feature points in the video frame can be taken into account during the data set construction process, and more accurate pixel coordinates of the feature points can be obtained, and then a more accurate and effective target data set can be constructed based on the pixel coordinates and the longitude and latitude information collected by the vehicle-mounted equipment. In the data collection, the test personnel can drive the target vehicle into the target road to collect the longitude and latitude information, which can avoid the safety hazards when collecting longitude and latitude information in conventional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart showing a method for constructing a data set according to an embodiment of the present invention;

[0019] Figure 2 A schematic diagram showing a flow chart of a method for constructing a data set according to an embodiment of the present invention;

[0020] Figure 3 A schematic diagram showing the positional relationship between a target vehicle, a target surface, and an optical center according to an embodiment of the present invention;

[0021] Figure 4 A schematic diagram showing the position between the roadside equipment and the target vehicle according to an embodiment of the present invention;

[0022] Figure 5 A schematic diagram showing shooting angles according to an embodiment of the present invention;

[0023] Figure 6 A structural block diagram showing a data set construction device according to an embodiment of the present invention;

[0024] Figure 7 A block diagram showing the structure of a device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the present invention. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. In addition, for clarity and brevity, the description of known functions and structures is omitted.

[0026] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0027] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0028] At present, the collection and construction scheme of data sets mainly includes:

[0029] (1) The test personnel need to enter the road and use the RTK tool to collect longitude and latitude information on the road surface one by one. Therefore, this method has certain safety risks. In addition, the amount of data collected is small, and the number of points collected is mostly a few or dozens, which cannot cover the entire data intersection well, resulting in poor subsequent mapping effect.

[0030] (2) Manual labeling is used, in which testers label longitude and latitude information on high-precision maps and obtain pixel values ​​corresponding to longitude and latitude through feature markers in the video to construct a mapping dataset. However, the dataset obtained through this manual data labeling method is of poor quality, time-consuming, and difficult to reuse at multiple intersections.

[0031] Specifically, the embodiments of the present invention provide a data set construction method, apparatus, device, program product and medium, which solve the problems that the existing data set construction methods have potential safety hazards and the obtained data sets have poor quality.

[0032] First embodiment

[0033] like Figure 1 As shown, an embodiment of the present invention provides a method for constructing a data set, which specifically includes the following steps:

[0034] Step 11: Obtain the latitude and longitude information and at least one video frame corresponding to the target road, wherein the latitude and longitude information is acquired by an onboard device installed on the target vehicle, and the at least one video frame is acquired by a roadside device.

[0035] It should be noted that the target vehicle is equipped with an on-board device (with a positioning function and capable of collecting vehicle trajectory data). The on-board device may specifically be an on-board unit (OBU), and the on-board device may specifically be installed at a center point perpendicular to the chassis of the target vehicle.

[0036] Specifically, Figure 2As shown, in the data collection stage, a roadside device can be set near the target road, and the roadside device has a shooting function, which can collect and obtain a video stream about the target road. In addition, a marker can be set at a conspicuous position on the target vehicle, and the tester drives the target vehicle on the target road in turn (for example, in the order of lanes from left to right) in each lane, and collects longitude and latitude information and time information (that is, the time point of collecting the longitude and latitude information, which can be accurate to two decimal places after the second) through the on-board device during driving. In addition, the video stream is shot by a roadside device (such as a roadside end camera or other equipment with a shooting function), so that at least one video frame corresponding to the target vehicle can be obtained, and these video frames contain images of the target vehicle on the target road. It should be noted that, using these video frames, a detection model that can accurately identify the target vehicle can be trained.

[0037] It is understandable that after acquiring the video stream, the embodiment of the present invention can also pre-process the video frames in the video stream, for example, select the position in the video frame that will not block the road condition information and add a timestamp watermark (the timestamp can be accurate to six decimal places in seconds); for example, in order to increase the accuracy of the watermark when it is recognized in the subsequent process, a watermark with a black background and yellow font can be used. Afterwards, the video frame and the longitude and latitude information can be stored in the video database and the trajectory database respectively. Among them, the longitude and latitude information includes: longitude information, latitude information, and time information (that is, the time point of collecting the longitude information and latitude information).

[0038] Step 12: For each of the video frames, based on the angle influence factor, obtain the pixel coordinates corresponding to the feature points of the target vehicle in the video frame, wherein the feature points are used to identify the position of the vehicle-mounted equipment on the target vehicle, and the angle influence factor is used to characterize the degree of influence of the shooting angle on the position of the feature points in the video frame when the roadside equipment collects the video frame.

[0039] It should be noted that the distance of the target vehicle from the roadside equipment (such as a camera) will affect the angle between the target vehicle and the roadside equipment (that is, the shooting angle of the target vehicle shot by the roadside equipment), and the target vehicle is a three-dimensional object. Therefore, the relationship between the feature point of the target vehicle (which can also be understood as the position of the vehicle-mounted equipment on the target vehicle) and the identification rectangular box (that is, the bounding box corresponding to the target vehicle identified by the above detection model from the video frame) is not static. In the embodiment of the present invention, an angle influence factor (represented by α) is introduced to characterize the influence of the shooting angle on the position of the feature point in the video frame, so that the position of the feature point of the target vehicle can be more accurately confirmed by considering the position influence brought by the shooting angle through the algorithm.

[0040] Step 13: Obtain a target data set according to the longitude and latitude information and the pixel coordinates corresponding to the feature points in the video frame.

[0041] Here, data calibration can be performed through step 13, and the obtained target data set includes latitude and longitude information and pixel coordinates with a mapping relationship. The target data set can be provided to a pixel longitude and latitude mapping model (the trained model can identify the longitude and latitude information corresponding to the pixel points in the video frame) for model training, which can lay a solid foundation for model construction and large-scale integrated fusion visualization of intersection projects.

[0042] In this embodiment, for the video frames collected by the roadside equipment, the pixel coordinates corresponding to the feature points of the target vehicle in the video frame can be obtained based on the angle influence factor. In this way, by introducing the angle influence factor, the influence of the shooting angle on the location of the feature points in the video frame can be taken into account during the data set construction process, and more accurate pixel coordinates of the feature points can be obtained, and then a more accurate and effective target data set can be constructed based on the pixel coordinates and the longitude and latitude information collected by the vehicle-mounted equipment. In this data set construction method, when collecting data, the test personnel can drive the target vehicle into the target road to collect longitude and latitude information, which can avoid the safety hazards when collecting longitude and latitude information in conventional methods.

[0043] Optionally, in the above step 12, obtaining the pixel coordinates corresponding to the feature points of the target vehicle in the video frame based on the angle influence factor includes:

[0044] Step 1201: perform target detection on the video frame to obtain position information of a bounding box corresponding to the target vehicle.

[0045] Here, the target detection algorithm can be used to detect the target in the video frame and identify the position of the target vehicle in the video frame. Specifically, the trained detection model can be used to identify the target vehicle and obtain the bounding box (usually a rectangular box) corresponding to the target vehicle. The return value of the detection model can be the position information of the bounding box (for example, the position information can specifically include two diagonal coordinates: the coordinates of the upper left vertex and the coordinates of the lower right vertex).

[0046] Step 1202: Determine the angle influence factor corresponding to the video frame according to the position information of the bounding box, the actual size information of the target vehicle, and the installation height of the roadside equipment relative to the ground.

[0047] Here, the angle influence factor α may be determined based on the position information of the bounding box obtained in step 1201 and other known information (such as the actual size of the target vehicle, etc.). In some optional embodiments, step 1202 may specifically include the following steps:

[0048] (1) Determine the projection height of the bounding box on the camera target surface according to the position information of the bounding box, the actual size information of the target vehicle, and the height of the camera target surface corresponding to the roadside equipment.

[0049] It should be noted that the position information of the bounding box includes the coordinates of the first vertex and the second vertex on the bounding box, and the first vertex and the second vertex are located on the same diagonal line of the bounding box. Here, the first vertex is the upper left vertex of the bounding box and the second vertex is the lower right vertex of the bounding box.

[0050] Assume that the coordinates of the upper left corner of the bounding box are: (x lt ,y lt ), the coordinates of the lower right vertex are: (x rb ,y rb ), the video frame resolution corresponding to the roadside equipment is: x r ×y r , the focal length of the camera corresponding to the roadside equipment is d, the installation height of the roadside equipment relative to the ground is H, and the height of the camera target surface corresponding to the roadside equipment is h b .

[0051] Then, according to the formula: b=((y rb -y lt ) / y r )×h b , determine the projection height (represented by b) corresponding to the bounding box in the camera target surface.

[0052] (2) Determine the horizontal distance between the roadside equipment and the target vehicle based on the projection height, the actual size information of the target vehicle, the installation height, and the focal length of the camera corresponding to the roadside equipment.

[0053] Here, if Figure 3 As shown, we can use the similarity relationship according to the formula: The distance (expressed as L) between the target vehicle and the roadside equipment is calculated (for example, the distance between the target vehicle and the camera base can be specifically calculated).

[0054] The actual size information of the target vehicle includes the height and length of the target vehicle, where h represents the height of the target vehicle and w represents the length of the target vehicle. Figure 4 As shown, Indicates the actual height of the target vehicle detected from the perspective of the roadside equipment (such as a camera) calculated by trigonometric functions.

[0055] (3) Determine a shooting angle between the roadside equipment and the target vehicle based on the horizontal distance and the installation height.

[0056] Here, if Figure 5 As shown, the shooting angle between the roadside equipment and the target vehicle can be determined according to the formula: θ=arctan(L / H), where θ represents the shooting angle, L represents the horizontal distance between the target vehicle and the roadside equipment, and H represents the installation height of the roadside equipment relative to the ground.

[0057] (4) Determine an angle influence factor corresponding to the video frame according to the shooting angle.

[0058] Here, the angle influence factor α corresponding to the video frame may be determined according to the formula: α=θ / 90, where α represents the angle influence factor and θ represents the shooting angle.

[0059] Step 1203: Obtain pixel coordinates corresponding to feature points of the target vehicle in the video frame according to the position information of the bounding box and the angle influence factor.

[0060] It should be noted that the pixel coordinates corresponding to the feature points of the target vehicle in the video frame are actually the accurate pixel coordinates corresponding to the latitude and longitude positions measured by the OBU, which are closer to the ground pixel coordinates of the target vehicle's location. Since the pixel coordinates are two-dimensional coordinates, it is difficult to reflect the depth information. Therefore, if the pixel coordinates of the feature points are taken blindly, the target data set finally formed will have errors. In the embodiment of the present invention, the error problem of the pixel coordinates of the feature points can be solved based on the angle influence factor, and a more accurate and effective target data set is obtained.

[0061] Specifically, in step 1203, the value of the angle influence factor α can be determined according to the value of the shooting angle θ, and the angle influence factor α can be used to represent the upper and lower position ratio of the feature point in the bounding box. Figure 5 As shown, it can be understood that the larger the shooting angle θ is, the farther the target vehicle is from the roadside equipment, the flatter the shooting angle, and the closer the position of the feature point is to the bottom of the bounding box. Therefore, the vertical coordinate position interval of the feature point is generally located between: the four-equal points from the top to the bottom of the bounding box to the bottom line of the bounding box. The pixel coordinates (x t ,y t ):

[0062] y t =(α×0.75+0.25)×(y rb -y lt )+y lt

[0063] x t =(x rb -x lt)×0.5+x lt

[0064] In this way, the shooting angle is determined by detecting the distance L between the target vehicle and the roadside equipment, and the positions of pixel feature points (ie, feature points of the target vehicle) are strategically selected according to the shooting angle to obtain more accurate pixel coordinates of the feature points.

[0065] In the above embodiment, the pixel coordinates corresponding to the feature points of the target vehicle in the video frame are determined based on the angle influence factor, and the influence of the shooting angle on the selection of the feature points of the target vehicle is fully considered, so that a more accurate position of the feature points of the target vehicle can be obtained.

[0066] Optionally, in the above step 13, obtaining the target data set according to the longitude and latitude information and the pixel coordinates corresponding to the feature points in the video frame includes:

[0067] Step 1301: Screen the at least one video frame according to the pixel coordinates corresponding to the feature points in the video frames to obtain at least one target video frame.

[0068] In some optional embodiments, step 1301 may specifically include: traversing the at least one video frame in sequence, and obtaining a first target video frame that meets a first preset condition according to the pixel coordinates corresponding to the feature point in the reference video frame; when the first target video frame is obtained, storing the first target video frame as the target video in a preset set, and updating the reference video frame to the first target video frame. The initial value of the reference video frame is a video frame with the earliest timestamp in the at least one video frame; the first preset condition includes: (1) there are at least M video frames between the first target video frame and the reference video frame, where M is a positive integer; (2) the target distance between the first target video frame and the reference video frame is greater than or equal to a preset value, and the target distance is determined according to the pixel coordinates corresponding to the feature point in the reference video frame, the pixel coordinates corresponding to the feature point in the first target video frame, and the distance factor corresponding to the first target video frame, and the distance factor is used to characterize the distance between the target vehicle and the roadside equipment.

[0069] In a specific embodiment, taking M=3 as an example, assuming that the coordinates of the center point of the lower boundary of the video frame are (x c ,y c )(For example, if the resolution of the video frame is 1920*1080, then x c =960,y c =1080), the video resolution is x r ×y r, the feature point of the target vehicle in the first frame (the first video frame, as the reference video frame) is initialized to point init , whose pixel coordinates are (x init ,y init ), set the initial value of flag to 0; after that, the value of flag increases by 1 for each frame update; when the flag value is equal to 4, obtain the position of the target vehicle in the current frame (that is, the position of the feature point) as point last , whose pixel coordinates are (x last ,y last );Judgment point init With point last The positional relationship of the target is used to determine the target distance (i.e. distance).

[0070] Based on the rule that near is big and far is small, it can be specifically calculated according to the formula: Determine the target distance (also known as the distance between frames).

[0071] in, Represents the distance factor, that is, the distance from the feature point of the current target vehicle to (x c ,y c ) to the diagonal length of the video frame, which can reflect the distance between the current position of the target vehicle and the roadside equipment; Indicates: the pixel displacement difference of the feature point of the target vehicle between the two frames when flag=4 and the last time flag=0.

[0072] When the target distance (distance) is greater than a preset value (such as 10), it is determined that the target distance meets the requirement for distinguishing video frames, and the video frame is determined as the first target video frame that meets the first preset condition, and the pixel coordinates (x last ,y last ) Save; if the target distance is less than or equal to 10, continue to recursively find the feature points of the target vehicle in the following video frames and set them as point last , the pixel coordinate value is (x last ,y last ), recalculate the distance value until the distance is greater than 10. After that, reset the flag to 0 and set the current point last The pixel coordinate value (x last ,y last ) is assigned to point init (x init ,y init), that is, taking the current video frame as the reference video frame, and continuing to search backward for the first target video frame that meets the first preset condition. Repeat the above process until the video frame screening is completed.

[0073] In order to construct an accurate and efficient target dataset, the target distance is introduced. This value can effectively represent the positional relationship of the target vehicle before and after it moves. By judging the size of this value, video frames whose target distance exceeds the preset value are selected. This can automatically filter video frames and remove redundant data (such as multiple video frames collected when the target vehicle is stationary for a period of time).

[0074] In this way, by filtering the video frames through the above steps, duplicate and redundant data can be removed, which can reduce the workload of testers and reduce the time cost of building the target data set.

[0075] Step 1302: Obtain a target data set according to the latitude and longitude information and the pixel coordinates of the feature point in the at least one target video frame.

[0076] Here, the longitude and latitude information may specifically include: the longitude information and latitude information of the target vehicle, and the time point when the longitude information and latitude information are collected.

[0077] In some optional embodiments, step 1302 may specifically include the following steps:

[0078] (1) for each of the target video frames, according to the timestamp corresponding to the target video frame, obtaining the target longitude information and the target latitude information corresponding to the target video frame from the longitude and latitude information, wherein the time difference between the acquisition time point of the target longitude information and the target latitude information and the timestamp is less than or equal to a first preset duration.

[0079] It should be noted that here, the longitude and latitude information whose time difference is less than or equal to the first preset time length can be directly selected to obtain the above-mentioned target longitude information and target latitude information; the timestamp and time point can also be unified in format first, and then based on the timestamp and time point after the unified format, the longitude and latitude information whose time point is consistent with the timestamp corresponding to the target video frame can be selected to obtain the above-mentioned target longitude information and target latitude information.

[0080] Specifically, in an optional example, the OCR digital recognition technology can be used to identify the timestamp corresponding to the video frame, and the identified timestamp can also be verified. For example, it is determined whether the timestamp conforms to the preset structure (for example: xxxx.xx.xx.xx.xx.xx.xxxxxx, that is: year. month. day. hour. minute. second. six decimal places after the second), and whether the timestamp conforms to the real natural law (for example, the year is not greater than the current year, the month is not greater than December, the day is not greater than 31 days, the hour is not greater than 24 hours, the minute is not greater than 60 minutes, the second is not greater than 60 seconds, etc.). If the timestamp verification fails, the target video frame corresponding to the timestamp is discarded; if the timestamp verification is successful, the time point in the timestamp and longitude and latitude information can be further formatted, for example, the six decimal places after the seconds of the timestamp are rounded off and integrated into a format with two decimal places. Afterwards, the longitude and latitude information is searched in the trajectory database according to the timestamp, that is, according to the timestamp corresponding to the target video frame, the target longitude information and target latitude information corresponding to the target video frame are obtained.

[0081] (ii) storing the pixel coordinates of the feature point in the target video frame and the target longitude information and target latitude information corresponding to the target video frame into the target data set.

[0082] That is, if the target longitude information and target latitude information corresponding to the target video frame can be successfully found, the pixel coordinates of the feature point in the target video frame and the target longitude information and target latitude information corresponding to the target video frame are stored in the target data set.

[0083] In the above steps, by adding timestamps to the video frames, the pixel coordinates can be aligned with the longitude and latitude information, providing high-precision pairing labels for the two. In this way, based on timestamps, it is easier to organize large amounts of data and construct mapping datasets, realize the automated construction of datasets, and provide a rich and convenient data foundation for mapping.

[0084] Here, a pseudo code for filtering video frames in a specific embodiment of the present invention is given as an example:

[0085] initialization:

[0086] Int flag = 0; / / Initialize the flag value and make a judgment every four frames

[0087] Int x_r, y_r = video resolution; / / Initialize the value of video resolution

[0088] Int x_c = x_r / 2;

[0089] Int y_c = x_c; / / Initialize the value of the center point of the lower boundary of the video

[0090] Vector <cv::point>Use_Point / / Initialize feature point save structure

[0091] Video frame loop:

[0092] While (target vehicle is identified): / / When the target vehicle is identified, enter the loop

[0093] If the first frame:

[0094] Initially assign the target vehicle feature point to point_init; / / Get the target vehicle feature point in the first frame Else:

[0095] Identify the target vehicle feature points and assign them to point_last; / / Get the target vehicle feature points in the current frame

[0096] If flag == 4:

[0097] If the inter-frame distance calculated by the distance judgment algorithm is > 10:

[0098] Use_Point.push_back(Point_last); / / When the result is True, store the feature point to save the structure

[0099] flag=0; / / and reset flag to 0

[0100] / / When the result is False, the flag value remains unchanged and continues to compare with the next frame Else:

[0101] Flag++; / / When flag is not equal to 4, add 1 to flag in each frame

[0102] In addition, it should be noted that if the target longitude information and target latitude information corresponding to the target video frame cannot be successfully found, the longitude information and latitude information corresponding to the determined timestamp can be obtained by fitting the time series historical trajectory data, as described below:

[0103] In some optional embodiments, the method further comprises:

[0104] (1) in the case where the target longitude information and the target latitude information corresponding to the target video frame are not obtained, obtaining N first longitude and latitude information that meet a second preset condition from the longitude and latitude information according to the timestamp corresponding to the target video frame, wherein the second preset condition includes: the time difference between the first time point for collecting the first longitude and latitude information and the timestamp is less than or equal to a second preset duration, the first time point is earlier than the timestamp, and N is a positive integer.

[0105] It should be noted that if the target longitude information and target latitude information corresponding to the target video frame cannot be obtained, it may be because the acquisition frequency of the vehicle-mounted equipment and the shooting frequency of the video frame are not aligned. In this case, the longitude and latitude information corresponding to the N most recent timestamps before the timestamp can be retrieved according to the timestamp corresponding to the target video frame (as an optional example, N can be 5). In this way, the longitude and latitude information corresponding to the timestamp can be subsequently fitted according to the N time series longitude and latitude information to determine the longitude and latitude information corresponding to the timestamp. Among them, the acquisition time of these N longitude and latitude information (that is, the first time point) is no more than the second preset time length (for example, 1s) from the timestamp. If it exceeds the second preset time length, the fitting process is abandoned and the video frame corresponding to this timestamp is discarded.

[0106] (ii) Determine the longitude and latitude information corresponding to the timestamp based on the N first longitude and latitude information, that is, the above-mentioned fitting process. In some optional embodiments, this step may specifically include the following steps:

[0107] (1) For the N trajectory points of the target vehicle corresponding to the N first longitude and latitude information, determine the distance between every two adjacent trajectory points in the N trajectory points according to the N first longitude and latitude information; wherein the distance includes the longitude distance and the latitude distance.

[0108] (2) Determine at least one speed change corresponding to the target vehicle based on the distance and time difference between every two adjacent trajectory points.

[0109] (3) determining a target speed change corresponding to the target vehicle based on the at least one speed change and a weight value corresponding to each speed change; wherein, when the distance is a longitude distance, the speed change is a first speed change of the target vehicle in longitude, and the target speed change is a first target speed change of the target vehicle in longitude; and when the distance is a latitude distance, the speed change is a second speed change of the target vehicle in latitude, and the target speed change is a second target speed change of the target vehicle in latitude.

[0110] (4) According to the target speed change and the N first longitude and latitude information, respectively determine the longitude information and latitude information corresponding to the timestamp.

[0111] In a specific embodiment, assuming that the timestamp corresponding to the target video frame is t, among the N track points corresponding to the first longitude and latitude information that meet the preset conditions (N is 5 for example), the time corresponding to the i-th track point is t i , whose corresponding longitude and latitude are lat i and lon i . It is assumed that the smaller i is, the newer the data is.

[0112] Then according to the formula: Determine the first target speed change (a lat ); can be based on the formula: Determine the second target speed change (a lon ). In the two formulas here, the values ​​of i are 3, 4, and 5.

[0113] That is to say, the average speed between the most recent time (i.e., time point t1 corresponding to the first track point before timestamp t) and the other four times (i.e., time points t2, t3, t4, and t5 corresponding to the four track points before t1) can be calculated in sequence (in the longitude direction: (lat1-lat i ) / (t1-t i ), in the latitude direction: (lon1-lon i ) / (t1-t i ), i=3, 4, 5), and calculate the difference between two adjacent average speeds to determine the speed change (including a first speed change in longitude and a second speed change in latitude), and then perform weighted calculation on the speed change (when N is 5, the number of the first speed change and the second speed change are both 3) to obtain the target speed change (including a first target speed change in longitude and a second target speed change in latitude).

[0114] Here, the weighted calculation part in the above formula is explained: lat ) and the second target speed change (a lon ), the values ​​of i are 3, 4, and 5. Here, based on the principle that the closer the time is to timestamp t, the greater the weight, when i=3, it is closest to timestamp t. Therefore, the weight assigned to t3 is 5, and t4 and t5 are 4 and 3 respectively. When i=3, (8-i) / 12 in the above formula is: (8-3) / (3+4+5).

[0115] After that, according to the formula: lat = (a lat +(lat1-lat i ) / (t1-t i ))×(t-t1)+lat1, determine the longitude information lat; according to the formula: lon=(a lon +(lon1-lon i ) / (t1-t i ))×(t-t1)+lon1, determine the latitude information lon.

[0116] Among them, (lat1-lat i ) / (t1-t i ) indicates that t1 and t i The average longitude speed between the average longitude speed plus the first target speed change (a lat ), which is the average speed at the next distance t1, and the longitude lat and latitude lon of the fitted timestamp t are calculated based on t1.

[0117] (iii) storing the pixel coordinates of the feature point in the target video frame and the longitude information and latitude information corresponding to the timestamp into the target data set.

[0118] Here, the final form of the target data set can be a csv file or a txt file with 2-channel input (pixel coordinates of feature points) and 2-channel output (longitude information and latitude information).

[0119] In this embodiment, the N historical trajectory points before the timestamp corresponding to the target video frame are analyzed to obtain the average speed and the target speed change, and finally the longitude and latitude information corresponding to the timestamp is fitted. In this way, it is possible to provide a fault tolerance for inconsistent acquisition frequency, and the speed change of the historical trajectory points is comprehensively considered, so that the fitted longitude and latitude data has a higher credibility.

[0120] In the above embodiment, in order to accurately identify the target vehicle, a marker can be placed on the target vehicle, and a detection model that can identify the target vehicle can be trained; then the vehicle-mounted device is placed on the target vehicle, and the tester drives the target vehicle on the target road to collect the trajectory data (i.e., longitude and latitude information) of the target vehicle, and at the same time, the video data of the target road is collected through the roadside equipment, and a real-time timestamp watermark is added to each video frame in the video data. After the on-site data collection is completed, the detection model is used to identify the video data collected by the roadside equipment, the boundary box of the target vehicle is obtained, the pixel coordinates of the feature points of the target vehicle are obtained based on the angle influence factor, and then the movement of the target vehicle is judged based on the pixel coordinates of the feature points, and the video frame is screened, and the target video frame whose displacement of the target vehicle exceeds the threshold is selected. For the screened target video frame, the OCR technology is used to identify the timestamp of the video frame, and the identified time information is error-checked (verified). If the verification fails, the video frame corresponding to the timestamp is discarded. Afterwards, the timestamp of successful verification is compared with the time data in the longitude and latitude information; if the comparison fails, the location of the current video frame (i.e. longitude and latitude) is obtained by fitting the time series historical trajectory data; if the comparison is successful, the longitude and latitude data corresponding to the timestamp are selected and stored in the target data set together with the corresponding pixel coordinates, thereby automatically completing the construction of the mapping data set.

[0121] In an embodiment of the present invention, a mapping data set between pixel coordinates and longitude and latitude can be automatically constructed based on the longitude and latitude information collected by the vehicle-mounted equipment and the video frames collected by the roadside equipment. The data set construction method collects more data in a more comprehensive manner and can be reused at multiple intersections, thereby achieving fast, safe, and reusable data collection and calibration, saving time costs.

[0122] Second embodiment

[0123] like Figure 6 As shown, an embodiment of the present invention provides a data set construction device 600, including:

[0124] A first acquisition module 601 is used to acquire latitude and longitude information corresponding to a target road and at least one video frame, wherein the latitude and longitude information is acquired by an on-board device installed on a target vehicle, and the at least one video frame is acquired by a roadside device;

[0125] A second acquisition module 602 is used to acquire, for each of the video frames, pixel coordinates corresponding to feature points of the target vehicle in the video frame based on an angle influence factor, wherein the feature point is used to identify the position of the vehicle-mounted device on the target vehicle, and the angle influence factor is used to characterize the degree of influence of the shooting angle on the position of the feature point in the video frame when the roadside device collects the video frame;

[0126] The first processing module 603 is used to obtain a target data set according to the longitude and latitude information and the pixel coordinates corresponding to the feature points in the video frame.

[0127] In this embodiment, for the video frames collected by the roadside equipment, the pixel coordinates corresponding to the feature points of the target vehicle in the video frame can be obtained based on the angle influence factor. In this way, by introducing the angle influence factor, the influence of the shooting angle on the location of the feature points in the video frame can be taken into account during the data set construction process, and more accurate pixel coordinates of the feature points can be obtained, and then a more accurate and effective target data set can be constructed based on the pixel coordinates and the longitude and latitude information collected by the vehicle-mounted equipment. In this data set construction method, when collecting data, the test personnel can drive the target vehicle into the target road to collect longitude and latitude information, which can avoid the safety hazards when collecting longitude and latitude information in conventional methods.

[0128] Optionally, the second acquisition module 602 includes:

[0129] The target detection submodule is used to perform target detection on the video frame to obtain the position information of the bounding box corresponding to the target vehicle;

[0130] A first processing submodule, configured to determine an angle influence factor corresponding to the video frame according to the position information of the bounding box, the actual size information of the target vehicle, and the installation height of the roadside equipment relative to the ground;

[0131] The second processing submodule is used to obtain the pixel coordinates corresponding to the feature points of the target vehicle in the video frame according to the position information of the bounding box and the angle influence factor.

[0132] Optionally, the first processing submodule includes:

[0133] A first processing unit is used to determine a projection height corresponding to the bounding box in the camera target surface according to the position information of the bounding box, the actual size information of the target vehicle and the height of the camera target surface corresponding to the roadside equipment;

[0134] A second processing unit is used to determine the horizontal distance between the roadside equipment and the target vehicle according to the projection height, the actual size information of the target vehicle, the installation height and the focal length of the camera corresponding to the roadside equipment;

[0135] A third processing unit, configured to determine a shooting angle between the roadside equipment and the target vehicle according to the horizontal distance and the installation height;

[0136] The fourth processing unit is used to determine an angle influence factor corresponding to the video frame according to the shooting angle.

[0137] Optionally, the first processing module 603 includes:

[0138] A third processing submodule, configured to filter the at least one video frame according to pixel coordinates corresponding to the feature points in the video frame to obtain at least one target video frame;

[0139] The fourth processing submodule is used to obtain a target data set according to the longitude and latitude information and the pixel coordinates of the feature point in the at least one target video frame.

[0140] Optionally, the third processing submodule includes:

[0141] A video frame screening unit, configured to sequentially traverse the at least one video frame, and acquire a first target video frame satisfying a first preset condition according to pixel coordinates corresponding to the feature points in the reference video frame;

[0142] a fifth processing unit, configured to, when acquiring the first target video frame, store the first target video frame as the target video in a preset set, and update the reference video frame to the first target video frame;

[0143] Wherein, the initial value of the reference video frame is a video frame with the earliest timestamp among the at least one video frame;

[0144] The first preset condition includes:

[0145] There is at least M video frames between the first target video frame and the reference video frame, where M is a positive integer;

[0146] The target distance between the first target video frame and the reference video frame is greater than or equal to a preset value, and the target distance is determined based on the pixel coordinates corresponding to the feature point in the reference video frame, the pixel coordinates corresponding to the feature point in the first target video frame, and the distance factor corresponding to the first target video frame, and the distance factor is used to characterize the distance between the target vehicle and the roadside equipment.

[0147] Optionally, the longitude and latitude information includes: the longitude and latitude information of the target vehicle, and the time point when the longitude and latitude information are collected;

[0148] The fourth processing submodule includes:

[0149] a sixth processing unit, configured to obtain, for each of the target video frames, target longitude information and target latitude information corresponding to the target video frame from the longitude and latitude information according to the timestamp corresponding to the target video frame, wherein the time difference between the acquisition time point of the target longitude information and the target latitude information and the timestamp is less than or equal to a first preset duration;

[0150] The first storage unit is used to store the pixel coordinates of the feature point in the target video frame and the target longitude information and target latitude information corresponding to the target video frame into the target data set.

[0151] Optionally, the method further comprises:

[0152] a seventh processing unit, configured to obtain, if the target longitude information and the target latitude information corresponding to the target video frame are not obtained, N first longitude and latitude information satisfying a second preset condition from the longitude and latitude information according to the timestamp corresponding to the target video frame, wherein the second preset condition includes: a time difference between a first time point for collecting the first longitude and latitude information and the timestamp is less than or equal to a second preset duration, the first time point is earlier than the timestamp, and N is a positive integer;

[0153] an eighth processing unit, configured to determine the longitude information and the latitude information corresponding to the timestamp according to the N first longitude and latitude information;

[0154] The second storage unit is used to store the pixel coordinates of the feature point in the target video frame and the longitude information and latitude information corresponding to the timestamp into the target data set.

[0155] Optionally, the eighth processing unit includes:

[0156] A first processing subunit is used to determine, for each of the N track points of the target vehicle corresponding to the N first longitude and latitude information, the distance between each two adjacent track points in the N track points according to the N first longitude and latitude information;

[0157] A second processing subunit is used to determine at least one speed change corresponding to the target vehicle according to the distance and time difference between every two adjacent trajectory points;

[0158] A third processing subunit, configured to determine a target speed change corresponding to the target vehicle according to the at least one speed change and a weight value corresponding to each speed change;

[0159] a fourth processing subunit, configured to determine the longitude information and the latitude information corresponding to the timestamp according to the target speed change and the N first longitude and latitude information;

[0160] Wherein, the distance includes longitude distance and latitude distance;

[0161] In the case where the distance is a longitude distance, the speed change is a first speed change of the target vehicle in longitude, and the target speed change is a first target speed change of the target vehicle in longitude;

[0162] In the case where the distance is a latitude distance, the speed change is a second speed change of the target vehicle at the latitude, and the target speed change is a second target speed change of the target vehicle at the latitude.

[0163] The second embodiment of the present invention corresponds to the method of the first embodiment. All implementation means in the first embodiment are applicable to the embodiment of the data set construction device and can achieve the same technical effect.

[0164] Third embodiment

[0165] In order to better achieve the above goals, Figure 7 As shown, the third embodiment of the present invention further provides a device, including:

[0166] A processor 700; and a memory 720 connected to the processor 700 via a bus interface, wherein the memory 720 is used to store programs and data used by the processor 700 when performing operations, and the processor 700 calls and executes the programs and data stored in the memory 720.

[0167] The transceiver 710 is connected to the bus interface and is used to receive and send data under the control of the processor 700; the processor 700 is used to read the program in the memory 720 to perform the following steps:

[0168] Acquire latitude and longitude information and at least one video frame corresponding to the target road, wherein the latitude and longitude information is acquired by an on-board device installed on the target vehicle, and the at least one video frame is acquired by a roadside device;

[0169] For each of the video frames, based on the angle influence factor, the pixel coordinates corresponding to the feature points of the target vehicle in the video frame are obtained, the feature points are used to identify the position of the vehicle-mounted device on the target vehicle, and the angle influence factor is used to characterize the degree of influence of the shooting angle on the position of the feature points in the video frame when the roadside device collects the video frame;

[0170] A target data set is obtained according to the longitude and latitude information and the pixel coordinates corresponding to the feature points in the video frame.

[0171] In this embodiment, for the video frames collected by the roadside equipment, the pixel coordinates corresponding to the feature points of the target vehicle in the video frame can be obtained based on the angle influence factor. In this way, by introducing the angle influence factor, the influence of the shooting angle on the location of the feature points in the video frame can be taken into account during the data set construction process, and more accurate pixel coordinates of the feature points can be obtained, and then a more accurate and effective target data set can be constructed based on the pixel coordinates and the longitude and latitude information collected by the vehicle-mounted equipment. In this data set construction method, when collecting data, the test personnel can drive the target vehicle into the target road to collect longitude and latitude information, which can avoid the safety hazards when collecting longitude and latitude information in conventional methods.

[0172] Among them, Figure 7 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by the processor 700 and various circuits of the memory represented by the memory 720 are linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 710 may be a plurality of components, namely, a transmitter and a transceiver, providing a unit for communicating with various other devices on a transmission medium. For different terminals, the user interface 730 may also be an interface capable of externally connecting or internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like. The processor 700 is responsible for managing the bus architecture and general processing, and the memory 720 may store data used by the processor 700 when performing operations.

[0173] Optionally, when the processor 700 obtains the pixel coordinates corresponding to the feature point of the target vehicle in the video frame based on the angle influence factor, it is specifically configured to:

[0174] Performing target detection on the video frame to obtain position information of a bounding box corresponding to the target vehicle;

[0175] Determining an angle influence factor corresponding to the video frame according to the position information of the bounding box, the actual size information of the target vehicle, and the installation height of the roadside equipment relative to the ground;

[0176] According to the position information of the bounding box and the angle influence factor, the pixel coordinates corresponding to the feature points of the target vehicle in the video frame are obtained.

[0177] Optionally, when the processor 700 determines the angle impact factor corresponding to the video frame according to the position information of the bounding box, the actual size information of the target vehicle, and the installation height of the roadside equipment relative to the ground, it is specifically configured to:

[0178] Determine the projection height of the bounding box on the camera target surface according to the position information of the bounding box, the actual size information of the target vehicle and the height of the camera target surface corresponding to the roadside equipment;

[0179] Determine the horizontal distance between the roadside equipment and the target vehicle according to the projection height, the actual size information of the target vehicle, the installation height and the focal length of the camera corresponding to the roadside equipment;

[0180] Determining a shooting angle between the roadside equipment and the target vehicle according to the horizontal distance and the installation height;

[0181] According to the shooting angle, an angle influence factor corresponding to the video frame is determined.

[0182] Optionally, when the processor 700 obtains the target data set according to the latitude and longitude information and the pixel coordinates corresponding to the feature point in the video frame, it is specifically configured to:

[0183] Filtering the at least one video frame according to pixel coordinates corresponding to the feature points in the video frames to obtain at least one target video frame;

[0184] A target data set is obtained according to the latitude and longitude information and the pixel coordinates of the feature point in the at least one target video frame.

[0185] Optionally, when the processor 700 screens the at least one video frame according to the pixel coordinates corresponding to the feature points in the video frames to obtain at least one target video frame, it is specifically configured to:

[0186] Traversing the at least one video frame in sequence, and acquiring a first target video frame that meets a first preset condition according to pixel coordinates corresponding to the feature point in the reference video frame;

[0187] When the first target video frame is acquired, the first target video frame is stored as the target video in a preset set, and the reference video frame is updated to the first target video frame;

[0188] Wherein, the initial value of the reference video frame is a video frame with the earliest timestamp among the at least one video frame;

[0189] The first preset condition includes:

[0190] There is at least M video frames between the first target video frame and the reference video frame, where M is a positive integer;

[0191] The target distance between the first target video frame and the reference video frame is greater than or equal to a preset value, and the target distance is determined based on the pixel coordinates corresponding to the feature point in the reference video frame, the pixel coordinates corresponding to the feature point in the first target video frame, and the distance factor corresponding to the first target video frame, and the distance factor is used to characterize the distance between the target vehicle and the roadside equipment.

[0192] Optionally, the longitude and latitude information includes: the longitude and latitude information of the target vehicle, and the time point when the longitude and latitude information are collected;

[0193] When the processor 700 obtains the target data set according to the latitude and longitude information and the pixel coordinates of the feature point in the at least one target video frame, it is specifically configured to:

[0194] For each of the target video frames, according to the timestamp corresponding to the target video frame, the target longitude information and the target latitude information corresponding to the target video frame are obtained from the longitude and latitude information, and the time difference between the acquisition time point of the target longitude information and the target latitude information and the timestamp is less than or equal to the first preset duration;

[0195] The pixel coordinates of the feature points in the target video frame and the target longitude information and target latitude information corresponding to the target video frame are stored in the target data set.

[0196] Optionally, the processor 700 is further configured to:

[0197] In the case where the target longitude information and the target latitude information corresponding to the target video frame are not obtained, obtaining N first longitude and latitude information that meet a second preset condition from the longitude and latitude information according to the timestamp corresponding to the target video frame, wherein the second preset condition includes: the time difference between the first time point of collecting the first longitude and latitude information and the timestamp is less than or equal to a second preset duration, the first time point is earlier than the timestamp, and N is a positive integer;

[0198] Determine the longitude and latitude information corresponding to the timestamp according to the N first longitude and latitude information;

[0199] The pixel coordinates of the feature point in the target video frame and the longitude information and latitude information corresponding to the timestamp are stored in the target data set.

[0200] Optionally, when the processor 700 determines the longitude information and the latitude information corresponding to the timestamp according to the N first longitude and latitude information, it is specifically configured to:

[0201] For the N track points of the target vehicle corresponding to the N first longitude and latitude information, determine the distance between every two adjacent track points in the N track points according to the N first longitude and latitude information;

[0202] Determine at least one speed change corresponding to the target vehicle according to the distance and time difference between every two adjacent trajectory points;

[0203] Determining a target speed change corresponding to the target vehicle according to the at least one speed change and a weight value corresponding to each speed change;

[0204] Determine the longitude information and latitude information corresponding to the timestamp respectively according to the target speed change and the N first longitude and latitude information;

[0205] Wherein, the distance includes longitude distance and latitude distance;

[0206] In the case where the distance is a longitude distance, the speed change is a first speed change of the target vehicle in longitude, and the target speed change is a first target speed change of the target vehicle in longitude;

[0207] In the case where the distance is a latitude distance, the speed change is a second speed change of the target vehicle at the latitude, and the target speed change is a second target speed change of the target vehicle at the latitude.

[0208] It should be noted here that the above-mentioned device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned data set construction method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0209] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 1 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0210] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing related hardware through a computer program, wherein the computer program includes instructions for executing part or all of the steps of the above method; and the computer program may be stored in a readable storage medium, and the storage medium may be any form of storage medium.

[0211] In addition, a specific embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the method in the first embodiment described above are implemented. The same technical effect can be achieved, and to avoid repetition, it will not be described here.

[0212] In addition, it should be noted that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it is understood that all or any steps or components of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or a network of computing devices in hardware, firmware, software or a combination thereof, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0213] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general device. Therefore, the purpose of the present invention can also be achieved by simply providing a program product containing a program code that implements the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order. Some steps can be performed in parallel or independently of each other.

[0214] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.< / cv::point>

Claims

1. A method for constructing a data set, characterized in that: include: Acquire latitude and longitude information and at least one video frame corresponding to the target road, wherein the latitude and longitude information is acquired by an on-board device installed on the target vehicle, and the at least one video frame is acquired by a roadside device; For each of the video frames, based on the angle influence factor, the pixel coordinates corresponding to the feature points of the target vehicle in the video frame are obtained, the feature points are used to identify the position of the vehicle-mounted device on the target vehicle, and the angle influence factor is used to characterize the degree of influence of the shooting angle on the position of the feature points in the video frame when the roadside device collects the video frame; Obtaining a target data set according to the latitude and longitude information and the pixel coordinates corresponding to the feature points in the video frame; Wherein, obtaining the pixel coordinates corresponding to the feature points of the target vehicle in the video frame based on the angle influence factor includes: Performing target detection on the video frame to obtain position information of a bounding box corresponding to the target vehicle; Determining an angle influence factor corresponding to the video frame according to the position information of the bounding box, the actual size information of the target vehicle, and the installation height of the roadside equipment relative to the ground; According to the position information of the bounding box and the angle influence factor, the pixel coordinates corresponding to the feature points of the target vehicle in the video frame are obtained.

2. The method according to claim 1, characterized in that The determining, according to the position information of the bounding box, the actual size information of the target vehicle, and the installation height of the roadside equipment relative to the ground, the angle influence factor corresponding to the video frame includes: Determine the projection height of the bounding box on the camera target surface according to the position information of the bounding box, the actual size information of the target vehicle and the height of the camera target surface corresponding to the roadside equipment; Determine the horizontal distance between the roadside equipment and the target vehicle according to the projection height, the actual size information of the target vehicle, the installation height and the focal length of the camera corresponding to the roadside equipment; Determining a shooting angle between the roadside equipment and the target vehicle according to the horizontal distance and the installation height; According to the shooting angle, an angle influence factor corresponding to the video frame is determined.

3. The method according to claim 1, characterized in that The step of obtaining a target data set according to the latitude and longitude information and the pixel coordinates corresponding to the feature points in the video frame includes: Filtering the at least one video frame according to pixel coordinates corresponding to the feature points in the video frames to obtain at least one target video frame; A target data set is obtained according to the latitude and longitude information and the pixel coordinates of the feature point in the at least one target video frame.

4. The method according to claim 3, characterized in that The step of screening the at least one video frame according to the pixel coordinates corresponding to the feature points in the video frames to obtain at least one target video frame includes: Traversing the at least one video frame in sequence, and acquiring a first target video frame that meets a first preset condition according to pixel coordinates corresponding to the feature point in the reference video frame; When the first target video frame is acquired, the first target video frame is stored as the target video in a preset set, and the reference video frame is updated to the first target video frame; Wherein, the initial value of the reference video frame is a video frame with the earliest timestamp among the at least one video frame; The first preset condition includes: There is at least M video frames between the first target video frame and the reference video frame, where M is a positive integer; The target distance between the first target video frame and the reference video frame is greater than or equal to a preset value, and the target distance is determined based on the pixel coordinates corresponding to the feature point in the reference video frame, the pixel coordinates corresponding to the feature point in the first target video frame, and the distance factor corresponding to the first target video frame, and the distance factor is used to characterize the distance between the target vehicle and the roadside equipment.

5. The method according to claim 3, characterized in that: The longitude and latitude information includes: the longitude and latitude information of the target vehicle, and the time point of collecting the longitude and latitude information; The step of obtaining a target data set according to the latitude and longitude information and the pixel coordinates of the feature point in the at least one target video frame includes: For each of the target video frames, according to the timestamp corresponding to the target video frame, the target longitude information and the target latitude information corresponding to the target video frame are obtained from the longitude and latitude information, and the time difference between the acquisition time point of the target longitude information and the target latitude information and the timestamp is less than or equal to the first preset duration; The pixel coordinates of the feature points in the target video frame and the target longitude information and target latitude information corresponding to the target video frame are stored in the target data set.

6. The method according to claim 5, characterized in that The method further comprises: In the case where the target longitude information and the target latitude information corresponding to the target video frame are not obtained, obtaining N first longitude and latitude information that meet a second preset condition from the longitude and latitude information according to the timestamp corresponding to the target video frame, wherein the second preset condition includes: the time difference between the first time point of collecting the first longitude and latitude information and the timestamp is less than or equal to a second preset duration, the first time point is earlier than the timestamp, and N is a positive integer; Determine the longitude and latitude information corresponding to the timestamp according to the N first longitude and latitude information; The pixel coordinates of the feature point in the target video frame and the longitude information and latitude information corresponding to the timestamp are stored in the target data set.

7. The method according to claim 6, characterized in that The determining, according to the N first longitude and latitude information, the longitude information and latitude information corresponding to the timestamp includes: For the N track points of the target vehicle corresponding to the N first longitude and latitude information, determine the distance between every two adjacent track points in the N track points according to the N first longitude and latitude information; Determine at least one speed change corresponding to the target vehicle according to the distance and time difference between every two adjacent trajectory points; Determining a target speed change corresponding to the target vehicle according to the at least one speed change and a weight value corresponding to each speed change; Determine the longitude information and latitude information corresponding to the timestamp respectively according to the target speed change and the N first longitude and latitude information; Wherein, the distance includes longitude distance and latitude distance; In the case where the distance is a longitude distance, the speed change is a first speed change of the target vehicle in longitude, and the target speed change is a first target speed change of the target vehicle in longitude; In the case where the distance is a latitude distance, the speed change is a second speed change of the target vehicle at the latitude, and the target speed change is a second target speed change of the target vehicle at the latitude.

8. A data set construction device, characterized in that: include: A first acquisition module is used to acquire the latitude and longitude information corresponding to the target road and at least one video frame, wherein the latitude and longitude information is acquired by an on-board device installed on the target vehicle, and the at least one video frame is acquired by a roadside device; A second acquisition module is used to acquire, for each of the video frames, pixel coordinates corresponding to feature points of the target vehicle in the video frame based on an angle influence factor, wherein the feature point is used to identify the position of the vehicle-mounted device on the target vehicle, and the angle influence factor is used to characterize the degree of influence of the shooting angle on the position of the feature point in the video frame when the roadside device collects the video frame; A first processing module, configured to obtain a target data set according to the latitude and longitude information and the pixel coordinates corresponding to the feature points in the video frame; The second acquisition module includes: The target detection submodule is used to perform target detection on the video frame to obtain the position information of the bounding box corresponding to the target vehicle; A first processing submodule, configured to determine an angle influence factor corresponding to the video frame according to the position information of the bounding box, the actual size information of the target vehicle, and the installation height of the roadside equipment relative to the ground; The second processing submodule is used to obtain the pixel coordinates corresponding to the feature points of the target vehicle in the video frame according to the position information of the bounding box and the angle influence factor.

9. A data processing device comprising: A transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data set construction method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the data set construction method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data set construction method according to any one of claims 1 to 7 are implemented.

Images