Track generation method and device, electronic equipment and storage medium

Abstract

The invention provides a track generation method and device, electronic equipment and a storage medium, and relates to the technical field of monitoring, and the method comprises the steps: obtaining a video set of videos shot by at least one camera device for a target area; based on the positions of all the camera devices and the timestamps of the videos shot by the camera devices, determining a camera device point location set; determining a geographic coordinate corresponding to a target pixel point included in each frame of target image in the video set, and determining a geographic coordinate set based on the geographic coordinate corresponding to each target pixel point and the shooting moment of the corresponding target image; compensating the camera equipment point position set based on the geographic coordinate set; and generating a track of the target object based on the compensated point position set. The method not only considers the camera equipment point location set, but also compensates the error introduced by the inaccurate camera equipment point location through the geographic coordinate set, thereby improving the accuracy of track generation.

Description

Technical Field

[0001] The present invention relates to the field of monitoring technology, and in particular to a trajectory generation method, device, electronic equipment and storage medium. Background Art

[0002] In urban management, video equipment is used to monitor movable objects in the city. The historical trajectory of the movable objects is restored through different dimensions (time dimension, space dimension) based on the video captured by the video equipment, so that the trajectory of the movable objects can be traced, providing strong evidence for urban event analysis.

[0003] In the related art, the geographical location (point coordinates) of each camera device is usually used to connect the videos including movable objects captured by each camera device in chronological order, and then connect the positions of the movable objects into lines based on the connected videos to generate a trajectory, which is displayed on a map.

[0004] However, in the above-mentioned related technologies, the trajectory is formed only based on the geographical location of each camera device, which will cause errors in the generated trajectory, thereby reducing the accuracy of trajectory generation. Summary of the invention

[0005] The present invention provides a trajectory generation method, device, electronic device and storage medium, which are used to solve the defect of reducing the accuracy of trajectory generation in the prior art.

[0006] The present invention provides a trajectory generation method, comprising: Acquire a video set of videos shot by at least one camera device for a target area within a preset time period; Determine a camera device point set based on the positions of all the camera devices and the timestamps of the videos shot by the camera devices; Determine the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determine a geographic coordinate set based on the geographic coordinates corresponding to each of the target pixel points and the shooting time of the corresponding target image, wherein the target image is an image including the target object, and the target pixel points are pixel points on the ground where the target object is located in the target image; Compensating the camera device point set based on the geographic coordinate set to obtain a compensated point set; Based on the compensated point set, a trajectory of the target object is generated.

[0007] According to a trajectory generation method provided by the present invention, the determining of the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set includes: For each frame of the target image, based on installation information of a target camera device corresponding to the target image and size information of the target image, determine the Cartesian coordinates of a target pixel point included in the target image in a target camera device coordinate system; Convert the Cartesian coordinates into azimuth coordinates in the geographic azimuth coordinate system of the target camera device; The azimuth coordinates are converted into the geographic coordinates.

[0008] According to a trajectory generation method provided by the present invention, the installation information of the target camera device includes the height of the target camera device and the pitch angle of the target camera device, and the size information of the target image includes the width of the target image and the height of the target image; The determining, based on installation information of a target camera device corresponding to the target image and size information of the target image, Cartesian coordinates of a target pixel point included in the target image in a coordinate system of the target camera device comprises: Determine a first deflection angle of the target pixel in a vertical direction in a target camera device coordinate system based on a height of the target image, a width of the target image, and a longitudinal coordinate of a target pixel included in the target image; Determine a target pitch angle of the target pixel point in a target camera device coordinate system based on the pitch angle of the target camera device and the first deflection angle; Based on the target pitch angle and the height of the target camera device, determining the ordinate of the target pixel point included in the target image in the target camera device coordinate system; Based on the ordinate, the Cartesian coordinate is determined.

[0009] According to a trajectory generation method provided by the present invention, the installation information of the target camera device also includes the horizontal field of view angle of the target camera device; The determining the Cartesian coordinate based on the ordinate comprises: Determine a second deflection angle of the target pixel in the horizontal direction in the target camera device coordinate system based on the width of the target image, the horizontal coordinate of the target pixel included in the target image, and the horizontal field angle of the target camera device; Based on the second deflection angle and the vertical coordinate, determining the horizontal coordinate of the target pixel point included in the target image in the target camera device coordinate system; The ordinate and the abscissa are determined as the Cartesian coordinates.

[0010] According to a trajectory generation method provided by the present invention, the converting the azimuth coordinates into the geographic coordinates comprises: Determine, based on the azimuth coordinates, the distance between the target point of the target pixel point in the geographic azimuth coordinate system and the target camera device and the azimuth angle of the target point relative to the target camera device; The geographic coordinates are determined based on the distance, the azimuth, the longitude of the target imaging device, and the latitude of the target imaging device.

[0011] According to a trajectory generation method provided by the present invention, the trajectory of the target object is generated based on the compensated point set, including: Performing denoising and thinning processing on the points in the compensated point set to obtain a processed point set; Based on the processed point set, a trajectory of the target object is generated.

[0012] According to a trajectory generation method provided by the present invention, generating the trajectory of the target object based on the processed point set includes: Acquire all road coordinate data in the target area; Based on all the road coordinate data, correcting the points in the processed point set that are not located on the road to obtain a target point set; Based on the target point set, a trajectory of the target object is generated.

[0013] The present invention also provides a trajectory generating device, comprising: An acquisition unit, used to acquire a video set of videos shot by at least one camera device for a target area within a preset time period; A first determining unit, configured to determine a camera device point set based on positions of all the camera devices and timestamps of videos shot by each of the camera devices; A second determining unit is used to determine the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determine a set of geographic coordinates based on the geographic coordinates corresponding to each of the target pixel points and the shooting time of the corresponding target image, wherein the target image is an image including the target object, and the target pixel points are pixel points on the ground where the target object is located in the target image; A compensation unit, configured to compensate the camera point set based on the geographic coordinate set to obtain a compensated point set; A generating unit is used to generate a trajectory of the target object based on the compensated point set.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the above-mentioned trajectory generation methods is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the trajectory generation method described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned trajectory generation methods.

[0017] The trajectory generation method, device, electronic device and storage medium provided by the present invention obtain a video set of videos shot by at least one camera device for a target area within a preset time period, determine a camera point set based on the positions of all camera devices and the timestamps of the videos shot by each camera device, determine the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determine the geographic coordinate set based on the geographic coordinates corresponding to each target pixel point and the shooting time of the corresponding target image, compensate the camera point set based on the geographic coordinate set to obtain a compensated point set, and finally generate the trajectory of the target object based on the compensated point set. It can be seen that the present invention not only considers the camera point set, but also determines the geographic coordinate set based on the geographic coordinates corresponding to the target pixel points on the ground where the target object is located in each target image, and compensates the errors introduced by inaccurate camera point positions through the geographic coordinate set, thereby improving the accuracy of trajectory generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is one of the flow charts of the trajectory generation method provided by an embodiment of the present invention.

[0020] Figure 2 This is the second flowchart of the trajectory generation method provided by the embodiment of the present invention.

[0021] Figure 3 It is a schematic diagram of the geographic orientation coordinate system of the target camera device provided by an embodiment of the present invention.

[0022] Figure 4 Schematic diagram of a target camera device coordinate system provided by an embodiment of the present invention.

[0023] Figure 5It is a schematic diagram of the ordinate of a target pixel point in a target camera device coordinate system provided by an embodiment of the present invention.

[0024] Figure 6 It is a schematic diagram of the horizontal coordinate of a target pixel point in a target camera device coordinate system provided by an embodiment of the present invention.

[0025] Figure 7 It is a schematic diagram of the trajectory of the target object before and after correction provided by an embodiment of the present invention.

[0026] Figure 8 It is a structural schematic diagram of a trajectory generating device provided by an embodiment of the present invention.

[0027] Fig. 9 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] In the related technology, the geographical location (point coordinates) of each camera device is usually used to connect the videos including movable objects collected by each camera device in chronological order, and then connect the positions of the movable objects into lines based on the connected videos to generate trajectories and display them on the map. Specifically, first install a camera device for shooting videos at key traffic checkpoints, communities, key areas or other scenes that need to be monitored, and record the geographical location of the current camera device. Then extract all video data of movable objects appearing within the monitoring range within a certain period of time, find each camera device corresponding to the video data, and obtain the geographical location of each camera device. Sort the geographical location of each camera device according to the time of the movable object to form an orderly point string. Use map drawing technology to connect the geographical locations in order and display them on the map.

[0030] However, the relevant technology only considers the geographical location of each camera device in the video, and lacks consideration of the continuous and real geographical coordinates of the movable object within a specific time. It is impossible to present the physical spatial position of the movable object at multiple times in detail. The trajectory presentation accuracy is low and the error is large. There will be many situations that are not in line with reality and common sense, such as: trajectory "jumping" (drifting / interruption), "crossing" at road corners, and trajectory "stacking" at the residence. In addition, for closed roads such as highways, elevated roads, and tunnels, it is also necessary to distinguish the impact of traffic behaviors such as motor vehicles, non-motor vehicles, and walking on the trajectory correction effect, thereby reducing the accuracy of trajectory generation.

[0031] Based on this, the present invention provides a trajectory generation method, which not only considers the camera point set, but also determines the geographic coordinate set based on the geographic coordinates corresponding to the target pixel points on the ground where the target object is located in each target image, and compensates for the errors introduced by inaccurate camera points through the geographic coordinate set, thereby improving the accuracy of trajectory generation.

[0032] Combine the following Figure 1-Figure 7 The trajectory generation method of the present invention is described. The execution subject of the trajectory generation method can be an electronic device such as a terminal or a computer, or a trajectory generation device set in the electronic device, and the trajectory generation device can be implemented by software, hardware, or a combination of the two. The present invention can be applied to scenarios such as location service engines, route planning engines, and trajectory tracing in super brain business systems.

[0033] Figure 1 is one of the flow charts of the trajectory generation method provided by an embodiment of the present invention, such as Figure 1 As shown, the trajectory generation method includes the following steps: Step 101: Obtain a video set of videos shot by at least one camera device within a preset time period for a target area.

[0034] For example, the video set V={V1,...,Vn} of all video data of the target object A captured by each camera device in the target area within a preset time period is queried. The set of preset time periods of all cameras in the target area can be expressed as [T1,...,Tn], where n represents the number of cameras.

[0035] Step 102: Determine a camera device point set based on the positions of all the camera devices and the timestamps of the videos shot by each camera device.

[0036] For example, when each camera device is installed, the position of each camera device is recorded, and when the camera device shoots a video, the timestamp corresponding to the shot video is also recorded. Therefore, based on the positions of all camera devices in the target area and the timestamps of the videos shot by each camera device, the camera device point set L=[L1,..., Ln] can be determined, where Ln=(longitude of the camera device, latitude of the camera device, timestamp of the camera device shooting).

[0037] It should be noted that when installing each camera device, it is necessary to ensure that the location of the camera device is not too far from the road, and the error is controlled within a preset distance range (for example, 5 meters to 10 meters). If it exceeds the preset distance range, in order to ensure the accuracy of subsequent trajectory correction, it is necessary to add roads, that is, connect the camera equipment and the main road according to the actual situation to form a branch road.

[0038] Step 103, determine the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determine a geographic coordinate set based on the geographic coordinates corresponding to each of the target pixel points and the shooting time of the corresponding target image, wherein the target image is an image including the target object, and the target pixel point is a pixel point on the ground where the target object is located in the target image.

[0039] For example, when a video set V={V1,...,Vn} is obtained, a key frame image set P={P1,...,Pm} is extracted from the time when the target object A starts to appear in the video set V={V1,...,Vn} and ends. Since there may be at least one target image including the target object A in the video data of each camera device, m is an integer greater than or equal to n. Each frame of the key frame image set is used as the target image, and the geographic coordinates corresponding to the target pixel point on the ground where the target object A is located in each frame of the target image are calculated. Then, based on the geographic coordinates corresponding to each target pixel point and the shooting time (timestamp) of the target image where the target pixel point is located, a geographic coordinate set N={N1,...,Nm} is determined, where Nm=(longitude of the target pixel point, latitude of the target pixel point, timestamp of the target image). The process of determining the geographic coordinate set is called geo-registration. The purpose of extracting the target image including the target object A is to calculate the real geographic coordinates corresponding to the target pixel point on the ground where the target object A is located in the target image, and to remove the objects in the video set that are not related to the target object A, so as to ensure the accuracy of the subsequent pixel acquisition.

[0040] It should be noted that the process of extracting key frame images from a video set may refer to the video analysis and extraction technology in the related art, and the present invention will not be repeated here.

[0041] Step 104: Compensate the camera point set based on the geographic coordinate set to obtain a compensated point set.

[0042] For example, the geographic coordinate set N={N1,...,Nm} is used as the compensation of the target object A in the camera point set L=[L1,..., Ln]. It is necessary to determine the time order according to the timestamp in the geographic coordinate set and the timestamp in the camera point set, and insert each point of the geographic coordinate set N={N1,...,Nm} into the camera point set L=[L1,..., Ln] in sequence based on the time order to form a compensated point set K={K1,…,Kg}, where Kg=(longitude, latitude, timestamp), and g=n+m.

[0043] Step 105: Generate a trajectory of the target object based on the compensated point set.

[0044] For example, when a compensated point set is obtained, each point in the compensated point set is marked on the map according to the longitude and latitude using existing mapping technology, and then each point is connected in a time sequence determined by the timestamp to obtain the trajectory of the target object A, and restore the true trajectory of the target object A within the preset time period.

[0045] The trajectory generation method provided by the present invention obtains a video set of videos shot by at least one camera device for a target area within a preset time period, determines a camera point set based on the positions of all camera devices and the timestamps of the videos shot by each camera device, determines the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determines the geographic coordinate set based on the geographic coordinates corresponding to each target pixel point and the shooting time of the corresponding target image, compensates the camera point set based on the geographic coordinate set to obtain a compensated point set, and finally generates the trajectory of the target object based on the compensated point set. It can be seen that the present invention not only considers the camera point set, but also determines the geographic coordinate set based on the geographic coordinates corresponding to the target pixel points on the ground where the target object is located in each target image, and compensates the errors introduced by inaccurate camera point positions through the geographic coordinate set, thereby improving the accuracy of trajectory generation.

[0046] In one embodiment, Figure 2 FIG. 2 is a flow chart of the trajectory generation method provided by an embodiment of the present invention. Figure 2 As shown, the above step 103 determines the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, which can be specifically implemented by the following steps: Step 1031 : for each frame of the target image, based on installation information of the target camera device corresponding to the target image and size information of the target image, determine the Cartesian coordinates of the target pixel points included in the target image in the target camera device coordinate system.

[0047] For example, when obtaining a video set of videos shot by at least one camera device in the target area, it is also necessary to obtain the installation information of each camera device. The video data of the camera device corresponds to the installation information of the camera device one by one. The installation information of the camera device includes: the geographic coordinates of the location of the camera device (that is, the location of the camera device mentioned above), the azimuth of the camera device, the pitch angle of the camera device, and the horizontal field of view angle HFov of the camera device. The geographic coordinates of the location of the camera device include the longitude and latitude of the location of the camera device, and the altitude at which the camera device is installed. The azimuth of the camera device is the horizontal angle from the north direction line of the point where the camera device is installed to the boundary line of the range observed by the camera device in a clockwise direction, and is expressed in degrees. For fixed camera devices, the azimuth has been determined when it is installed. The pitch angle of the camera device refers to the angle between the camera device and the horizontal plane. When the camera device is shooting on the horizontal plane, the pitch angle is 0 degrees. When the camera device is shooting from above, the pitch angle is a negative value. When the camera device is shooting from above, the pitch angle is -90 degrees. The azimuth angle is determined when the camera device is set up. The horizontal field of view HFov of the camera device refers to the maximum field of view of the video screen in the horizontal direction of the camera device. It is expressed in degrees and can be directly obtained by querying the parameters of the camera device product.

[0048] For each frame of the target image, using the existing image parsing library, the width Width of the target image and the height High of the target image can be obtained. Combined with the installation information of the target camera device corresponding to the target image, the width Width of the target image and the height High of the target image, the Cartesian coordinates Q (qx, qy) of the target pixel point included in the target image in the coordinate system of the target camera device can be determined.

[0049] Step 1032: convert the Cartesian coordinates into azimuth coordinates in the geographic azimuth coordinate system of the target camera device.

[0050] For example, when the Cartesian coordinates Q (qx, qy) of the target pixel point in the coordinate system of the target camera device are obtained, it is necessary to further convert the Cartesian coordinates Q (qx, qy) into the geographic orientation coordinate system of the target camera device. Figure 3 is a schematic diagram of a geographic orientation coordinate system of a target camera device provided by an embodiment of the present invention, such as Figure 3As shown, the geographic orientation coordinate system of the target camera device is established with the east direction as the X axis and the north direction as the Y axis when the target camera device is set up. Specifically, based on the following formulas (1) and (2), the Cartesian coordinates of the target pixel point in the coordinate system of the target camera device are converted into the orientation coordinates M (mx, my) in the geographic orientation coordinate system of the target camera device.

[0051]

[0052]

[0053] Step 1033: Convert the azimuth coordinates into the geographic coordinates.

[0054] For example, when the azimuth coordinate M (mx, my) of the target camera device in the geographic azimuth coordinate system is obtained, it is necessary to further convert the azimuth coordinate M (mx, my) of the target camera device in the geographic azimuth coordinate system into the geographic coordinate N (nx, ny) corresponding to the target pixel point.

[0055] In this embodiment, based on the installation information of the target camera device corresponding to the target image and the size information of the target image, the Cartesian coordinates of the target pixel point in the target camera device coordinate system are converted into the geographic coordinates corresponding to the target pixel point, thereby realizing the automatic determination of the geographic coordinates corresponding to the target pixel point, and further improving the efficiency of subsequent trajectory generation.

[0056] In one embodiment, the installation information of the target camera device includes the height of the target camera device and the pitch angle of the target camera device, and the size information of the target image includes the width of the target image and the height of the target image; the above step 1031 determines the Cartesian coordinates of the target pixel points included in the target image in the target camera device coordinate system based on the installation information of the target camera device corresponding to the target image and the size information of the target image, which can be specifically implemented in the following manner: Based on the height of the target image, the width of the target image and the longitudinal coordinate of the target pixel included in the target image, determine the first deflection angle of the target pixel in the vertical direction in the target camera device coordinate system; based on the pitch angle of the target camera device and the first deflection angle, determine the target pitch angle of the target pixel in the target camera device coordinate system; based on the target pitch angle and the height of the target camera device, determine the longitudinal coordinate of the target pixel included in the target image in the target camera device coordinate system; based on the longitudinal coordinate, determine the Cartesian coordinate.

[0057] For example, a target camera device coordinate system is defined, including a coordinate origin, a horizontal X-axis direction, and a vertical Y-axis direction. Figure 4is a schematic diagram of a target camera device coordinate system provided by an embodiment of the present invention, such as Figure 4 As shown, the installation position of the camera device is taken as the coordinate origin, the horizontal direction pointing to the right is the X axis, and the vertical direction pointing upward is the Y axis. The coordinates of the target camera device coordinate system are Cartesian coordinates, and the unit is meter. For each frame of the target image, the coordinates (px, py) of the target pixel point p on the ground where the target object A is located are extracted using the existing image analysis algorithm, and based on the height of the target camera device, the pitch angle of the target camera device, and the coordinates of the target pixel point p, a schematic diagram of the ordinate of the target pixel point in the target camera device coordinate system is drawn. Figure 5 is a schematic diagram of the ordinate of a target pixel point in a target camera device coordinate system provided by an embodiment of the present invention, such as Figure 5 As shown, according to Figure 5 The geometric figures shown can be used to calculate the distance OQ between point Q and the origin O of the camera coordinate system based on the following formulas (3) to (5), that is, the ordinate qy of the target pixel point p in the target camera coordinate system, and then use the ordinate qy as the ordinate of the Cartesian coordinate.

[0058]

[0059] Wherein, VDeflection represents the first deflection angle of the target pixel point in the vertical direction in the target camera device coordinate system, and QPitch represents the target pitch angle of the target pixel point in the target camera device coordinate system.

[0060] It should be noted that Figure 5 The distance between Min and Max refers to the distance that can be captured along the line of sight of the camera device.

[0061] In this embodiment, the target pitch angle of the target pixel point in the target camera device coordinate system can be determined based on the pitch angle of the target camera device and the first deflection angle of the target pixel point in the vertical direction in the target camera device coordinate system, and then the vertical coordinate of the target pixel point in the target camera device coordinate system can be determined based on the target pitch angle and the height of the target camera device, thereby realizing automatic determination of the vertical coordinate and further improving the efficiency of subsequent trajectory generation.

[0062] In one embodiment, the installation information of the target camera device further includes the horizontal field of view angle of the target camera device; the above-mentioned determination of the Cartesian coordinate based on the longitudinal coordinate can be specifically implemented in the following manner: Based on the width of the target image, the horizontal coordinate of the target pixel included in the target image and the horizontal field angle of the target camera device, determine the second deflection angle of the target pixel in the horizontal direction in the coordinate system of the target camera device; based on the second deflection angle and the vertical coordinate, determine the horizontal coordinate of the target pixel included in the target image in the coordinate system of the target camera device; determine the vertical coordinate and the horizontal coordinate as the Cartesian coordinates.

[0063] For example, based on the horizontal field angle of the target camera device and the coordinates of the target pixel point p, a schematic diagram of the horizontal coordinate of the target pixel point in the coordinate system of the target camera device is drawn. Figure 6 is a schematic diagram of the horizontal coordinate of the target pixel point in the target camera device coordinate system provided by an embodiment of the present invention, such as Figure 6 As shown, according to Figure 6 The geometric figure shown can be used to calculate the distance of point Q from the Y axis of the target camera coordinate system in the target camera coordinate system based on the following formulas (6) and (7), that is, the horizontal coordinate qx of the target pixel point p in the target camera coordinate system, and then the combination of the horizontal coordinate qx and the vertical coordinate qy is used as the Cartesian coordinate Q (qx, qy).

[0064] (6)

[0065] (7)

[0066] Wherein, HDeflection represents the second deflection angle.

[0067] In this embodiment, the horizontal coordinate of the target pixel point in the target camera device coordinate system can be determined based on the second deflection angle in the horizontal direction of the target pixel point in the target camera device coordinate system and the vertical coordinate of the target pixel point in the target camera device coordinate system, thereby realizing automatic determination of the horizontal coordinate and further improving the efficiency of subsequent trajectory generation.

[0068] In one embodiment, the above step 1033 converts the azimuth coordinates into the geographic coordinates, which can be specifically implemented in the following manner: Based on the azimuth coordinates, determine the distance between the target point of the target pixel point in the geographic azimuth coordinate system and the target camera device and the azimuth of the target point relative to the target camera device; based on the distance, the azimuth, the longitude of the target camera device and the latitude of the target camera device, determine the geographic coordinates.

[0069] For example, when the azimuth coordinate M (mx, my) of the target camera device in the geographic azimuth coordinate system is obtained, since the azimuth coordinate M (mx, my) is the projection coordinate in the measurement screen, the unit is meter, so the azimuth coordinate M (mx, my) needs to be further converted to obtain the real geographic coordinate N (nx, ny), the unit of which is longitude and latitude. Specifically, the distance length between the target pixel point and the target camera device in the geographic azimuth coordinate system is determined based on the following formula (8), and the azimuth angle alpha of the target point relative to the target camera device is determined based on the following formula (9), and finally the geographic coordinate N (nx, ny) is determined based on the following formula (10) and formula (11).

[0070] (8)

[0071] (9)

[0072] (10)

[0073] (11)

[0074] Among them, PI represents pi, and 6371393 represents the radius of the earth, in meters.

[0075] In this embodiment, the distance between the target point and the target camera device in the geographic azimuth coordinate system and the azimuth of the target point relative to the target camera device are determined based on the azimuth coordinates of the target pixel point, and the geographic coordinates are determined based on the distance, the azimuth, the longitude of the target camera device and the latitude of the target camera device, thereby realizing automatic determination of the geographic coordinates and further improving the efficiency of subsequent trajectory generation.

[0076] In one embodiment, the above step 105 generates the trajectory of the target object based on the compensated point set, which can be specifically implemented in the following manner: De-noising and thinning are performed on the points in the compensated point set to obtain a processed point set; and based on the processed point set, a trajectory of the target object is generated.

[0077] For example, when the compensated point set K={K1,…,Kg} is calculated, since the corresponding geographic coordinates are inferred based on the target pixel points of the target image, there may be errors caused by external factors (the longitude and latitude errors assumed by the camera device, the horizontal field of view angle or vertical field of view angle error of the camera device, the longitude and latitude inferred from the earth's surface distance, etc.). Therefore, there will be noise data in the compensated point set K={K1,…,Kg}, and there are too many target images including the target object A, resulting in too many trajectory points for the target object A. Therefore, the points in the compensated point set K={K1,…,Kg} need to be denoised and thinned out. For specific denoising and thinning processing, please refer to relevant technologies. For example, duplicate points can be removed and the Douglas algorithm can be used to thin trajectory data according to the distance threshold, so as to finally form a noise-free, smooth, and ordered processed point set S={S1,…,Sh}, where Sh=(longitude, latitude, timestamp). Each point of the processed point set S={S1,…,Sh} is marked on the map according to the longitude and latitude, and then each point is connected in the time order determined by the timestamp to obtain the trajectory of the target object.

[0078] In this embodiment, the points in the compensated point set are denoised and thinned to obtain a processed point set, and the trajectory of the target object is generated based on the processed point set to avoid errors caused by external factors, thereby improving the accuracy of the determined processed point set and further improving the accuracy of trajectory generation.

[0079] In one embodiment, the above-mentioned generating the trajectory of the target object based on the processed point set can be specifically implemented in the following manner: Acquire all road coordinate data in the target area; based on all the road coordinate data, correct the points in the processed point set that are not located on the road to obtain a target point set; based on the target point set, generate the trajectory of the target object.

[0080] For example, it is also necessary to obtain all road coordinate data in the target area, and the data format of the road coordinate data can be the ShapFile format commonly used for geographic information. Since some points in the processed point set S={S1,…,Sh} may not be on the road, it is necessary to combine all road coordinate data of the target object A within the target area, and use the Hidden Markov Model (HMM) correction algorithm to correct the points in the processed point set that are not on the road, that is, match the points that are not on the road to the road to obtain the final target point set, and then mark each point in the target point set on the map according to longitude and latitude, and then connect each point in the time order determined by the timestamp to obtain the trajectory of the target object. Figure 7is a schematic diagram of the trajectory of the target object before and after correction provided by an embodiment of the present invention, such as Figure 7 As shown, the trajectory on the left is the trajectory generated based on the processed point set S = {S1, ..., Sh}, and the trajectory on the right is the trajectory generated based on the target point set, matching the points that are not on the road to the road.

[0081] In this embodiment, based on all road coordinate data in the target area, the points in the processed point set that are not located on the road are corrected to obtain the target point set, and the trajectory of the target object is generated based on the target point set, which further improves the accuracy of trajectory generation and makes it more consistent with the actual trajectory or expected behavior, thereby achieving high-fidelity restoration and presentation of the trajectory of the target object in the urban scene, and improving the fidelity of digital mapping interaction and analysis applications of the trajectory of the target object in the physical world to the digital world.

[0082] The trajectory generation method provided by the embodiment of the present invention not only considers the camera point set, but also extracts the real geographic coordinates of the target pixel point in the target image by geo-referencing the video set, so as to correct or compensate for the errors and deviations introduced by the inaccurate camera position or other factors, and improves the accuracy and consistency of the trajectory data of the target object by combining the correction and denoising of the trajectory points.

[0083] The trajectory generating device provided by the present invention is described below. The trajectory generating device described below and the trajectory generating method described above can be referred to each other.

[0084] Figure 8 is a schematic diagram of the structure of a trajectory generating device provided by an embodiment of the present invention. Figure 8 The trajectory generating device 800 includes an acquiring unit 801, a first determining unit 802, a second determining unit 803, a compensating unit 804 and a generating unit 805; wherein: An acquisition unit 801 is used to acquire a video set of videos shot by at least one camera device for a target area within a preset time period; A first determining unit 802 is used to determine a camera device point set based on the positions of all the camera devices and the timestamps of the videos shot by the camera devices; The second determining unit 803 is used to determine the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determine the geographic coordinate set based on the geographic coordinates corresponding to each of the target pixel points and the shooting time of the corresponding target image, wherein the target image is an image including the target object, and the target pixel points are the pixel points of the ground where the target object is located in the target image; A compensation unit 804 is used to compensate the camera point set based on the geographic coordinate set to obtain a compensated point set; The generating unit 805 is configured to generate a trajectory of the target object based on the compensated point set.

[0085] The trajectory generation device provided by the present invention obtains a video set of videos shot by at least one camera device for a target area within a preset time period, determines a camera device point set based on the positions of all camera devices and the timestamps of the videos shot by each camera device, determines the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determines the geographic coordinate set based on the geographic coordinates corresponding to each target pixel point and the shooting time of the corresponding target image, compensates the camera device point set based on the geographic coordinate set to obtain a compensated point set, and finally generates the trajectory of the target object based on the compensated point set. It can be seen that the present invention not only considers the camera device point set, but also determines the geographic coordinate set based on the geographic coordinates corresponding to the target pixel points on the ground where the target object is located in each target image, and compensates the errors introduced by inaccurate camera device points through the geographic coordinate set, thereby improving the accuracy of trajectory generation.

[0086] Based on any of the foregoing embodiments, the second determining unit 803 is specifically configured to: For each frame of the target image, based on installation information of a target camera device corresponding to the target image and size information of the target image, determine the Cartesian coordinates of a target pixel point included in the target image in a target camera device coordinate system; Convert the Cartesian coordinates into azimuth coordinates in the geographic azimuth coordinate system of the target camera device; The azimuth coordinates are converted into the geographic coordinates.

[0087] Based on any of the above embodiments, the installation information of the target camera device includes the height of the target camera device and the pitch angle of the target camera device, and the size information of the target image includes the width of the target image and the height of the target image; the second determining unit 803 is further specifically used to: Determine a first deflection angle of the target pixel in a vertical direction in a target camera device coordinate system based on a height of the target image, a width of the target image, and a longitudinal coordinate of a target pixel included in the target image; Determine a target pitch angle of the target pixel point in a target camera device coordinate system based on the pitch angle of the target camera device and the first deflection angle; Based on the target pitch angle and the height of the target camera device, determining the ordinate of the target pixel point included in the target image in the target camera device coordinate system; Based on the ordinate, the Cartesian coordinate is determined.

[0088] Based on any of the above embodiments, the installation information of the target camera device further includes a horizontal field of view angle of the target camera device; and the second determining unit 803 is further specifically configured to: Determine a second deflection angle of the target pixel in the horizontal direction in the target camera device coordinate system based on the width of the target image, the horizontal coordinate of the target pixel included in the target image, and the horizontal field angle of the target camera device; Based on the second deflection angle and the vertical coordinate, determining the horizontal coordinate of the target pixel point included in the target image in the target camera device coordinate system; The ordinate and the abscissa are determined as the Cartesian coordinates.

[0089] Based on any of the foregoing embodiments, the second determining unit 803 is further specifically configured to: Determine, based on the azimuth coordinates, the distance between the target point of the target pixel point in the geographic azimuth coordinate system and the target camera device and the azimuth angle of the target point relative to the target camera device; The geographic coordinates are determined based on the distance, the azimuth, the longitude of the target imaging device, and the latitude of the target imaging device.

[0090] Based on any of the above embodiments, the generating unit 805 is specifically used for: Performing denoising and thinning processing on the points in the compensated point set to obtain a processed point set; Based on the processed point set, a trajectory of the target object is generated.

[0091] Based on any of the above embodiments, the generating unit 805 is further specifically configured to: Acquire all road coordinate data in the target area; Based on all the road coordinate data, correcting the points in the processed point set that are not located on the road to obtain a target point set; Based on the target point set, a trajectory of the target object is generated.

[0092] Fig. 9 is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Fig. 9As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the trajectory generation method, which includes: obtaining a video set of videos shot by at least one camera device for a target area within a preset time period; Determine a camera device point set based on the positions of all the camera devices and the timestamps of the videos shot by the camera devices; Determine the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determine a geographic coordinate set based on the geographic coordinates corresponding to each of the target pixel points and the shooting time of the corresponding target image, wherein the target image is an image including the target object, and the target pixel points are pixel points on the ground where the target object is located in the target image; Compensating the camera device point set based on the geographic coordinate set to obtain a compensated point set; Based on the compensated point set, a trajectory of the target object is generated.

[0093] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0094] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the trajectory generation method provided by the above methods, the method comprising: obtaining a video set of videos shot by at least one camera device for a target area within a preset time period; Determine a camera device point set based on the positions of all the camera devices and the timestamps of the videos shot by the camera devices; Determine the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determine a geographic coordinate set based on the geographic coordinates corresponding to each of the target pixel points and the shooting time of the corresponding target image, wherein the target image is an image including the target object, and the target pixel points are pixel points on the ground where the target object is located in the target image; Compensating the camera device point set based on the geographic coordinate set to obtain a compensated point set; Based on the compensated point set, a trajectory of the target object is generated.

[0095] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the trajectory generation method provided by the above methods, the method comprising: obtaining a video set of videos shot by at least one camera device for a target area within a preset time period; Determine a camera device point set based on the positions of all the camera devices and the timestamps of the videos shot by the camera devices; Determine the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determine a geographic coordinate set based on the geographic coordinates corresponding to each of the target pixel points and the shooting time of the corresponding target image, wherein the target image is an image including the target object, and the target pixel points are pixel points on the ground where the target object is located in the target image; Compensating the camera device point set based on the geographic coordinate set to obtain a compensated point set; Based on the compensated point set, a trajectory of the target object is generated.

[0096] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A trajectory generation method, characterized in that: include: Acquire a video set of videos shot by at least one camera device for a target area within a preset time period; Determine a camera device point set based on the positions of all the camera devices and the timestamps of the videos shot by the camera devices; Determine the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determine a geographic coordinate set based on the geographic coordinates corresponding to each of the target pixel points and the shooting time of the corresponding target image, wherein the target image is an image including the target object, and the target pixel points are pixel points on the ground where the target object is located in the target image; Compensating the camera device point set based on the geographic coordinate set to obtain a compensated point set; Based on the compensated point set, a trajectory of the target object is generated.

2. The trajectory generation method according to claim 1, characterized in that: The determining of the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set includes: For each frame of the target image, based on installation information of a target camera device corresponding to the target image and size information of the target image, determine the Cartesian coordinates of a target pixel point included in the target image in a target camera device coordinate system; Convert the Cartesian coordinates into azimuth coordinates in the geographic azimuth coordinate system of the target camera device; The azimuth coordinates are converted into the geographic coordinates.

3. The trajectory generation method according to claim 2, characterized in that: The installation information of the target camera device includes the height of the target camera device and the pitch angle of the target camera device, and the size information of the target image includes the width of the target image and the height of the target image; The determining, based on installation information of a target camera device corresponding to the target image and size information of the target image, Cartesian coordinates of a target pixel point included in the target image in a coordinate system of the target camera device comprises: Determine a first deflection angle of the target pixel in a vertical direction in a target camera device coordinate system based on a height of the target image, a width of the target image, and a longitudinal coordinate of a target pixel included in the target image; Determine a target pitch angle of the target pixel point in a target camera device coordinate system based on the pitch angle of the target camera device and the first deflection angle; Based on the target pitch angle and the height of the target camera device, determining the ordinate of the target pixel point included in the target image in the target camera device coordinate system; Based on the ordinate, the Cartesian coordinate is determined.

4. The trajectory generation method according to claim 3, characterized in that: The installation information of the target camera device also includes the horizontal field of view angle of the target camera device; The determining the Cartesian coordinate based on the ordinate comprises: Determine a second deflection angle of the target pixel in the horizontal direction in the target camera device coordinate system based on the width of the target image, the horizontal coordinate of the target pixel included in the target image, and the horizontal field angle of the target camera device; Based on the second deflection angle and the vertical coordinate, determining the horizontal coordinate of the target pixel point included in the target image in the target camera device coordinate system; The ordinate and the abscissa are determined as the Cartesian coordinates.

5. The trajectory generation method according to claim 2, characterized in that: The converting the azimuth coordinates into the geographic coordinates comprises: Determine, based on the azimuth coordinates, the distance between the target point of the target pixel point in the geographic azimuth coordinate system and the target camera device and the azimuth angle of the target point relative to the target camera device; The geographic coordinates are determined based on the distance, the azimuth, the longitude of the target imaging device, and the latitude of the target imaging device.

6. The trajectory generation method according to any one of claims 1 to 5, characterized in that: The step of generating the trajectory of the target object based on the compensated point set includes: Performing denoising and thinning processing on the points in the compensated point set to obtain a processed point set; Based on the processed point set, a trajectory of the target object is generated.

7. The trajectory generation method according to claim 6, characterized in that: The generating the trajectory of the target object based on the processed point set includes: Acquire all road coordinate data in the target area; Based on all the road coordinate data, correcting the points in the processed point set that are not located on the road to obtain a target point set; Based on the target point set, a trajectory of the target object is generated.

8. A trajectory generating device, characterized in that: include: An acquisition unit, used to acquire a video set of videos shot by at least one camera device for a target area within a preset time period; A first determining unit, configured to determine a camera device point set based on positions of all the camera devices and timestamps of videos shot by each of the camera devices; A second determining unit is used to determine the geographic coordinates corresponding to the target pixel points included in each frame of the target image in the video set, and determine a set of geographic coordinates based on the geographic coordinates corresponding to each of the target pixel points and the shooting time of the corresponding target image, wherein the target image is an image including the target object, and the target pixel points are pixel points on the ground where the target object is located in the target image; A compensation unit, configured to compensate the camera point set based on the geographic coordinate set to obtain a compensated point set; A generating unit is used to generate a trajectory of the target object based on the compensated point set.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the trajectory generation method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the trajectory generation method according to any one of claims 1 to 7 is implemented.

Images