Traffic scene twinborn and video synchronous display method and device

Abstract

The invention discloses a traffic scene twinning and video synchronous display method and device, and the method comprises the steps: obtaining a video image of a road scene, collecting the video image through a perception system disposed on a road, displaying a twinning image of the traffic scene, and enabling the twinning image to comprise a twinning object of the road and a picture frame of an accompanying road, the picture frame is used for displaying the video image, and at least one part of the video image is rotated, so that the included angle between the middle line of the road in the video image and the middle line of the twin object of the road is changed. The problem that images are lost when videos collected by the two road sensing systems are switched due to different shooting angles is solved, accurate and real-time synchronization of video images and twin images is guaranteed, and accurate management of vehicle tracking is facilitated.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a method, device, computer equipment and medium for synchronous display of traffic scene twins and videos. Background Art

[0002] In recent years, road traffic has developed rapidly, the number of various vehicles has gradually increased, and the intelligence level of vehicle monitoring has also been continuously improved.

[0003] The relevant intelligent traffic management technology mainly uses the elephant twin method to map the real scene of the road condition into a two-dimensional road scene using the road and vehicle information captured by video and radar. It tracks and manages a large number of vehicles. However, it is currently difficult to ensure accurate synchronization between the monitoring video and the vehicles mapped into the two-dimensional road scene on the monitoring screen, which brings blind spots to vehicle monitoring.

[0004] However, due to the position of the camera in collecting images, the movement direction of the vehicle in the video is often different from that in the twin graphics, and sometimes even in the opposite direction. When the visual image includes a wide range of application scenarios and vehicles, it affects the convenience of traffic management. Summary of the invention

[0005] The present application proposes a method and device for synchronously displaying twins and videos of traffic scenes to solve the problem of inconsistent motion between the target vehicle displayed in the video and the twin display.

[0006] Based on this, it is necessary to propose a method, device, computer equipment and medium for accurate and synchronous display of traffic scene twins and videos to address the above-mentioned technical problems and solve the problem of blind spots in monitoring.

[0007] In a first aspect, the present application provides a method for synchronously displaying traffic scene twins and videos, the method comprising:

[0008] Obtaining video images of the road scene, the video images are collected by a perception system set up on the road;

[0009] A twin image showing a traffic scene, wherein the twin image includes a twin object of a road and a frame accompanying the road, and the frame is used to display a video image;

[0010] At least a portion of the video image is rotated so that an angle between a middle line of the road in the video image and a middle line of a twin object of the road changes.

[0011] In one embodiment, current perception information of a target vehicle at a road scene is obtained, and the current perception information is obtained through a perception system set on the road;

[0012] In the frame, a video image of a target vehicle is included;

[0013] Rotate at least a portion of the video image so that the target vehicle in the video image moves in the same direction as the target vehicle's twin object.

[0014] In one embodiment, a first line between two boundaries of the road is determined at a first position near the road end;

[0015] Determine a second line between two boundaries of the road at a second position at the far end of the road;

[0016] A trapezoidal area in the video image is determined according to two end points of the first connecting line and two end points of the second connecting line.

[0017] In one of the embodiments, at least a portion of the video image is captured and converted into a rectangular image through coordinate transformation.

[0018] In one of the embodiments, at least a portion of the video image is stretched to fit within a set frame size.

[0019] In one of the embodiments, the position of the frame for displaying the video image is determined according to the acquisition position of the video image.

[0020] In one of the embodiments, obtaining road geographic information data, the geographic information data including geographic coordinates of the road;

[0021] In the twin image of the traffic scene, the centerline direction of the road is mapped to the x direction, and the width direction of the road is mapped to the y direction. A rectangular coordinate system is established to display the two-dimensional graphics of the road.

[0022] Determine the x-coordinate of any point in the two-dimensional figure, which corresponds to the cumulative length of the position of any point in the geographic information data of the road along the direction parallel to the center line of the road.

[0023] In one embodiment, current perception information of a target vehicle at a road scene is obtained, and the current perception information is obtained through a perception system set on the road;

[0024] The twin image of the traffic scene includes the twin object of the target vehicle;

[0025] Determine the x-coordinate of the twin object of the target vehicle in the two-dimensional graph, corresponding to the cumulative length of the position of the target vehicle at the road scene along a direction parallel to the centerline of the road;

[0026] Determine the y coordinate of the twin object of the target vehicle in the two-dimensional graph, corresponding to the relative position of the target vehicle at the road scene along the road width direction.

[0027] In one embodiment, at least one of the following is determined based on the stake numbers arranged along the road:

[0028] The cumulative length of any point on the road site along the direction parallel to the centerline of the road;

[0029] The cumulative length of the target vehicle position in the road scene along the direction parallel to the center line of the road;

[0030] The cumulative length of the road equipment location along the direction parallel to the road centerline;

[0031] The x-coordinate of any point in the road scene in the two-dimensional graph;

[0032] The x-coordinate of the twin object of the target vehicle in the road scene in the two-dimensional graph;

[0033] The x-coordinate of the twin object of the road equipment in the 2D graph.

[0034] In one of the embodiments, the twin scene includes a plurality of frames arranged along a moving direction of a twin object of the target vehicle;

[0035] A frame for displaying the video image of the target vehicle is determined according to a video image acquisition position corresponding to the position of the target vehicle.

[0036] In one of the embodiments, at least one position in the road scene is mapped as a first position in the road twin object and mapped as a second position in the frame; in the twin scene, the first position and the second position are aligned in a direction perpendicular to the centerline of the road.

[0037] In one of the embodiments, at least one position of a target vehicle at a road scene is mapped as a third position in a twin object of the road and is mapped as a fourth position in a video image in a frame; at at least one moment in the twin scene, the third position and the fourth position are aligned in a direction perpendicular to the centerline of the road.

[0038] In one of the embodiments, in the twin scene, the identity information of the vehicle simultaneously displayed in the video image at at least one moment is determined based on the identity information of the twin object bound to the target vehicle.

[0039] In one of the embodiments, at least one position in the road scene is determined based on stake numbers arranged along the road, and is mapped as a first position in the road twin object and as a second position in the frame;

[0040] The position of the target vehicle is determined according to the stake numbers arranged along the road, and is mapped as the third position in the twin object of the road and as the fourth position in the frame.

[0041] According to the stake numbers arranged along the road, the cumulative length of any point in the geographic information data of the road along the center line of the road is determined.

[0042] In a second aspect, the present application also provides a traffic scene twin and video synchronous display device, the device comprising:

[0043] An acquisition module is used to acquire video images of the road scene, and the video images are collected by a perception system set up on the road;

[0044] A display module, used to display a twin picture of a traffic scene, wherein the twin picture includes a twin object of a road and a picture frame accompanying the road, and the picture frame is used to display a video image;

[0045] The video processing module is used to rotate at least a portion of the video image so that the angle between the middle line of the road in the video image and the middle line of the twin object of the road changes.

[0046] In one embodiment,

[0047] The acquisition module is also used to acquire current perception information of the target vehicle on the road scene, and the current perception information is obtained through a perception system set on the road;

[0048] A display module, used for displaying a twin picture of a traffic scene, wherein the twin picture includes a twin object of a road and a picture frame accompanying the road, and the picture frame includes a video image of a target vehicle;

[0049] The video processing module is used to rotate at least a portion of the video image so that the target vehicle in the video image moves in the same direction as the twin object of the target vehicle.

[0050] In one of the embodiments, the position of the frame is determined according to the stake number where the perception system is arranged along the road.

[0051] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program is executed by a processor to implement the steps of the method provided in any embodiment of the first aspect above.

[0052] In a fourth aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method provided in any one of the embodiments of the first aspect when executing the computer program.

[0053] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: first, the traffic scene with geographic information data is twinned to a two-dimensional graphic, and the vehicles, road equipment, and pile numbers are twinned to the two-dimensional graphic through the sensing device, and the positions of the twin vehicles and twin road equipment in the two-dimensional graphic are measured according to the pile number positions. In the display screen, the video image of the road scene is aligned with the road twin scene and the video image is displayed synchronously with the twin vehicle in the frame. By adjusting the video shooting angle, it is ensured that there is no image loss for the video images of adjacent frames, and at the same time, it is ensured that the driving angles of each video image and the twin scene are always consistent. The accurate and real-time synchronization of the video image and the twin scene and the uniformity of the driving angle are guaranteed, which facilitates the precise management of vehicle tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0055] Figure 1 A diagram of the traffic scene application environment provided in the embodiment of the present application;

[0056] Figure 2 A flow chart of the method for synchronously displaying traffic scene twins and videos provided in an embodiment of the present application;

[0057] Figure 3.1 to Figure 3.5 A schematic diagram of a video adjustment process in an embodiment of the present application;

[0058] Figure 4 A flowchart for determining the position of a target vehicle twin object provided by another embodiment of the present application;

[0059] Figure 5 A flow chart of a method for determining a synchronous display position provided in an embodiment of the present application;

[0060] Figure 6 A flow chart of a method for further optimizing synchronous display of images provided in an embodiment of the present application;

[0061] Figure 7 A schematic diagram of a twin scene effect of an embodiment of the present application;

[0062] Figure 8 This is a structural diagram of the traffic scene twin and video synchronization display device proposed in the embodiment of the present application;

[0063] Fig. 9 This is a structural diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0065] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0066] Figure 1 A diagram of the traffic scenario application environment provided in an embodiment of the present application.

[0067] like Figure 1 As shown, a road device 105 including a prompt device and a multi-sensing device 103 including a radar and / or a camera are combined on a pile 104 and perform wired or wireless communication with a computer device 101 with a visualization function through a network 102 .

[0068] The multi-sensing device 104 may be a radar or / and a camera, such as at least one of a millimeter wave radar, a laser radar, and a camera, etc. The road device 105 may be an information board, a directional sound device, and the types of the multi-sensing device\road device are not specifically limited here. The computer device 101 may be a roadside computing unit / terminal / edge server, and optionally, the computer device may also be a cloud server, a vehicle-mounted computing unit / terminal, etc. The embodiment of the present application does not specifically limit the type of the computer device.

[0069] Figure 2 Flowchart of the method for synchronous display of traffic scene twins and videos provided in an embodiment of the present application.

[0070] Step 210: Acquire a video image of the road scene, where the video image is collected by a perception system set on the road;

[0071] The perception system can be composed of multiple road equipment, including but not limited to radar, cameras, information boards, etc.

[0072] In this embodiment, the vehicle's driving information, license plate number, status of road equipment, etc. are obtained from the perception system.

[0073] In other words, the perception system is composed of multiple road devices to generate comprehensive information.

[0074] Step 220: Rotate at least a portion of the video image to change the angle between the middle line of the road in the video image and the middle line of the twin object of the road.

[0075] In this embodiment, the purpose of rotating the video image can be achieved by rotating the camera angle or deforming and rotating the video image. The effect after rotating the video image needs to achieve a change in the angle between the middle line of the road and the middle line of the road's twin object.

[0076] The "angle change" in this application document can make the angle between the two reach the desired angle range. The scheme of this application is not limited to measuring the result of increasing or decreasing the angle between the two by a set threshold. For example, in the image captured by a camera on the road scene, when the middle line of the road and the middle line of the road twin object in the video image are at a clear angle, the video image can be rotated to make the middle line of the road roughly parallel to the middle line of the road twin object.

[0077] For example, the camera rotation angle or the video image deformation rotation is performed based on the center line of the road. This rotation method has less image deformation than other methods.

[0078] Further, step 220A, determines a first connecting line between two boundaries of the road at a first position at the near end of the road; determines a second connecting line between two boundaries of the road at a second position at the far end of the road; and determines a trapezoidal area in the video image based on the two end points of the first connecting line and the two end points of the second connecting line.

[0079] The first line may be a line between two road boundaries at the starting point of the road photographed by the camera. The second line may be a line between two road boundaries at the end of the road photographed by the camera. In this embodiment, according to the visual principle that objects are larger when near and smaller when far, the first line is longer than the second line, and the first line, the second line and the road boundary line form a trapezoidal diagram. In other words, the trapezoidal diagram forms a section of the road area.

[0080] Further, in step 220B, at least a portion of the video image is intercepted and converted into a rectangular image through coordinate transformation. In this embodiment, the road is converted from a trapezoidal shape to a rectangular shape through graphic transformation. That is, the four fixed-angle coordinates of the trapezoidal graph are correspondingly transformed so that the trapezoidal graph becomes a rectangular graph. In this embodiment, for example, the second connecting line is stretched based on the length of the first connecting line to form a rectangular graph. Lengthening the second connecting line with reference to the first connecting line is conducive to reducing the error caused by stretching.

[0081] Furthermore, in step 220C, at least a portion of the video image is stretched to reach a set frame size.

[0082] Preferably, in step 220C, when stretching the image, the position of the first line in the road entity is mapped to the first position in the twin graph of the road, and the first position is aligned with one end of the frame; the position of the second line in the road entity is mapped to the second position in the twin graph of the road, and the second position is aligned with the other end of the frame. In other words, the length of the frame after the image is stretched is adapted to the distance between the first position and the second position.

[0083] Step 230: Display a twin image of the traffic scene, wherein the twin image includes a twin object of the road and a frame accompanying the road, and the frame is used to display the video image.

[0084] The twin scene can be a visual expression of the traffic scene presented on the screen, which includes but is not limited to static or dynamic scene twin display, visual charts, rendering services and interoperability tools.

[0085] Twin objects can be visual expressions of objects in the real physical world, including but not limited to static objects such as buildings, landforms, and transportation facilities, as well as dynamic objects such as traffic participants (vehicles, pedestrians, etc.), weather environment, and traffic time.

[0086] In this embodiment, the twin scene includes a twin object of the road and a frame accompanying the road, and a video shot at the road scene is played in real time in the frame.

[0087] In other words, the twin image integrates the twin road and the video of the road scene, making it easier for road managers to refer to the road conditions between the actual scene and the twin object.

[0088] Figure 3.1 to Figure 3.5 Schematic diagram of the video adjustment process in an embodiment of the present application.

[0089] like Figure 3.1 As shown, at the road boundary near the video screen, a tangent line perpendicular to the middle line of the road is made to intersect with the boundary lines on both sides of the road, and the line segment formed by connecting the two intersection points constitutes the starting end of the road.

[0090] like Figure 3.2 As shown, at the far end of the road boundary in the video screen, a line parallel to the starting end intersects with the boundary lines on both sides of the road to form the far end, and the line segments of the starting end and the far end are connected to form a trapezoid.

[0091] like Figure 3.3 As shown, the trapezoidal area in the video screen is cropped, the far end is pulled up toward the center line to form an isosceles trapezoid, and a plane rotation is performed to make the road level.

[0092] like Figure 3.4 As shown, with the long base of the trapezoid as the axis, a three-dimensional projection rotation is performed to form a rectangle.

[0093] like Figure 3.5 As shown, the trapezoidal area in the video picture is stretched horizontally to form a rectangle, for example, with an aspect ratio of 4:3 or 16:9.

[0094] It should be noted that the intersection calculation method includes but is not limited to manual and AI visual recognition. The intersection line can be appropriately translated forward and backward through parameter configuration. The rectangular area can be stretched in aspect ratio through parameter configuration. The technology of video rotation, stretching and cropping includes but is not limited to opencv, HTML technology, etc.

[0095] Figure 4 A flowchart for determining the position of a target vehicle twin object is provided in accordance with another embodiment of the present application.

[0096] The location where the video image is collected is the location of the equipment pile or the location of other road equipment where the camera is installed.

[0097] Step 410: Acquire current perception information of the target vehicle on the road scene, where the current perception information is obtained through a perception system set on the road;

[0098] The current perception information may be real-time perception data.

[0099] In this embodiment, the current perception information includes the real-time position of the vehicle, the status of road equipment, etc.

[0100] That is to say, the current perception information of the vehicle is obtained, and the position of the twin vehicle is rendered and refreshed in real time in the twin scene, so that the position of the twin vehicle is consistent with the actual position of the vehicle on the road, which is conducive to accurate matching.

[0101] Step 420: Rotate at least a portion of the video image so that the target vehicle in the video image moves in the same direction as the twin object of the target vehicle.

[0102] In this embodiment, after the video is deformed and twisted, the driving direction of the vehicle in the road scene in the video is consistent with the driving direction of the twin vehicle of the target vehicle in the twin scene. The driving direction of the vehicle in the video is consistent with the driving direction of the twin vehicle, which is convenient for road administrators to monitor.

[0103] It should be noted that the "consistent direction" here means traveling in the same direction as the road, for example, traveling towards the "Beijing direction", including but not limited to the angle between the vectors of the two moving directions being 0, approximately 0, less than the expected angle, within the expected angle range, or the moving direction of any one of the two changing randomly in the horizontal direction of the road at any point in time.

[0104] Furthermore, as shown in steps 220A-C, at least a portion of the video image is stretched so that at least a portion matches a frame of a set size in the form of a rectangular image.

[0105] Furthermore, different parts of the image are stretched in varying proportions so that the moving speeds of the target vehicle in the video image and the target vehicle's twin object are similar (or, at least the same for part of the time; or, at least at both ends of the image, the difference in moving speeds between the target vehicle in the video image and the target vehicle's twin object is less than a set threshold).

[0106] Preferably, the stretching ratio of the far end of the image is greater than the stretching ratio of the near end of the image, so that the moving speed of the target vehicle in the video image matches the twin object of the target vehicle on the road. Due to the video shooting angle, the near-end target vehicle in the image moves faster, and the far-end target vehicle moves slower. When rotating and stretching, stretching the image at the same ratio will cause the target vehicle to move faster at one end of the video image and slower at the other end. Therefore, stretching the video image at a non-uniform ratio can make the moving speed of the target vehicle in the video image roughly consistent with the moving speed of the vehicle's twin object.

[0107] Step 430: In the frame, a video image of the target vehicle is included.

[0108] Based on step 420, the video image is superimposed on a system including visualization of the road and the target vehicle twin object to improve the correspondence between the video and the perception result. The video information of the target vehicle is matched with the twin object of the target vehicle on the road in real time.

[0109] Preferably, in terms of visual effect, moving objects move faster when they are near and slower when they are far away. When considering the direction of movement, the moving speed of the object should also be considered. The speed of the twin vehicles at the end of the twin road is slower than that at the starting end.

[0110] Figure 5 A flow chart of a method for determining a synchronous display position provided in an embodiment of the present application.

[0111] Based on the embodiment of the first aspect of the present application (steps 210 to 230, 410 to 430), further, the position of the frame for displaying the video image is determined according to the acquisition position of the video image.

[0112] Step 510: Include multiple frames arranged along the moving direction of the twin object of the target vehicle in the twin scene.

[0113] In this embodiment, a plurality of frames are arranged in the moving direction of the twin vehicles, each of which may contain a video image. The actual situation of the twin vehicles can be observed through the video image in at least a portion of the plurality of frames.

[0114] Step 520: Determine a frame for displaying the video image of the target vehicle according to the video image acquisition position corresponding to the position of the target vehicle.

[0115] In this embodiment, the position of the frame is determined by the position of the camera. The real-time video of the current vehicle is formed by multiple cameras, which is convenient for monitoring in sections.

[0116] Step 530: Align the twin object of the road position with the video image.

[0117] At least one position in the road scene is mapped as a first position in the road twin object and mapped as a second position in the frame; in the twin scene, the first position and the second position are aligned in a direction perpendicular to the centerline of the road;

[0118] Step 540 : The twin object of the target vehicle is synchronized with the video image.

[0119] At least one position of the target vehicle at the road scene is mapped as a third position in the twin object of the road and mapped as a fourth position in the video image in the frame; at at least one moment in the twin scene, the third position and the fourth position are aligned in a direction perpendicular to the centerline of the road.

[0120] In this embodiment, the position in the road scene is mapped one-to-one with the position of the frame in the twin scene, and the position of the target vehicle is mapped one-to-one with the position of the video in the frame. The real-time video of the current vehicle has a one-to-one mapping relationship with the position of the twin vehicle in the twin scene, forming a complete monitoring without omissions.

[0121] Step 550: In the twin scene, according to the identity information of the twin object bound to the target vehicle, determine the identity information of the vehicle simultaneously displayed in the video image at at least one moment.

[0122] In this embodiment, the target vehicle is bound to the twin vehicle in the twin scene by identity, so that the target vehicle in the twin scene can have identity information. The target vehicle in the twin scene has license plate information, which is convenient for tracking and monitoring the target vehicle.

[0123] Based on the embodiment of the first aspect of the present application (steps 210 to 230, 410 to 430), further, Figure 6 A flow chart of a method for further optimizing synchronous display of images provided in an embodiment of the present application.

[0124] Step 610: Acquire road geographic information data, where the geographic information data includes the geographic coordinates of the road.

[0125] Step 620: In the twin image of the traffic scene, the centerline direction of the road is mapped to the x direction, the width direction of the road is mapped to the y direction, a rectangular coordinate system is established, and a two-dimensional graph of the road is displayed.

[0126] Step 630, determine the x-coordinate of any point in the two-dimensional figure (for example, any position in the road, the acquisition position of the video image, the position of the target vehicle), corresponding to the cumulative length of the position of the arbitrary point in the three-dimensional geographic information data of the road along the direction parallel to the centerline of the road.

[0127] In other words, converting the actual values ​​of geographic coordinates into relative values ​​of rectangular coordinates can greatly simplify the amount of data to be processed.

[0128] Step 640: Acquire current perception information of the target vehicle at the road scene, where the current perception information is obtained through a perception system set on the road.

[0129] Step 650: The twin image of the traffic scene includes a twin object of the target vehicle.

[0130] Step 660: Determine the x-coordinate of the twin object of the target vehicle in the two-dimensional graph, corresponding to the cumulative length of the position of the target vehicle at the road scene along a direction parallel to the centerline of the road.

[0131] Step 670: Determine the y coordinate of the twin object of the target vehicle in the two-dimensional graph, corresponding to the relative position of the target vehicle at the road scene along the road width direction.

[0132] In other words, by converting the actual values ​​of geographic coordinates into relative values ​​of rectangular coordinates, the data processing for dynamic vehicles is greatly reduced, making it easier to track moving objects.

[0133] Further, according to the stake numbers arranged along the road, at least one of the following is determined:

[0134] The cumulative length of any point on the road site along the direction parallel to the centerline of the road;

[0135] The cumulative length of the target vehicle position in the road scene along the direction parallel to the center line of the road;

[0136] The cumulative length of the road equipment location along the direction parallel to the road centerline;

[0137] The x-coordinate of any point in the road scene in the two-dimensional graph;

[0138] The x-coordinate of the twin object of the target vehicle in the road scene in the two-dimensional graph;

[0139] The x-coordinate of the twin object of the road equipment in the 2D graph.

[0140] It should be noted that the geographic location of the road equipment (such as a multi-source perception device including a laser radar) included in the geographic information system can be used to calculate the cumulative length based on the geographic location information. Furthermore, after the perception information of the target vehicle is acquired through the multi-source perception device, at least the relative position information between the target vehicle and the location of the multi-source perception device can be determined. Based on the cumulative length of the location of the road equipment along the direction parallel to the center line of the road, the cumulative length of the location of the target vehicle in the road scene along the direction parallel to the center line of the road can be determined; or, based on the x-coordinate of the twin object of the road equipment in the two-dimensional graph, and the relative position information between the target vehicle and the location of the road equipment, the x-coordinate of the target vehicle in the two-dimensional graph can be determined.

[0141] When the above-mentioned cumulative lengths are represented by the information of the pile number, or when the x-coordinate corresponds to the pile number, the cumulative lengths or the x-coordinates can be determined according to the pile numbers arranged along the road.

[0142] Furthermore, the present application uses information indicating the pile number to indicate the cumulative length, and any position-related information of the present application can be indicated by the pile number, for example, the acquisition position of the video image, the road position, the position of the traffic participant (including the target vehicle), the position of the road equipment, etc.

[0143] It should be noted that in the present application, the first position, the third position, and the position of the road equipment can all be represented by stake numbers.

[0144] Further, at least one position in the road scene is determined according to the stake numbers arranged along the road, and is mapped as a first position in the road twin object and as a second position in the frame;

[0145] According to the stake numbers arranged along the road, the position of the target vehicle is determined, and is mapped as the third position in the twin object of the road and as the fourth position in the frame;

[0146] According to the stake numbers arranged along the road, the cumulative length of any point in the three-dimensional geographic information data of the road along the center line of the road is determined.

[0147] For example, the pile number is an integer in meters. An example of a common expression for positioning on a highway section is: T20+527 is the pile number of the equipment pile, the letter T represents the up or down direction, 20 is kilometers, and 527 is meters. The position of the twin object can be uniformly expressed by the pile number, or the position resolution can be further improved based on the pile number.

[0148] It should also be noted that one equipment pile can be set every 100 meters. For example, the entrance to the tunnel is taken as the reference, where the pile number is T20+200. If the target vehicle is at the entrance, its pile number position is T20+200. After driving forward 1 meter, its real-time position expressed by the pile number position is T20+201; if it has not entered the tunnel and is 2 meters away from the entrance, its real-time position expressed by the pile number position is T20+198.

[0149] It should also be noted that the pile number twin expresses the position in the pile number coordinate system, and there is a mapping relationship with the geodetic coordinate system, that is, the longitude and latitude of the road in the geodetic coordinate system are mapped to the pre-processed two-dimensional data space through this mapping relationship. In this way, the pile number twin maps the high-precision map twin to the "pile number twin map". In other words, the pile number converts the actual value of the geographic coordinate into the relative value of the rectangular coordinate, which greatly simplifies the amount of data to be processed. Figure 7 This is a schematic diagram of the twin scene effect of an embodiment of the present application.

[0150] exist Figure 7 In the figure, the pile number is used as a reference (for example, "YK1608", "YK1608+200", etc. in the figure), the twin object of the road is represented in the form of a straight line on the screen, the vehicle traveling on the road is displayed as a twin vehicle, the current position of the twin vehicle is dynamically displayed on the twin road, the video of the current road section is displayed above the twin road, and the twin road equipment of the road equipment displays the equipment status in real time above the video. In at least one frame (shown in the upper right corner of the figure), the video image of the vehicle therein is processed by the solution of the present application and is consistent with the motion state of the twin object of the target vehicle.

[0151] Figure 8 This is a structural diagram of a traffic scene twin and video synchronous display device proposed in an embodiment of the present application. This is a method embodiment for implementing the first aspect of the present application. The traffic scene twin and video synchronous display device proposed in the present application includes:

[0152] An acquisition module 810 is used to acquire a video image of a road scene, where the video image is collected by a perception system set on the road;

[0153] A display module 820 is used to display a twin image of a traffic scene, wherein the twin image includes a twin object of a road and a frame accompanying the road, and the frame is used to display a video image;

[0154] Furthermore, the display module determines the position of the frame according to the stake number where the perception system is arranged along the road.

[0155] The video processing module 830 is used to rotate at least a portion of the video image so that the angle between the middle line of the road in the video image and the middle line of the twin object of the road changes.

[0156] Furthermore, the video processing module is also used to intercept and / or stretch at least a portion of the video image so that at least a portion of the video image reaches a set frame size.

[0157] In another embodiment of the present application, the acquisition unit is further used to acquire current perception information of the target vehicle at the road scene, where the current perception information is obtained through a perception system set on the road;

[0158] A display module, used for displaying a video image of a target vehicle in a frame;

[0159] The video processing module is used to rotate at least a portion of the video image so that the target vehicle in the video image moves in the same direction as the twin object of the target vehicle.

[0160] In one embodiment, the acquisition module 810 includes:

[0161] A first acquisition unit acquires road geographic information data, wherein the geographic information data includes the geographic coordinates of the road;

[0162] In the twin image of the traffic scene, the centerline direction of the road is mapped to the x direction, and the width direction of the road is mapped to the y direction. A rectangular coordinate system is established to display the two-dimensional graphics of the road.

[0163] The x-coordinate of any point in the two-dimensional figure is determined, which corresponds to the cumulative length of the position of the arbitrary point in the three-dimensional geographic information data of the road along the direction parallel to the center line of the road.

[0164] The second acquisition unit determines at least one of the following according to the stake numbers arranged along the road:

[0165] The cumulative length of any point on the road site along the direction parallel to the centerline of the road;

[0166] The cumulative length of the target vehicle position in the road scene along the direction parallel to the center line of the road;

[0167] The cumulative length of the road equipment location along the direction parallel to the road centerline;

[0168] The x-coordinate of any point in the road scene in the two-dimensional graph;

[0169] The x-coordinate of the twin object of the target vehicle in the road scene in the two-dimensional graph;

[0170] The x-coordinate of the twin object of the road equipment in the 2D graph.

[0171] A third acquisition unit is used to acquire current perception information of the target vehicle on the road scene, where the current perception information is obtained through a perception system set on the road;

[0172] The twin image of the traffic scene includes the twin object of the target vehicle;

[0173] Determine the x-coordinate of the twin object of the target vehicle in the two-dimensional graph, corresponding to the cumulative length of the position of the target vehicle at the road scene along a direction parallel to the centerline of the road;

[0174] Determine the y coordinate of the twin object of the target vehicle in the two-dimensional graph, corresponding to the relative position of the target vehicle at the road scene along the road width direction.

[0175] The fourth acquisition unit is used to acquire video images of the road scene, and the video images are collected by a perception system set up on the road.

[0176] In one embodiment, the display module 820 includes:

[0177] A first display unit is used to display a twin image of a traffic scene, wherein the twin image includes a twin object of a road and a frame accompanying the road, and the frame is used to display a video image;

[0178] The second display unit determines the position of the frame for displaying the video image according to the acquisition position of the video image.

[0179] The third display unit is used to display the video image of the target vehicle in the frame;.

[0180] The fourth display unit displays a plurality of frames arranged along the moving direction of the twin object of the target vehicle in the twin scene; and determines a frame for displaying the video image of the target vehicle according to a video image acquisition position corresponding to the position of the target vehicle.

[0181] In one embodiment, the fourth display unit includes:

[0182] The first subunit, at least one position in the road scene, is mapped as a first position in the road twin object and mapped as a second position in the frame; in the twin scene, the first position and the second position are aligned in a direction perpendicular to the center line of the road; or, at least one position of the target vehicle in the road scene is mapped as a third position in the twin object of the road and mapped as a fourth position in the video image in the frame; at at least one moment in the twin scene, the third position and the fourth position are aligned in a direction perpendicular to the center line of the road.

[0183] The second subunit is used to determine the identity information of the vehicle displayed simultaneously in the video image at at least one moment in the twin scene according to the identity information of the twin object bound to the target vehicle.

[0184] The third subunit is used to determine at least one position in the road site according to the pile numbers arranged along the road, and map it as the first position in the road twin object and as the second position in the frame.

[0185] The fourth subunit is used to determine the position of the target vehicle according to the stake numbers arranged along the road, and map it as the third position in the twin object of the road and as the fourth position in the frame.

[0186] According to the stake numbers arranged along the road, the cumulative length of any point in the three-dimensional geographic information data of the road along the center line of the road is determined.

[0187] In one embodiment, the video processing module 830 includes:

[0188] The first video processing unit is used to determine a first connecting line between two boundaries of the road at a first position at the near end of the road; determine a second connecting line between two boundaries of the road at a second position at the far end of the road; and determine a trapezoidal area in the video image based on the two end points of the first connecting line and the two end points of the second connecting line.

[0189] The second video processing unit is used to capture at least a part of the video image and transform it into a rectangular image through coordinate transformation.

[0190] The third video processing unit is used to stretch at least a portion of the video image to reach a set frame size.

[0191] The fourth video processing unit is used to rotate at least a part of the video image so that the target vehicle in the video image moves in the same direction as the twin object of the target vehicle.

[0192] The fifth video processing unit is used to rotate at least a part of the video image so that the angle between the middle line of the road in the video image and the middle line of the twin object of the road changes.

[0193] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] Therefore, the present application also proposes a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any embodiment of the present application is implemented.

[0195] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0196] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0197] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0198] Fig. 9 This is a structural diagram of a computer device in an embodiment of the present application.

[0199] Furthermore, the present application also proposes a computer device (or computing device), including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present application when executing the computer program.

[0200] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. Memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium. Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0201] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0202] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for synchronously displaying traffic scene twins and videos, characterized in that: include: Acquire a video image of a road scene, wherein the video image is collected by a perception system arranged on the road; Displaying a twin image of a traffic scene, wherein the twin image includes a twin object of a road and a frame accompanying the road, wherein the frame is used to display the video image; At least a portion of the video image is rotated to change an angle between a middle line of a road in the video image and a middle line of a twin object of the road.

2. A method for synchronously displaying traffic scene twins and videos, characterized in that: Acquire current perception information of a target vehicle at a road scene, wherein the current perception information is obtained through a perception system set on the road; Acquire a video image of a road scene, wherein the video image is collected by a perception system arranged on the road; Displaying a twin image of a traffic scene, wherein the twin image includes a twin object of a road and a frame accompanying the road, wherein the frame includes a video image of the target vehicle; Rotate at least a portion of the video image so that the target vehicle in the video image moves in the same direction as a twin object of the target vehicle.

3. A traffic scene twin and video synchronous display method as claimed in claim 1 or 2, characterized in that: Determining the at least one portion of the video image comprises the following steps: Determining a first line between two boundaries of the road at a first position near the road end; Determine a second line between two boundaries of the road at a second position at the far end of the road; A trapezoidal area in the video image is determined according to the two end points of the first connecting line and the two end points of the second connecting line.

4. A method for synchronously displaying traffic scene twins and videos as claimed in claim 1 or 2, characterized in that: At least a portion of the video image is intercepted and converted into a rectangular image through coordinate transformation.

5. A traffic scene twin and video synchronous display method as claimed in claim 1 or 2, characterized in that: The at least one portion of the video image is stretched to reach a set frame size.

6. A method for synchronously displaying traffic scene twins and videos as claimed in claim 1 or 2, characterized in that: The position of a frame for displaying the video image is determined according to the acquisition position of the video image.

7. A method for synchronously displaying traffic scene twins and videos as claimed in claim 1 or 2, characterized in that: Acquire road geographic information data, wherein the geographic information data includes the geographic coordinates of the road; In the twin image of the traffic scene, the centerline direction of the road is mapped as the x direction, the width direction of the road is mapped as the y direction, a rectangular coordinate system is established, and a two-dimensional graph of the road is displayed; The x-coordinate of any point in the two-dimensional graph is determined, corresponding to the cumulative length of the position of the any point in the geographic information data of the road along a direction parallel to the center line of the road.

8. The method for synchronously displaying traffic scene twins and videos as claimed in claim 1 or 2, characterized in that: Acquire current perception information of a target vehicle at a road scene, wherein the current perception information is obtained through a perception system set on the road; The twin image of the traffic scene includes a twin object of the target vehicle; Determine an x-coordinate of the twin object of the target vehicle in the two-dimensional graph, corresponding to a cumulative length of the position of the target vehicle at the road scene along a direction parallel to a centerline of the road; Determine the y coordinate of the twin object of the target vehicle in the two-dimensional graph, corresponding to the relative position of the target vehicle at the road scene along the road width direction.

9. The method for synchronously displaying traffic scene twins and videos as claimed in claim 1 or 2, characterized in that: Based on the stakes arranged along the road, at least one of the following is determined: The cumulative length of any point in the road scene along a direction parallel to the road centerline; The cumulative length of the target vehicle position in the road scene along a direction parallel to the road centerline; The cumulative length of the road equipment location along the direction parallel to the road centerline; The x-coordinate of any point in the road scene in the two-dimensional graph; The x-coordinate of the twin object of the target vehicle in the road scene in the two-dimensional graph; The x-coordinate of the twin object of the road equipment in the two-dimensional graph.

10. The method for synchronously displaying traffic scene twins and videos according to claim 1 or 2, characterized in that: The twin scene includes a plurality of frames arranged along a moving direction of a twin object of the target vehicle; A frame for displaying the video image of the target vehicle is determined according to a video image acquisition position corresponding to the position of the target vehicle.

11. The method for synchronously displaying traffic scene twins and videos according to claim 1 or 2, characterized in that: At least one position in the road scene is mapped as a first position in the road twin object and mapped as a second position in the frame; in the twin scene, the first position and the second position are aligned in a direction perpendicular to the centerline of the road; and / or, At least one position of a target vehicle at a road scene is mapped as a third position in the twin object of the road and is mapped as a fourth position in the video image in the frame; at least one moment in the twin scene, the third position and the fourth position are aligned in a direction perpendicular to the centerline of the road.

12. The method for synchronously displaying traffic scene twins and videos according to claim 1 or 2, characterized in that: In the twin scene, the identity information of the vehicle simultaneously displayed in the video image at at least one moment is determined according to the identity information of the twin object bound to the target vehicle.

13. The method for synchronously displaying traffic scene twins and videos according to claim 1 or 2, characterized in that: Determine at least one position in the road scene according to stake numbers arranged along the road, and map it as a first position in the twin object of the road and as a second position in the frame; Determine the position of the target vehicle according to the stake numbers arranged along the road, and map it as a third position in the twin object of the road and as a fourth position in the frame; According to the stake numbers arranged along the road, the cumulative length of the position of any point in the geographic information data of the road along the center line direction of the road is determined.

14. A traffic scene twin and video synchronous display device, characterized in that: The device comprises: An acquisition module, used to acquire a video image of a road scene, wherein the video image is collected by a perception system arranged on the road; A display module, used to display a twin picture of a traffic scene, wherein the twin picture includes a twin object of a road and a picture frame accompanying the road, and the picture frame is used to display the video image; The video processing module is used to rotate at least a portion of the video image so that the angle between the middle line of the road in the video image and the middle line of the twin object of the road changes.

15. A traffic scene twin and video synchronous display device, characterized in that: The device comprises: An acquisition module is used to acquire current perception information of a target vehicle at a road scene, the current perception information being obtained through a perception system disposed on the road; and is also used to acquire a video image of the road scene, the video image being collected through a perception system disposed on the road; A display module, used for displaying a twin picture of a traffic scene, wherein the twin picture includes a twin object of a road and a picture frame accompanying the road, wherein the picture frame includes a video image of the target vehicle; The video processing module is used to rotate at least a portion of the video image so that the target vehicle in the video image moves in the same direction as the twin object of the target vehicle.

16. A traffic scene twin and video synchronous display device as claimed in claim 14 or 15, characterized in that: The position of the frame is determined according to the stake number where the perception system is arranged along the road.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

18. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 13 is implemented.

Images