Traffic panoramic operation and maintenance management and control method and device

Abstract

The invention discloses a traffic panorama operation and maintenance management and control method, and the method comprises the steps: displaying a twinborn image of a traffic scene, the twinborn image comprising twinborn objects of a road and frames displayed in parallel; acquiring a video image of a road site, wherein the video image is acquired through a sensing system arranged on a road; and determining the position of a picture frame for displaying the video image according to the acquisition position of the video image. The invention further comprises a device for implementing the method. According to the technical scheme, the technical problems that in the traffic operation and maintenance process in the prior art, the digital twinborn visual data processing efficiency is low, road equipment is inconvenient to manage and control in time, and the service effect of the road equipment is inconvenient to evaluate are solved, the twinborn data processing efficiency and the visual processing efficiency are improved, and the service capacity is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to an operation and maintenance, control method, device, computer equipment and medium for a one-picture overview of a traffic scene. Background Art

[0002] The existing technology has low data processing efficiency and low visualization resource utilization, which makes it difficult to realize a large number of dynamic functions. The existing technology of road video monitoring technology and twin technology of roads and traffic participants are functionally different devices or systems. When managing and controlling road service equipment, it is inconvenient to confirm the service effect. In particular, during the operation of the equipment, it is impossible to systematically observe the operating status of the equipment in real time, nor is it possible to understand the working content of the equipment in real time. When a road accident occurs, it is difficult to evaluate the implementation of the response strategy. Summary of the invention

[0003] The present application proposes a method and device for panoramic traffic operation and maintenance control, which solves the technical problems in the existing technology of traffic operation and maintenance that the visualization data processing efficiency of digital twins is low, which makes it inconvenient to timely control road equipment and evaluate the service effect of road equipment.

[0004] In a first aspect, the present application embodiment proposes a traffic panoramic operation and maintenance control method, including:

[0005] A twin picture showing a traffic scene, wherein the twin picture includes a twin object of a road and a frame displayed in parallel;

[0006] Obtaining video images of the road scene, the video images are collected by a perception system set up on the road;

[0007] The position of the frame used to display the video image is determined according to the acquisition position of the video image.

[0008] In one of the embodiments, the twin picture includes a plurality of picture frames arranged along the road direction;

[0009] A frame for displaying the video image of the target vehicle is determined according to a video image acquisition position corresponding to the position of the target vehicle.

[0010] In one of the embodiments, at least one position in the road scene is mapped as a first position in the road twin object and mapped as a second position in the frame;

[0011] In the twin view, the first position and the second position are aligned in a direction perpendicular to the centerline of the road.

[0012] In one of the embodiments, current perception information of a target vehicle on a road scene is obtained, and the current perception information is obtained through a perception system set up on the road; the twin image of the traffic scene includes a twin object of the target vehicle; and the frame includes a video image of the target vehicle.

[0013] In one of the embodiments, at least one position of the target vehicle at the road scene is mapped as a third position in the twin object of the road and mapped as a fourth position in the video image in the frame;

[0014] At at least one moment in the twin view, the third position and the fourth position are aligned in a direction perpendicular to the centerline of the road.

[0015] In one of the embodiments, in a set area of ​​a road scene, captured information of a target vehicle and perceived information of the target vehicle are obtained;

[0016] According to the captured information of the target vehicle, the identity information of the target vehicle is displayed;

[0017] In the twin picture of the traffic scene, the twin object of the target vehicle and the identity information of the target vehicle are bound.

[0018] In one of the embodiments, in the twin scene, the identity information of the vehicle simultaneously displayed in the video image at at least one moment is determined based on the identity information of the twin object bound to the target vehicle.

[0019] In one embodiment, the steps of displaying the twin object of the road are:

[0020] Get geographic information data,

[0021] In the twin image, the centerline direction of the road is taken as the x direction, and the width direction of the road is taken as the y direction. A rectangular coordinate system is established to display the two-dimensional graph of the road.

[0022] The x-coordinate of any point in the two-dimensional figure corresponds to the cumulative length of the position of any point in the geographic information data along the direction parallel to the centerline of the road.

[0023] In one embodiment, the x-coordinate of the collection position in the twin image is a rectangular coordinate system established with the centerline direction of the road as the x-direction and the width direction of the road as the y-direction, corresponding to the cumulative length of the collection position in the geographic information data along the direction parallel to the centerline of the road.

[0024] In one of the embodiments, the type and location information of the road equipment is obtained; in the twin scene, the twin object of the road equipment is displayed according to the type and location information of the road equipment.

[0025] In one of the embodiments, current working status information and / or current working content information of the road equipment is obtained, and in the twin scene, a dynamic twin object of the road equipment is displayed according to the current working status information and / or current working content information.

[0026] In one of the embodiments, at least one position in the road scene is determined based on stake numbers arranged along the road, and is mapped as a first position in the road twin object and as a second position in the frame;

[0027] In one of the embodiments, the position of the target vehicle is determined based on stake numbers arranged along the road, and is mapped as a third position in the twin object of the road and as a fourth position in the frame;

[0028] In one of the embodiments, the cumulative length of the position of any point in the geographic information data along the direction parallel to the center line of the road is determined based on the stakes arranged along the road.

[0029] In one of the embodiments, in response to a video image or a twin object, a preset working content and / or preset working status of a road device within a range that meets a set contact condition in the road is determined; and an instruction to execute the preset working content and / or preset working status is sent to the road device.

[0030] In one of the embodiments, in the twin scene, a dynamic twin object of the road equipment is displayed according to the current working state information and / or the current working content information of the road equipment;

[0031] Compare the positions of the dynamic twin object and the target vehicle twin object to determine whether the road equipment meets the set contact conditions.

[0032] In a second aspect, the present application proposes a traffic panoramic operation and maintenance control device, which is used to implement the method described in any embodiment of the first aspect of the present application, and the device includes:

[0033] An image acquisition module is used to acquire video images of the road scene, where the video images are collected by a perception system set in the road;

[0034] A display module, used to display a twin picture of a traffic scene, wherein the twin picture includes a twin object of a road and a picture frame displayed in parallel with the road;

[0035] The determination module is used to determine the position of the frame for displaying the video image according to the acquisition position of the video image.

[0036] In one embodiment, the device includes a management and control module for determining, in response to a video image or a twin object, a preset working content and / or a preset working status of road equipment within a range that meets a set contact condition in the road; and sending an instruction to the road equipment to execute the preset working content and / or the preset working status.

[0037] In one of the embodiments, the device includes a comparison module for comparing the positions of the dynamic twin object and the target vehicle twin object to determine whether the road equipment meets the set contact conditions.

[0038] In a third aspect, the present application further proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of the embodiments in the first aspect of the present application.

[0039] In a fourth aspect, the present application further proposes a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in any one of the embodiments in the first aspect of the present application is implemented.

[0040] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0041] Based on high-precision map twins or pile number twins, this application realizes a one-to-one correspondence between the vehicle trajectory and the real-time video, thereby improving the twin data processing efficiency and visualization processing efficiency.

[0042] Based on a comprehensive overview, the C-end information service can be confirmed intuitively and conveniently, and the effect of the accompanying information service for the vehicle can be confirmed, which is convenient for managers to confirm the actual service effect on site and improve service capabilities.

[0043] Based on a comprehensive overview, it is possible to monitor the status of equipment and, based on pile number management, twin the status and operating results of various types of equipment. This enables intuitive and quick confirmation of the owner's control effects or abnormal equipment, further improving service capabilities.

[0044] Furthermore, based on the above function set, the solution of the present application can realize accident detection and post-accident diversion strategy evaluation, further improving service capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1A diagram of the traffic panoramic operation and maintenance control application environment provided in the embodiment of the present application;

[0047] Figure 2 A flow chart of a traffic panoramic operation and maintenance control method provided in one embodiment of the present application;

[0048] Figure 3 A flow chart of a traffic panoramic operation and maintenance control method for straightening a road provided by another embodiment of the present application;

[0049] Figure 4 A flow chart of a traffic panoramic operation and maintenance control method including equipment twin information provided in another embodiment of the present application;

[0050] Figure 5 A schematic diagram of a twin scene of an embodiment of the present application;

[0051] Figure 6 It is a flow chart of a traffic panoramic operation and maintenance control method with contact confirmation proposed in an embodiment of the present application;

[0052] Figure 7 It is a twin picture schematic diagram showing the actual service effect on site in the embodiment of the present application;

[0053] Figure 8 This is a structural diagram of the traffic panoramic operation and maintenance control device proposed in the embodiment of the present application;

[0054] Fig. 9 This is a structural diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0056] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0057] Figure 1 This is a diagram of the traffic panoramic traffic operation and maintenance control application environment provided by the embodiment of this application. Figure 1As shown, the application environment in which the method provided by the embodiment of the present application can be implemented. Among them, the application environment includes a computer 101, a communication network 102, a multi-source perception device 103, a device pile 104 and a road device 105; the computer device and the multi-source perception device can communicate through a wired network or a wireless network; wherein the computer device can be a roadside computing unit / terminal / edge server, optionally, the computer device can also be a cloud server, a vehicle-mounted computing unit / terminal on the vehicle side, etc., and the embodiment of the present application does not specifically limit the type of computer device. Among them, the multi-source perception device is a roadside perception device or a road test multi-source fusion perception system set at the entrance of the traffic scene and a set position inside the traffic scene. For example, it can be on piles evenly distributed in the traffic scene, or it can also be a millimeter wave radar sensor, a lidar sensor, and at least one of a camera and a camera, etc., and the embodiment of the present application does not specifically limit the type of multi-source perception device.

[0058] The current perception information of the digital twin of the computer device can also be visualized, and the visualization processing module can be integrated on the computer device or set separately at other locations.

[0059] It may also include a data storage system for storing data that the computer device needs to process. The data storage system may be integrated with the computer device or placed separately on the cloud or other network.

[0060] Figure 2 A flow chart of a traffic panoramic operation and maintenance control method provided in an embodiment of the present application, this embodiment includes the following steps 210 to 230.

[0061] Step 210: Display a twin image of the traffic scene, wherein the twin image includes a twin object of the road and a frame displayed in parallel with the road.

[0062] The twin scene refers to the visual expression of the traffic scene presented on the screen, which includes but is not limited to static or dynamic scene twin displays, visual charts, rendering services and interoperability tools.

[0063] Twin objects refer to the visual expressions of objects in the real physical world, including but not limited to static objects such as buildings, landforms, and transportation facilities, as well as dynamic objects such as traffic participants (vehicles, pedestrians, etc.), meteorological environment, and traffic time.

[0064] During the use of the twin scene, the generated static twin object can always be maintained, and only the generated dynamic twin object is updated.

[0065] The twin image of the traffic scene includes a twin object of the target vehicle; and the frame includes a video image of the target vehicle.

[0066] Furthermore, in order to achieve the positional alignment of the twin object of the road and the frame, it is necessary to read the information of at least one position of the road. At least one position in the road scene is mapped as a first position in the road twin object and mapped as a second position in the frame. In the twin scene, the first position and the second position are aligned in a direction perpendicular to the centerline of the road.

[0067] Step 220: Acquire a video image of the road scene, where the video image is collected by a perception system set up on the road.

[0068] Further, current perception information of the target vehicle on the road scene is obtained, where the current perception information is obtained through a perception system set on the road;

[0069] The perception system can be a combination of one or more of a smart base station, a smart camera, a millimeter-wave radar sensor, or a lidar sensor. In actual application, the perception system set up in areas such as roads or tunnels to be controlled can perceive vehicles in real time to obtain perception information of the vehicles.

[0070] The current perception information in the embodiment of the present application refers to the perception information at the current moment. In the present application, the current moment generally refers to any moment when the perception system perceives dynamic or static objects in the road scene, that is, the computer device can receive the perception information sent by the perception system of at least a part of the objects or all objects in the road scene perceived at the current moment (any moment).

[0071] To achieve identity recognition, the captured information and perceived information of the target vehicle can be obtained in the set area of ​​the road scene. The identity information of the target vehicle is displayed based on the captured information of the target vehicle. In the twin picture of the traffic scene, the twin object of the target vehicle and the identity information of the target vehicle are bound to generate vehicle fusion information.

[0072] Step 230: Determine the position of the frame for displaying the video image according to the acquisition position of the video image.

[0073] At least one position in the road scene is mapped as a first position in the road twin object and mapped as a second position in the frame. Furthermore, in order to achieve the correspondence between the twin object of the target vehicle on the road and the spatial position of the video image in the frame, in the twin scene, the first position and the second position are aligned in a direction perpendicular to the centerline of the road.

[0074] Furthermore, in order to achieve synchronous movement of the twin object of the target vehicle on the road and the target vehicle in the video image in the frame, the image processing process and the twin object display process are synchronized. For example, the equipment is timed by a satellite system and calibrated regularly. When merging the video data and the road twin data, the timing of twin object display is determined based on the timestamps in the video image and the twin data, so that at least one moment of the twin scene and at least one position of the target vehicle on the road scene are mapped to a third position in the twin object of the road and to a fourth position in the video image in the frame, and the third position and the fourth position are aligned in a direction perpendicular to the centerline of the road.

[0075] Furthermore, in the twin scene, based on the identity information of the twin object bound to the target vehicle, the identity information of the vehicle simultaneously displayed in the video image at at least one moment can also be determined.

[0076] Furthermore, a plurality of frames arranged along the moving direction of the twin object of the target vehicle may be included in the twin scene. According to the video image acquisition position corresponding to the position of the target vehicle, a frame for displaying the video image of the target vehicle is determined.

[0077] Figure 3 A flow chart of a traffic panoramic operation and maintenance control method for road straightening provided in another embodiment of the present application includes steps 310 to 320.

[0078] Step 310: Acquire three-dimensional geographic information data.

[0079] Based on high-precision map data, static objects including but not limited to roads, building gantries, roadside poles, guardrails, LED screens, trees, etc. in real scenes are modeled. Terrain and landforms are modeled by sensing data and importing 3D GSI data, and digital support is provided for the definition of urban terrain.

[0080] In the prior art, a twin instance object of a specific terrain is created through an API, and other systems can interact with the object during runtime, including data interaction and API calls.

[0081] In the present application, the display process of the road twin object is performed based on the two-dimensional data after preprocessing the three-dimensional geographic information data in step 320 .

[0082] Step 320: In the twin scene, a rectangular coordinate system is established with the centerline direction of the road as the x direction and the width direction of the road as the y direction to display a two-dimensional graphic of the road; the x coordinate of any point in the two-dimensional graphic corresponds to the cumulative length of the position of the arbitrary point in the three-dimensional geographic information data along a direction parallel to the centerline of the road.

[0083] In this way, the image of the road is "straightened" into a two-dimensional figure with the coordinate of the center line of the road in the y direction unchanged (for example, y = 0). Correspondingly, the y coordinate of any point in the twin object of the road corresponds to the distance between the position of the arbitrary point in the three-dimensional geographic information data and the center line of the road.

[0084] Furthermore, the positions of twin objects of static objects such as the collection position on the twin scene are also processed according to the above algorithm. That is to say, the x-coordinate of any position in the twin scene is based on the centerline direction of the road as the x-direction and the width direction of the road as the y-direction. The rectangular coordinate system is established to correspond to the cumulative length of the position of any point in the three-dimensional geographic information data along the direction parallel to the centerline of the road.

[0085] In one of the embodiments, in the twin scene, the x-coordinate of the acquisition position of the video image is a rectangular coordinate system established with the centerline direction of the road as the x-direction and the width direction of the road as the y-direction, which corresponds to the cumulative length of the acquisition position in the geographic information data along the direction parallel to the centerline of the road.

[0086] Furthermore, the positions of the twin objects of static objects such as road equipment on the twin scene are also processed according to the above algorithm, that is, the x-coordinate of the twin object of the static object in the two-dimensional graph is determined, which corresponds to the cumulative length of the static object at the road scene along the direction parallel to the center line of the road; the y-coordinate of the twin object of the static object in the two-dimensional graph is determined, which corresponds to the relative position of the static object at the road scene along the road width direction.

[0087] Furthermore, the position of the dynamic twin object of the traffic participant (such as the target vehicle, etc.) on the twin scene is also processed according to the above algorithm, that is, the x-coordinate of the twin object of the traffic participant in the two-dimensional graph is determined, which corresponds to the cumulative length of the position of the traffic participant at the road scene along the direction parallel to the center line of the road; the y-coordinate of the twin object of the traffic participant in the two-dimensional graph is determined, which corresponds to the relative position of the position of the traffic participant at the road scene along the road width direction.

[0088] The cumulative length can be a relative position, indicating the distance between the current position and the starting point. The starting point can be customized.

[0089] For example, the first position of the present application is 2200 meters away from the starting point, and the coordinates of the first position in the two-dimensional graph are values ​​corresponding to 2200. When mapped to the screen, the position of the first position in the twin scene is also a value corresponding to 2200.

[0090] For example, in the third position of the present application, the target vehicle is located at a distance of 2200 meters from the starting point, and the coordinates of the third position in the two-dimensional graph are values ​​corresponding to 2200. When mapped to the screen, the position of the twin object of the target vehicle in the twin scene is also a value corresponding to 2200.

[0091] Furthermore, the optimal embodiment of the present application locates the positions of twin objects of a large number of physical entities in the twin scene by stake numbers.

[0092] For example, according to the stake numbers arranged along the road, at least one position in the road scene is determined, mapped as a first position in the road twin object and mapped as a second position in the frame;

[0093] For example, the position of the target vehicle is determined based on the stake numbers arranged along the road, and is mapped as the third position in the twin object of the road and as the fourth position in the frame;

[0094] For example, according to the stake numbers arranged along the road, the cumulative length of the position of any point in the three-dimensional geographic information data along the direction parallel to the center line of the road is determined.

[0095] It should be noted that the geographic location of the road equipment (such as a multi-source perception device including a laser radar) included in the geographic information system can be used to calculate the cumulative length based on the geographic location information. Furthermore, after the perception information of the target vehicle is acquired through the multi-source perception device, at least the relative position information between the target vehicle and the location of the multi-source perception device can be determined. Based on the cumulative length of the location of the road equipment along the direction parallel to the center line of the road, the cumulative length of the location of the target vehicle in the road scene along the direction parallel to the center line of the road can be determined; or, based on the x-coordinate of the twin object of the road equipment in the two-dimensional graph, and the relative position information between the target vehicle and the location of the road equipment, the x-coordinate of the target vehicle in the two-dimensional graph can be determined.

[0096] When the above-mentioned cumulative lengths are represented by the information of the pile number, or when the x-coordinate corresponds to the pile number, the cumulative lengths or the x-coordinates can be determined according to the pile numbers arranged along the road.

[0097] Furthermore, the present application uses information indicating the pile number to indicate the cumulative length, and any position-related information of the present application can be indicated by the pile number, for example, the acquisition position of the video image, the road position, the position of the traffic participant (including the target vehicle), the position of the road equipment, etc.

[0098] It should be noted that in the present application, the first position, the third position, and the position of the road equipment can all be expressed by pile numbers. For example, the pile number is an integer, measured in meters. An example of a common expression for positioning a highway section is: T20+527 is the pile number of the equipment pile, the letter T represents the up or down direction, 20 is kilometers, and 527 is meters. The position of the twin object can be uniformly expressed by the pile number, or the position resolution expression can be further improved based on the pile number.

[0099] It should also be noted that one equipment pile can be set every 100 meters. For example, the entrance to the tunnel is taken as the reference, where the pile number is T20+200. If the target vehicle is at the entrance, its pile number position is T20+200. After driving forward 1 meter, its real-time position expressed by the pile number position is T20+201; if it has not entered the tunnel and is 2 meters away from the entrance, its real-time position expressed by the pile number position is T20+198.

[0100] It should also be noted that the pile number twin expresses the position in the pile number coordinate system, and there is a mapping relationship with the geodetic coordinate system, that is, the longitude and latitude of the vehicle road in the geodetic coordinate system are mapped to the preprocessed two-dimensional data space through this mapping relationship. In this way, the pile number twin maps the high-precision map twin to the "pile number twin map".

[0101] Figure 4 A flow chart of a traffic panoramic operation and maintenance control method including equipment twin information provided in another embodiment of the present application includes steps 410 to 430.

[0102] Step 410: Obtain the type and location information of road equipment.

[0103] The road equipment of the present application includes traffic facilities and equipment installed in the road or on the side of the road, including but not limited to indication facilities, playback facilities (such as LED screens, directional sound playback equipment), lighting equipment (street lights, tunnel lights), gate facilities, tunnel facilities (fans, escape doors), sensing equipment (radars, cameras), communication equipment, etc.

[0104] Step 420: In the twin scene, display the twin object of the road equipment according to the type and location information of the road equipment.

[0105] The digital twin platform includes a twin object update interface for specific road equipment, and models and displays twin objects of road equipment based on high-precision map data.

[0106] In one embodiment, the position information of the road equipment is displayed based on the position coordinates in the two-dimensional graphics of step 320, and the twin object of the road equipment is displayed at the corresponding position in the twin scene.

[0107] In one embodiment, preferably, the position information of the road equipment is represented by the pile number information, and the twin object of the road equipment is displayed at the corresponding position in the twin scene.

[0108] Step 430: Obtain current working status information and / or current working content information of the road equipment, and display the dynamic twin object of the road equipment in the twin scene according to the current working status information and / or current working content information.

[0109] The current working state information refers to information used to indicate the working state of the road equipment at the current moment; the current working content information refers to information used to indicate the working content of the road equipment at the current moment.

[0110] Working status, such as the switching status of the device or switching function performance such as working level, working progress, etc.; working content, such as information function performance such as the device's prompt information, transmission information, etc.

[0111] The current working status information and working content information are obtained in real time through monitoring and communication means, which is the real working information of road equipment.

[0112] For example, a computer device may receive perception information of selected road equipment at a road scene perceived by the perception system at the current moment (any moment), and obtain the current working status information and current working content information of the road equipment.

[0113] For another example, the computer device may read the monitoring or management resources of the service device through communication according to the selected service device, and obtain its response to the current working status and working content of the device.

[0114] Figure 5 This is a schematic diagram (partial) of a twin scene of an embodiment of the present application, which includes twin objects of roads, equipment and vehicles, and video frames arranged in a two-dimensional graphic along the road. Symbols such as YK6+900 are video acquisition positions expressed by pile numbers, that is, the locations of the devices for video acquisition. In at least one frame, a video image 51 of a target vehicle is provided, and the position of the frame corresponds to the position of the twin object 52 of the target vehicle. And, an example 53 of a vehicle identification bound to the twin object of the vehicle. Road equipment, such as a directional sound broadcasting device 54 (a horn-shaped object in the figure), is located at pile number YK6+900. The figure also includes other equipment such as road lighting equipment 55, signboards 56, etc.

[0115] Figure 6 6 is a flow chart of a traffic panoramic operation and maintenance control method with contact confirmation proposed in an embodiment of the present application, including steps 610 to 640:

[0116] Step 610: An event occurs.

[0117] In response to the video image or the twin object, the preset working content and / or preset working status of the road equipment within the range that meets the set contact condition in the road is determined.

[0118] The preset working state refers to the working state of preset responses to road equipment for video images or twin object recognition features that meet the set conditions according to the traffic control strategy. For example, when the target vehicle drives into the set control area, the position of the twin object of the target vehicle overlaps with the position of the twin object of the set control area, and the preset response working state is the change of the street lamp lighting intensity level.

[0119] Preset work content refers to the work content of preset responses to road equipment based on the video image or twin object recognition features that meet the set conditions according to the traffic control strategy. For example, when the target vehicle changes lanes illegally while driving, the change in the position of the twin object of the target vehicle in the road twin object exceeds the set rules, and the preset response work content is a reminder message containing the target vehicle identification and behavior description.

[0120] Step 620: Control measures.

[0121] An instruction to execute a preset work content and / or a preset work status is sent to the road equipment.

[0122] It should be noted that the equipment and facilities in the real physical world are given a unique identity in the information model of the twin picture. Accordingly, when instructions are sent to road equipment through the communication network, the instructions are delivered according to the corresponding physical address of the equipment.

[0123] Step 630: Dynamic twinning.

[0124] In response to the control measures of step 620, the current working status information and / or current working content information of the road equipment is obtained, and in the twin scene, the dynamic twin object of the road equipment is displayed according to the current working status information and / or current working content information of the road equipment.

[0125] For example, according to the corresponding physical address of the road equipment, the monitoring or management resources of the service equipment are read to obtain its response to the current working status and working content of the equipment; for another example, the road equipment is monitored through the perception system to obtain the current working status information and current working content information of the road equipment.

[0126] Step 640: confirmation of reaching.

[0127] Compare the positions of the dynamic twin object and the target vehicle twin object to determine whether the road equipment meets the set contact conditions.

[0128] Further, within the range that meets the distance threshold, based on the digital twin object of the road equipment, compare the current working status information with the preset working status information, and / or compare the current working content information with the preset working content information, to determine that the road equipment meets the set reach condition.

[0129] Figure 7 This is a schematic diagram (partial) of the digital twin picture showing the actual on-site service effect in the embodiments of the present application, which includes digital twin objects of roads and equipment. The equipment includes information board equipment 71, directional sound equipment 72, fans 73, direction signs 74, lighting 75, gates 76, etc.

[0130] For example, based on the digital twin object of the information board at the position of YK6+550, a digital twin object 77 of a prompt message, such as "Do not speed, license plate number: 粤AGV***", is added to further form a digital twin object of a dynamic information board. Another example is that based on the digital twin object of the directional sound equipment at the position of YK6+620, a digital twin object 78 of a prompt message, namely "Do not speed, license plate number: 粤AGV***", is added to further form a digital twin object of a dynamic directional sound equipment.

[0131] Another example is that the digital twin object of the directional sound equipment at the position of YK6+340 is represented by a certain chromaticity (and, brightness or grayscale) as the state of unbroadcast information, and the digital twin object of the directional sound equipment at the position of YK6+620 is rendered by another chromaticity (and, brightness or grayscale) to represent the state of being broadcast.

[0132] In this embodiment, the prompt message, that is, the working content information described above, in the illustrated digital twin picture, the working content information shown is the digital twin information identifying the actual working content of the current equipment.

[0133] In this embodiment, the working status identified by different colors, that is, the working status information described above, in the illustrated digital twin picture, the working status information shown is the digital twin information representing the actual working status of the current equipment.

[0134] Among them, when the target vehicle "粤AGV***" speeds during driving, the change in the position of the digital twin object of the target vehicle in the position of the road digital twin object exceeds the set rule, and the preset response working content is a reminder message "Do not speed, license plate number: 粤AGV***" including the target vehicle identification and behavior description.

[0135] In this embodiment, by comparing the digital twin of the directional sound equipment or the information board with the position of the digital twin of the target vehicle, it is possible to evaluate whether the directional sound equipment or the information board meets the set reach condition.

[0136] Figure 8This is a structural diagram of the traffic panoramic operation and maintenance control device proposed in an embodiment of the present application. The device includes an acquisition module 811, a display module 812, a determination module 813, a control module 815, and a comparison module 816.

[0137] The image acquisition module is used to acquire video images of the road scene. The video images are collected by a perception system set up in the road.

[0138] The determination module is used to determine the position of the frame for displaying the video image according to the acquisition position of the video image.

[0139] The display module is used to display the twin picture of the traffic scene, and the twin picture includes the twin object of the road and the picture frame displayed in parallel with the road.

[0140] In one embodiment, the device also includes a management and control module, which is used to: determine the preset working content and / or preset working status of road equipment within the range that meets the set contact conditions in the road in response to the video image or twin object; and send an instruction to the road equipment to execute the preset working content and / or preset working status.

[0141] In one embodiment, the device further includes a comparison module for comparing the positions of the dynamic twin object and the target vehicle twin object to determine whether the road equipment meets the set contact conditions.

[0142] In one embodiment, the acquisition module further includes a video image acquisition unit 141 for acquiring a video image of a road scene;

[0143] In one embodiment, the acquisition module further includes a vehicle perception information acquisition unit, which is also used to acquire current perception information of the target vehicle on the road scene, where the current perception information is obtained through a perception system set on the road. The twin image of the traffic scene includes a twin object of the target vehicle; and the frame includes a video image of the target vehicle.

[0144] In one embodiment, the acquisition module may also include a capture unit and a binding unit.

[0145] A capture unit is used to obtain the capture information of the target vehicle and the perception information of the target vehicle in a set area on the road scene;

[0146] The binding unit is used to bind the twin object of the target vehicle and the identity information of the target vehicle in the twin picture of the traffic scene.

[0147] In one embodiment, the acquisition module further includes a road equipment acquisition unit for acquiring type and location information of road equipment.

[0148] In one embodiment, the display module is further used to include a plurality of frames arranged along the moving direction of the twin object of the target vehicle in the twin scene.

[0149] In one embodiment, the display module is also used for at least one position in the road scene, which is mapped as a first position in the road twin object and mapped as a second position in the picture frame. And / or, at least one position of the target vehicle in the road scene is mapped as a third position in the road twin object and mapped as a fourth position in the video image in the picture frame.

[0150] In one embodiment, the display module further includes at least one of a picture display unit, an alignment unit, a synchronization unit, a road equipment display unit, and a dynamic information display unit.

[0151] in,

[0152] The image display unit is used to display a twin image of a traffic scene, wherein the twin image includes a twin object of a road and a frame displayed in parallel with the road. Furthermore, the twin image includes multiple frames arranged along the moving direction of the twin object of the target vehicle.

[0153] An alignment unit is used to align the first position and the second position in a direction perpendicular to the center line of the road in the twin scene.

[0154] A synchronization unit is used to align the third position and the fourth position in a direction perpendicular to the center line of the road at at least one moment in the twin image.

[0155] The road equipment display unit is used to display the twin object of the road equipment in the twin scene according to the type and location information of the road equipment.

[0156] A dynamic information display unit is used to obtain the current working status information and / or current working content information of the road equipment, and to display the dynamic twin object of the road equipment in the twin scene based on the current working status information and / or current working content information; further, it is also used to display the dynamic twin object of the road equipment in the twin scene based on the current working status information and / or current working content information of the road equipment.

[0157] In one embodiment, the determination module is also used to determine a frame for displaying a video image of the target vehicle based on a video image acquisition position corresponding to the position of the target vehicle; in one embodiment, the determination module is also used to determine, in a twin scene, the identity information of a vehicle displayed simultaneously in the video image at at least one moment based on the identity information of a twin object bound to the target vehicle.

[0158] In one embodiment, the determination module is further configured to:

[0159] Determine at least one position in the road site according to the stake numbers arranged along the road, and map it as a first position in the road twin object and as a second position in the frame;

[0160] According to the stake numbers arranged along the road, the position of the target vehicle is determined, and is mapped as the third position in the twin object of the road and as the fourth position in the frame;

[0161] According to the stake numbers arranged along the road, the cumulative length of the position of any point in the three-dimensional geographic information data along the direction parallel to the center line of the road is determined.

[0162] Furthermore, the determination module also includes at least one of a picture frame position determination unit, a picture frame determination unit, a vehicle identity determination unit, a road position determination unit, and a vehicle position determination unit.

[0163] in,

[0164] The frame position determination unit is used to determine the position of the frame used to display the video image according to the acquisition position of the video image.

[0165] The frame determination unit is used to determine a frame for displaying the video image of the target vehicle according to a video image acquisition position corresponding to the position of the target vehicle.

[0166] The vehicle identity determination unit is used to determine the identity information of the vehicle displayed simultaneously in the video image at at least one moment in the twin scene based on the identity information of the twin object bound to the target vehicle.

[0167] A road position determination unit is used to determine at least one position in the road site according to the pile numbers arranged along the road, and map it as a first position in the road twin object and as a second position in the frame; and / or, to determine the cumulative length of the position of any point in the three-dimensional geographic information data along a direction parallel to the centerline of the road according to the pile numbers arranged along the road.

[0168] The vehicle position determination unit is used to determine the position of the target vehicle according to the pile numbers arranged along the road, and map it as the third position in the twin object of the road and as the fourth position in the frame.

[0169] In one embodiment of the present application, a preprocessing module is used to obtain geographic information data; in a twin image, a rectangular coordinate system is established with the centerline direction of the road as the x direction and the width direction of the road as the y direction to display a two-dimensional graph of the road; the x coordinate of any point in the two-dimensional graph corresponds to the cumulative length of the position of any point in the geographic information data along a direction parallel to the centerline of the road.

[0170] In one embodiment, the preprocessing module includes a road coordinate transformation unit for implementing the above-mentioned preprocessing module functions; further, it also includes a device coordinate transformation unit for implementing device position coordinate transformation; the device position coordinate transformation determines the x-coordinate of the device's twin object in the two-dimensional graphic, corresponding to the cumulative length of the device's position at the road site along a direction parallel to the center line of the road; determines the y-coordinate of the device's twin object in the two-dimensional graphic, corresponding to the relative position of the device at the road site along the road width direction.

[0171] In one embodiment, the determination module further includes a vehicle coordinate transformation unit for implementing the target vehicle position coordinate transformation.

[0172] The target vehicle position coordinates are transformed to determine the x-coordinate of the target vehicle's twin object in the two-dimensional graph, which corresponds to the cumulative length of the target vehicle's position at the road site along the direction parallel to the road centerline; the y-coordinate of the target vehicle's twin object in the two-dimensional graph is determined to correspond to the relative position of the target vehicle at the road site along the road width direction (i.e., the distance perpendicular to the road centerline).

[0173] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] Therefore, the present application also proposes a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any embodiment of the present application is implemented.

[0175] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0178] Fig. 9 This is a structural diagram of a computer device in an embodiment of the present application.

[0179] Furthermore, the present application also proposes a computer device (or electronic device), including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present application when executing the computer program.

[0180] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. Memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium. Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0181] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0182] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A traffic panoramic operation and maintenance control method, characterized in that: include: Displaying a twin picture of a traffic scene, wherein the twin picture includes a twin object of a road and a picture frame displayed in parallel; Acquire a video image of a road scene, wherein the video image is collected by a perception system arranged on the road; The position of a frame for displaying the video image is determined according to the acquisition position of the video image.

2. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: The twin scene includes a plurality of frames arranged along a road direction; According to the video image acquisition position corresponding to the position of the target vehicle, a frame for displaying the video image of the target vehicle is determined.

3. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: At least one position in the road scene is mapped as a first position in the twin object of the road and is mapped as a second position in the frame; In the twin view, the first position and the second position are aligned in a direction perpendicular to a center line of the road.

4. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: The current perception information of the target vehicle at the road scene is obtained, wherein the current perception information is obtained through a perception system set up on the road; the twin scene of the traffic scene includes a twin object of the target vehicle; and the frame includes a video image of the target vehicle.

5. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: At least one position of a target vehicle at a road scene is mapped as a third position in the twin object of the road and mapped as a fourth position in the video image in the frame; At at least one moment in the twin view, the third position and the fourth position are aligned in a direction perpendicular to a centerline of the road.

6. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: In a set area on the road scene, acquiring captured information of a target vehicle and perceived information of the target vehicle; Generating identity information of the target vehicle according to the captured information of the target vehicle; In the twin picture of the traffic scene, the twin object of the target vehicle and the identity information of the target vehicle are bound.

7. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: In the twin scene, the identity information of the vehicle simultaneously displayed in the video image at at least one moment is determined according to the identity information of the twin object bound to the target vehicle.

8. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: The steps to display the twin object of the road are: Obtain geographic information data; In the twin scene, a rectangular coordinate system is established with the centerline direction of the road as the x direction and the width direction of the road as the y direction to display a two-dimensional graph of the road; The x-coordinate of any point in the two-dimensional graph corresponds to the cumulative length of the position of the any point in the geographic information data along a direction parallel to the centerline of the road.

9. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: Obtain information on the type and location of road equipment; In the twin scene, the twin object of the road equipment is displayed according to the type and location information of the road equipment.

10. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: The current working state information and / or the current working content information of the road equipment is obtained, and in the twin scene, the dynamic twin object of the road equipment is displayed according to the current working state information and / or the current working content information.

11. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that At least one of the following: Determine at least one position in the road scene according to stake numbers arranged along the road, and map it as a first position in the twin object of the road and as a second position in the frame; Determine the position of the target vehicle according to the stake numbers arranged along the road, and map it as a third position in the twin object of the road and as a fourth position in the frame; According to the stake numbers arranged along the road, the cumulative length of the position of any point in the geographic information data along the direction parallel to the center line of the road is determined.

12. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: In response to the video image or the twin object, determining the preset working content and / or preset working state of the road equipment within the range that meets the set contact condition in the road; An instruction to execute the preset working content and / or the preset working state is sent to the road equipment.

13. The traffic panoramic operation and maintenance control method according to claim 1, characterized in that: In the twin scene, a dynamic twin object of the road equipment is displayed according to current working state information and / or current working content information of the road equipment; Compare the positions of the dynamic twin object and the target vehicle twin object to determine whether the road equipment meets the set contact conditions.

14. A traffic panoramic operation and maintenance control device, characterized in that: The device comprises: An image acquisition module, used to acquire a video image of a road scene, wherein the video image is collected by a perception system arranged in the road; A display module, used to display a twin picture of a traffic scene, wherein the twin picture includes a twin object of a road and a picture frame displayed in parallel; The determination module is used to determine the position of the frame used to display the video image according to the acquisition position of the video image.

15. A traffic panoramic operation and maintenance control device according to claim 15, characterized in that: The device comprises The control module is used to: In response to the video image or the twin object, determining the preset working content and / or preset working state of the road equipment within the range that meets the set contact condition in the road; An instruction to execute the preset working content and / or the preset working state is sent to the road equipment.

16. A traffic panoramic operation and maintenance control device as claimed in claim 15, characterized in that: The device comprises: Comparison module, used to Compare the positions of the dynamic twin object and the target vehicle twin object to determine whether the road equipment meets the set contact conditions.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

18. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 13 is implemented.

Images