Abnormity monitoring method and device for road traffic network, and storage medium

Abstract

The invention discloses an abnormity monitoring method and device for a road traffic network and a storage medium. The method comprises the following steps: constructing a graph structure which represents a road communication relationship in a road traffic network, and then obtaining satellite image data which is used for judging whether traffic network nodes are abnormal or not. And enabling the graph nodes and the edges to correspond to traffic network nodes and roads in the satellite image data, and determining attribute information of a first graph node and associated edges in the graph structure. Furthermore, based on the graph neural network, node features of the second graph nodes can be determined, and based on the node features, abnormal conditions of the second graph nodes are predicted. By combining the satellite image data and the graph structure representing the topological structure of the road traffic network, whether the traffic network node which is unknown to be abnormal or not is abnormal or not is judged through the traffic network node (and the road) which is known to be abnormal or not, so that the timeliness and the accuracy of abnormal monitoring on the road traffic network are improved.

Description

Technical Field

[0001] The present application relates to the fields of satellite imagery, knowledge graphs, and road traffic analysis technology, and in particular to a method, device, and storage medium for monitoring abnormalities in a road traffic network. Background Art

[0002] With the continuous development of satellite technology, satellite image data collected by low-orbit satellites can be used to monitor the road conditions of road traffic networks.

[0003] In existing technologies, satellite image data can be identified through image recognition to monitor abnormal road conditions in road traffic networks, such as determining whether there is a traffic jam, whether there is road damage, or whether a bridge has collapsed.

[0004] Existing technologies for identifying satellite imagery data and monitoring road traffic network anomalies have certain shortcomings. For example, in cloudy or foggy weather, some traffic network nodes may be obscured from the satellite's view, making it impossible to determine whether anomalies have occurred at these nodes using satellite imagery data. Consequently, existing technologies cannot determine whether anomalies have occurred at nodes not covered by satellite imagery data, reducing the timeliness and accuracy of anomaly monitoring of road traffic networks.

[0005] With respect to the technical problems existing in the above-mentioned prior art of abnormal monitoring of road traffic networks using satellite image data, such as low timeliness and low accuracy, no effective solution has been proposed so far. Summary of the Invention

[0006] The embodiments of the present disclosure provide a method, device and storage medium for abnormal monitoring of road traffic networks, so as to at least solve the technical problems existing in the prior art of low timeliness and low accuracy in abnormal monitoring of road traffic networks using satellite image data.

[0007] According to one aspect of an embodiment of the present disclosure, a method for monitoring abnormalities in a road traffic network is provided, comprising: determining a graph structure corresponding to a road traffic network in a predetermined area, the graph structure comprising graph nodes and edges connecting two graph nodes, wherein the graph nodes are used to represent traffic network nodes of the road traffic network, and the edges connecting two graph nodes are used to represent roads between the corresponding traffic network nodes; obtaining satellite image data corresponding to the predetermined area; determining attribute information corresponding to a first graph node and associated edges in the graph structure based on the satellite image data; constructing first graph data corresponding to the graph structure based on the attribute information; performing message transmission on the first graph data using a graph neural network to determine a first graph feature corresponding to the first graph data, and determining a node feature of a second graph node of the graph structure based on the first graph feature, wherein the satellite image data does not include a traffic network node corresponding to the second graph node; and determining whether there is an abnormality in the traffic network node corresponding to the second graph node based on the node feature of the second graph node.

[0008] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0009] According to another aspect of an embodiment of the present disclosure, a device for monitoring anomalies in a road traffic network is also provided, including: a graph structure determination module, used to determine a graph structure corresponding to a road traffic network in a predetermined area, the graph structure including graph nodes and edges connecting two graph nodes, wherein the graph nodes are used to represent traffic network nodes of the road traffic network, and the edges connecting two graph nodes are used to represent roads between the corresponding traffic network nodes; an image acquisition module, used to acquire satellite image data corresponding to the predetermined area; an attribute determination module, used to determine attribute information corresponding to a first graph node and associated edges in the graph structure based on the satellite image data; a graph data construction module, used to construct first graph data corresponding to the graph structure based on the attribute information; a feature determination module, used to perform message transmission on the first graph data using a graph neural network, determine a first graph feature corresponding to the first graph data, and determine a node feature of a second graph node of the graph structure based on the first graph feature, wherein the satellite image data does not include a traffic network node corresponding to the second graph node; and an anomaly determination module, used to determine whether there is an anomaly in the traffic network node corresponding to the second graph node based on the node feature of the second graph node.

[0010] According to another aspect of an embodiment of the present disclosure, a device for monitoring anomalies in a road traffic network is also provided, comprising: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: determining a graph structure corresponding to a road traffic network in a predetermined area, the graph structure comprising graph nodes and edges connecting two graph nodes, wherein the graph nodes are used to represent traffic network nodes of the road traffic network, and the edges connecting two graph nodes are used to represent roads between the corresponding traffic network nodes; obtaining satellite image data corresponding to the predetermined area; determining attribute information corresponding to a first graph node and associated edges in the graph structure based on the satellite image data; constructing first graph data corresponding to the graph structure based on the attribute information; performing message transmission on the first graph data using a graph neural network, determining a first graph feature corresponding to the first graph data, and determining a node feature of a second graph node of the graph structure based on the first graph feature, wherein the satellite image data does not include a traffic network node corresponding to the second graph node; and determining whether there is an abnormality in the traffic network node corresponding to the second graph node based on the node feature of the second graph node.

[0011] In an embodiment of the present disclosure, a graph structure representing the road connectivity relationship in a road traffic network is constructed, and then satellite image data is obtained for determining whether an abnormality has occurred in a traffic network node. Furthermore, by matching the graph nodes and edges in the graph structure with the traffic network nodes and roads that can be determined by satellite image data to determine whether an abnormality has occurred, the corresponding attribute information of the first graph node and the associated edge in the graph structure can be determined. Furthermore, based on the graph neural network, the node features of the second graph node, that is, the graph node corresponding to the traffic network node for which it is unknown whether an abnormality has occurred, can be determined. Furthermore, based on the node features, the abnormality of the second graph node can be predicted. In an embodiment of the present disclosure, by combining satellite image data with a graph structure representing the topological structure of a road traffic network, it is determined whether an abnormality has occurred in a traffic network node for which it is unknown whether an abnormality has occurred, thereby improving the timeliness and accuracy of abnormality monitoring of the road traffic network. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of this application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings: Figure 1 is a schematic diagram of a system for monitoring abnormalities in a road traffic network based on satellite images according to Example 1 of the present disclosure; Figure 2A is a schematic diagram of the hardware architecture of the satellite system 10 according to Embodiment 1 of the present disclosure; Figure 2Bis a schematic diagram of the hardware architecture of the ground system 20 according to Embodiment 1 of the present disclosure; Figure 3 1 is a flow chart of a method for monitoring abnormalities in a road traffic network according to the first aspect of Embodiment 1 of the present disclosure; Figure 4 is a schematic diagram of a graph structure provided by Example 1 of the present disclosure; Figure 5 This is a schematic diagram of the display format of the first image data provided in Example 1 of the present disclosure; Figure 6 This is a schematic diagram of the display format of the second graph data provided in Example 1 of the present disclosure; Figure 7A This is a schematic diagram of a first message transmission process provided by Example 1 of the present disclosure; Figure 7B This is a schematic diagram of a second message transmission process provided by Example 1 of the present disclosure; Figure 8 is a schematic diagram of an abnormality monitoring device for a road traffic network according to the first aspect of embodiment 2 of the present disclosure; and Figure 9 It is a schematic diagram of the abnormality monitoring device for a road traffic network according to the first aspect of Example 3 of the present disclosure. DETAILED DESCRIPTION

[0013] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0014] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0015] Example 1 According to this embodiment, an embodiment of a method for monitoring abnormalities in a road traffic network is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0016] Figure 1 A schematic diagram of a system for monitoring road traffic network anomalies based on satellite imagery according to this embodiment is shown. The system includes a ground system 20 and a satellite system 10. Satellite system 10 continuously collects satellite image data from the ground and transmits it to ground system 20. Ground system 20 receives the satellite image data from satellite system 10 and uses it to monitor anomalies in the road traffic network. The specific method for monitoring anomalies in the road traffic network based on satellite imagery data will be described in detail below.

[0017] Figure 2A It further shows Figure 1 Schematic diagram of the hardware architecture of the satellite system 10. Figure 2A As shown, the satellite system 10 includes an integrated electronic system, which includes: a processor, a memory, a bus management module and a communication interface. The memory is connected to the processor, so that the processor can access the memory, read the program instructions stored in the memory, read data from the memory or write data to the memory. The bus management module is connected to the processor and is also connected to a bus such as a CAN bus. The processor can communicate with the onboard peripherals connected to the bus through the bus managed by the bus management module. In addition, the processor is also connected to devices such as cameras, star sensors, measurement and control transponders, and data transmission equipment via the communication interface. It can be understood by those skilled in the art that Figure 2A The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 2A More or fewer components than shown, or with Figure 2A Different configurations shown.

[0018] Figure 2B It further shows Figure 1 Schematic diagram of the hardware architecture of the ground system 20. Figure 2BAs shown, the ground system 20 may include one or more processors (the processors may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, transmission device, and input / output interface are connected to the processor via a bus. In addition, it may also include a display, keyboard, and cursor control device connected to the input / output interface. Those skilled in the art will understand that Figure 2B The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 2B More or fewer components than shown, or with Figure 2B Different configurations shown.

[0019] It should be noted that Figure 2A and Figure 2B The one or more processors and / or other data processing circuits shown in the figure may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computing device. As discussed in the embodiments of the present disclosure, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0020] Figure 2A and Figure 2B The memory shown in the figure can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for monitoring road traffic network anomalies in the embodiments of the present disclosure. The processor executes the software programs and modules stored in the memory to perform various functional applications and data processing, thereby implementing the method for monitoring road traffic network anomalies in the aforementioned application. The memory can include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.

[0021] It should be noted that, in some optional embodiments, the above Figure 2A and Figure 2B The devices shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. Figure 2A and Figure 2B This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the apparatus described above.

[0022] In the above operating environment, according to the first aspect of this embodiment, a method for monitoring abnormalities in a road traffic network is provided. The method is implemented by the ground system 20 shown in FIG. 2 . Figure 3 A schematic diagram showing the process of the method is shown in FIG. Figure 3 As shown, the method includes: S302: Determine a graph structure corresponding to a road traffic network in a predetermined area, wherein the graph structure includes graph nodes and edges connecting two graph nodes, wherein a graph node is used to represent a traffic network node of the road traffic network, and an edge connecting two graph nodes is used to represent a road between the corresponding traffic network nodes; S304: Acquire satellite image data corresponding to the predetermined area; S306: Determine attribute information corresponding to a first graph node and an associated edge in the graph structure based on the satellite image data; S308: Constructing first graph data corresponding to the graph structure according to the attribute information; S310: performing message transmission on the first graph data using a graph neural network to determine a first graph feature corresponding to the first graph data, and determining a node feature of a second graph node of the graph structure based on the first graph feature, wherein the satellite image data does not include a transportation network node corresponding to the second graph node; and S312: Determine whether there is an abnormality in the traffic network node corresponding to the second graph node based on the node feature of the second graph node.

[0023] Specifically, refer to Figure 3 As shown, the ground system 20 determines a graph structure corresponding to the road traffic network of a predetermined area, wherein the graph structure includes graph nodes and edges connecting two graph nodes, wherein the graph nodes are used to represent traffic network nodes of the road traffic network, and the edges connecting two graph nodes are used to represent roads between the corresponding traffic network nodes (S302).

[0024] Figure 4 This is a schematic diagram of a graph structure provided in Example 1 of the present disclosure.

[0025] Figure 4 The graph nodes in the graph structure shown in are used to represent transportation network nodes. Figure 4The edge connecting two graph nodes in the graph structure shown in is used to represent the road between the two corresponding traffic network nodes. The graph structure can be constructed in advance. In addition, when constructing the graph structure, the correspondence between the graph nodes and the corresponding traffic network nodes, as well as the correspondence between the edges and the corresponding roads can be recorded, that is, the correspondence is used to indicate which traffic network node the graph node corresponds to in the actual road traffic network, and which road the edge corresponds to in the actual road traffic network. The correspondence related to the graph nodes may include the correspondence between the graph nodes and the names of the corresponding traffic network nodes and information such as geographic coordinates. Similarly, the correspondence related to the edges may include the correspondence between the edges and information such as the names of the corresponding roads and information such as geographic coordinates.

[0026] That is, the method for monitoring road traffic network anomalies provided herein can be combined with a graph structure representing the connectivity between various traffic network nodes and roads in the road traffic network to monitor road traffic network anomalies. Traffic network nodes can refer to important nodes in the road traffic network, such as intersections, T-junctions, road convergence points, and bridges. The anomalies mentioned here can include traffic congestion caused by natural disasters, traffic accidents, and traffic congestion.

[0027] Then, the ground system 20 may obtain satellite image data corresponding to the predetermined area ( S304 ), wherein the predetermined area mentioned here may refer to an area currently being monitored for abnormalities by satellite.

[0028] After acquiring the satellite image data, the ground system 20 may determine attribute information corresponding to the first graph node and the associated edges in the graph structure based on the satellite image data (S306). Specifically, the ground system 20 may first match the transportation network nodes contained in the satellite image data with the graph nodes in the graph structure to determine the first graph node in the graph structure, and then determine the attribute information corresponding to the first graph node and the associated edges in the graph structure.

[0029] The first graph node mentioned here represents a traffic network node whose anomaly can be determined using satellite imagery data. The attribute information mentioned here can represent the traffic network node and road conditions corresponding to the first graph node and its associated edges. The following section describes how to determine this attribute information.

[0030] Then, the ground system 20 may construct first graph data corresponding to the graph structure based on the attribute information ( S308 ). That is, based on the graph structure, the first graph data also carries attribute information corresponding to the first graph nodes and associated edges.

[0031] Figure 5 This is a schematic diagram of the display format of the first image data provided in Example 1 of the present disclosure.

[0032] It can be seen that Figure 5 The structure of the first graph data shown in Figure 4 The structure of the graph structure shown in is consistent with that in FIG. 1 , which can be understood as the first graph data is filled with attribute information obtained from satellite image data. Figure 4 The graph structure in . Among them, Figure 5 The white nodes shown in represent the first graph nodes, and the black nodes represent the second graph nodes mentioned below.

[0033] Furthermore, the ground system 20 can utilize a graph neural network to perform message transmission on the first graph data, determine a first graph feature corresponding to the first graph data, and determine a node feature of a second graph node in the graph structure based on the first graph feature, wherein the satellite imagery data does not include a transportation network node corresponding to the second graph node (S310). Alternatively, it can be understood that the second graph node refers to a graph node for which the satellite imagery data cannot determine whether the corresponding transportation network node is abnormal. The first graph feature can include node features corresponding to each graph node and edge features corresponding to each edge.

[0034] Finally, it may be determined whether there is an abnormality in the traffic network node corresponding to the second graph node according to the node feature of the second graph node ( S312 ).

[0035] As described in the background, existing methods for identifying satellite imagery data and subsequently monitoring road conditions in road traffic networks have certain shortcomings. For example, in cloudy or foggy weather, some traffic network nodes may be obscured from the satellite's view, making it impossible to determine whether anomalies have occurred in these nodes using satellite imagery. Furthermore, when simultaneously monitoring road conditions in a road traffic network using multiple nearby satellites, areas may be missed in the satellite-collected images, making it impossible to determine whether anomalies have occurred in these nodes using satellite imagery.

[0036] In light of this, this method constructs a graph structure representing the connectivity of roads in a road traffic network, then acquires satellite imagery data for determining whether anomalies have occurred at traffic network nodes (and roads). Furthermore, by mapping the graph nodes and edges in the graph structure to traffic network nodes and roads whose anomalies can be determined using satellite imagery data, attribute information corresponding to the first graph node and its associated edges in the graph structure can be determined. Furthermore, using a graph neural network, node features can be determined for a second graph node (i.e., a traffic network node whose anomaly is unknown). Furthermore, based on these node features, anomalies at the second graph node can be predicted.

[0037] It can be seen that this method combines satellite image data with a graph structure representing the topological structure of the road traffic network. It uses traffic network nodes (and roads) where it is known whether anomalies have occurred to determine whether anomalies have occurred at traffic network nodes where it is unknown whether anomalies have occurred, thereby improving the timeliness and accuracy of anomaly monitoring of the road traffic network.

[0038] Optionally, the operation of using a graph neural network to perform message passing on the first graph data to determine the first graph feature corresponding to the first graph data includes: using multiple pre-set graph neural networks to perform message passing on the first graph data respectively to obtain multiple second graph features corresponding to each graph neural network respectively; determining weight values ​​corresponding to the multiple graph neural networks respectively based on the second graph features; and performing weighted summation of the multiple second graph features based on the weight values ​​to determine the first graph feature.

[0039] The multiple pre-set graph neural networks mentioned above are somewhat different. For example, the multiple graph neural networks may be of the same type but with different network parameters; or, in another example, the multiple graph neural networks may be of different types. The graph neural network can specifically employ existing neural networks related to knowledge graphs, such as graph convolutional networks, and the specific type of graph neural network employed is not limited herein. The specific method for determining the weight values ​​corresponding to the multiple graph neural networks based on the second graph feature will be described below.

[0040] After determining multiple second graph features, weight values ​​corresponding to multiple graph neural networks can be determined. Each graph neural network corresponds to a second graph feature and also corresponds to a weight value. The weight values ​​of multiple image neural networks are used to perform weighted summation on the second graph features corresponding to the multiple graph neural networks to obtain the first graph feature.

[0041] Optionally, the operation of determining weight values ​​corresponding to multiple graph neural networks respectively based on the second graph feature includes: extracting at least one third graph node from the first graph node, masking the attribute information corresponding to the third graph node in the first graph data, constructing the second graph data, and using multiple graph neural networks to perform message passing on the second graph data respectively to obtain multiple third first graph features corresponding to each graph neural network respectively; determining multiple second node features corresponding to the multiple second graph features and the third graph node respectively, and determining multiple third node features corresponding to the multiple third graph features and the third graph node respectively; and determining the weight values ​​corresponding to the multiple graph neural networks respectively based on the similarity between the second node features and the third node features corresponding to each graph neural network.

[0042] First, the second graph data will be described graphically. Figure 6 This is a schematic diagram of the display format of the second graph data provided in Example 1 of the present disclosure.

[0043] refer to Figure 6 As shown, Figure 6 and Figure 5 The difference is that Figure 5 Some of the white nodes in are transformed into gray nodes, that is, Figure 6 There are three types of nodes: white nodes, black nodes and gray nodes. The gray nodes are the third graph nodes mentioned above. Figure 5 and Figure 6 It can be seen from the difference that the difference between the first graph data and the second graph data is that the second graph data is obtained by overwriting the attribute information originally possessed by some first graph nodes (ie, third graph nodes) in the first graph data.

[0044] The third graph node may be randomly extracted from the first graph node.

[0045] Figure 7A This is a flowchart of the first message transmission provided by Example 1 of the present disclosure.

[0046] Figure 7B This is a flow chart of the second message transmission provided in Example 1 of the present disclosure.

[0047] refer to Figure 7A and 7B When determining the weight values ​​corresponding to multiple graph neural networks, the ground system 20 can perform two message transfers on the graph data through the multiple graph neural networks. The first message transfer is performed on the first graph data to obtain multiple graph features corresponding to the first graph data (second graph features). The second message transfer is performed on the second graph data to obtain multiple graph features corresponding to the second graph data (third graph features).

[0048] For a third graph node, the second graph feature includes the node feature of the first graph node corresponding to the third graph node, and this node feature is used as the second node feature corresponding to the third graph node. The third graph feature includes the node feature of the third graph node as the third node feature corresponding to the third graph node. Thus, the second node feature corresponding to each graph neural network includes the second node feature corresponding to each third graph node, and the third node feature corresponding to each graph neural network includes the third node feature corresponding to each third graph node.

[0049] Then, the similarity between the second node feature and the third node feature of the same third graph node output by the same graph neural network can be determined (i.e., the similarity between the second node feature and the third node feature corresponding to each graph neural network in the above content). Based on the aforementioned similarity, the weight value corresponding to each graph neural network is determined. This similarity can be specifically determined using, for example, cosine distance.

[0050] That is, for a graph neural network, the number of similarities that can be calculated is the same as the number of third graph nodes. For example, if there is only one third graph node, each graph neural network will have a corresponding similarity (the similarity between the second node feature and the third node feature of the third graph node output by the graph neural network). These similarities can be normalized and used as the weight value corresponding to each graph neural network.

[0051] For example, if there are multiple third graph nodes, then for a graph neural network, multiple similarities corresponding to the graph neural network can be determined. For example, the multiple similarities can be averaged (or the maximum value) to obtain the average similarity value, and then the weight value corresponding to each graph neural network can be obtained based on the average similarity value corresponding to each graph neural network.

[0052] After determining the weight values ​​of each graph neural network, the ground system 20 can weight the corresponding second graph features based on the weight values ​​of each graph neural network, and then sum the weighted second graph features to obtain the first graph feature. In this way, the graph neural network, which is more effective in feature modeling through message passing, can play a greater role in determining the first graph feature, thereby improving the accuracy of anomaly judgment based on the node features of the second graph nodes contained in the first graph feature.

[0053] Since this specification primarily focuses on anomaly monitoring of traffic network nodes with unknown anomalies, the ground system 20 can determine the node features corresponding to the second graph node from the first graph features, and thereby determine whether an anomaly has occurred in the traffic network node corresponding to the second graph node based on the node features corresponding to the second graph node. Specifically, the ground system 20 can input the node features corresponding to the second graph node into a pre-trained classifier for anomaly determination, thereby using the classifier to determine whether an anomaly has occurred in the traffic network node corresponding to the second graph node.

[0054] It should be noted that the classifier in this method can be pre-trained in a supervised manner, and the graph neural network in this method can be pre-trained together with the classifier. When training the graph neural network and the classifier, the loss of the classifier can be shared with the graph neural network to enable joint training of the two.

[0055] Of course, the first graph feature determined in the above manner may also include edge features (that is, when transmitting messages through the graph neural network, not only the node features but also the edge features are updated). The edge features of the corresponding edges can be determined based on the first graph features, and then it can be determined whether an abnormality has occurred on the road corresponding to the unknown abnormal situation based on the edge features.

[0056] Optionally, before determining the attribute information corresponding to the first graph node and the associated edges in the graph structure based on the satellite image data, the method also includes: performing anomaly identification on the satellite image data for the traffic network nodes and the roads between the traffic network nodes to obtain an anomaly identification result; determining the first traffic network node based on the anomaly identification result, the first traffic network node refers to the traffic network node that can be identified as having an anomaly through the satellite image data; and determining the first graph node in the graph structure corresponding to the first traffic network node.

[0057] That is, before determining the attribute information corresponding to the first graph node and the associated edges, it is necessary to first determine which are the first graph nodes. The ground system 20 can first identify anomalies in the traffic network nodes present in the satellite image data through image recognition, and use the graph nodes corresponding to the traffic network nodes that can be identified as anomalies through the satellite image data as the first graph nodes.

[0058] Optionally, the operation of determining the attribute information corresponding to the first graph node and the associated edge in the graph structure based on the satellite image data includes: determining whether the traffic network node corresponding to the first graph node or the road corresponding to the associated edge is damaged based on the satellite image data; in the case that the corresponding traffic network node or the corresponding road is damaged, determining the attribute information of the first graph node corresponding to the corresponding traffic network node or the edge corresponding to the corresponding road based on the minimum value of the pre-set minimum vehicle spacing and / or minimum vehicle speed; in the case that the corresponding traffic network node or the corresponding road is not damaged and there is a vehicle on the corresponding traffic network node or the corresponding road In this case, the actual minimum vehicle spacing and / or the actual minimum vehicle speed in the corresponding traffic network node and the corresponding road are determined based on the satellite image data, and the attribute information of the first graph node corresponding to the corresponding traffic network node or the edge corresponding to the corresponding road is determined based on the actual minimum vehicle spacing and / or the actual minimum vehicle speed; and in the case that the corresponding traffic network node or the corresponding road is not damaged and there is no vehicle on the corresponding traffic network node or the corresponding road, the attribute information of the first graph node corresponding to the corresponding traffic network node or the edge corresponding to the corresponding road is determined based on the pre-set maximum value of the minimum vehicle spacing and / or the maximum vehicle speed.

[0059] When determining the attribute information corresponding to the first graph node and the corresponding edge, the attribute information can be determined according to different situations. As described above, this can be divided into three specific situations. The first situation is that the transportation network node or road is damaged; the second situation is that the transportation network node or road is not damaged and there are vehicles; the third situation is that the transportation network node or road is not damaged and there are no vehicles.

[0060] In the first case, since the traffic network node or road is damaged, the attribute information may be the minimum value of the set minimum vehicle spacing and / or minimum vehicle speed. The minimum vehicle spacing and / or minimum vehicle speed may be determined as the minimum value of the minimum vehicle spacing and / or minimum vehicle speed over all historical periods. Of course, the minimum value may also be manually preset.

[0061] In the second case, if the traffic network nodes or roads are not damaged and there are vehicles, the actual minimum distance between vehicles and / or the actual minimum speed of vehicles can be directly used as attribute information.

[0062] Optionally, the operation of determining the actual minimum speed of vehicles in the corresponding traffic network nodes and the corresponding roads based on the satellite image data includes: determining the actual minimum speed of vehicles in the corresponding traffic network nodes and the corresponding roads based on the satellite image data and satellite image data whose collection time interval with the satellite image data in history does not exceed a preset time period.

[0063] That is, the actual minimum speed of the vehicle can be determined by using the satellite image data that was collected at a similar time in history to the satellite image data.

[0064] In the third scenario, where the traffic network nodes or roads are intact and no vehicles are present (which can be considered an extreme case), a maximum value of a pre-set minimum vehicle spacing and / or minimum vehicle speed can be determined as attribute information. This maximum value can be determined using a method similar to the minimum value described above. For example, it can be determined as the maximum value of the statistically calculated minimum vehicle spacing and / or minimum vehicle speed over all historical periods. Of course, the maximum value of the minimum vehicle spacing and / or minimum vehicle speed can also be manually preset.

[0065] It should be noted that for the second graph nodes and edges corresponding to roads whose anomalies are unknown, the attribute information of the corresponding graph nodes and edges (i.e., the attribute information that needs to be determined using satellite imagery data) can be set to default attribute information. For example, the specific content of the attribute information can be filled with 0; in another example, the specific content of the attribute information can be set to a manually defined element (such as a letter).

[0066] The initial features of the corresponding graph nodes or edges can be obtained through the attribute information of each graph node and edge. After the message passing of the graph neural network, the initial features of each graph node and edge are updated (one round of message passing or multiple rounds of message passing can be performed), thereby obtaining the graph features containing the node features of each graph node and the edge features of each edge (regardless of whether it is the first graph feature, the second graph feature or the third graph feature, the format is the same, so they are all referred to as graph features here).

[0067] In addition, reference Figure 1 As shown, according to a third aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0068] Therefore, according to this embodiment, by combining the graph structure and satellite image data, it is possible to predict whether an anomaly has occurred at a traffic network node where anomalies are unknown, by referring to traffic network nodes where anomalies are known to exist. Furthermore, on this basis, the weights corresponding to each graph neural network are determined to weight the second graph features derived from each graph neural network to obtain the first graph features. This allows the graph neural network, which performs better feature modeling through message passing, to play a greater role in determining the first graph features, thereby improving the accuracy of anomaly judgments made using the node features of the second graph nodes contained in the first graph features.

[0069] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0070] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0071] Example 2 Figure 8 FIG. 8 shows an abnormality monitoring device 800 for a road traffic network according to the first aspect of this embodiment, which corresponds to the method according to the first aspect of embodiment 1. Figure 8 As shown, the device 800 includes: a graph structure determination module 810, which is used to determine a graph structure corresponding to a road traffic network in a predetermined area, wherein the graph structure includes graph nodes and edges connecting two graph nodes, wherein the graph nodes are used to represent traffic network nodes of the road traffic network, and the edges connecting two graph nodes are used to represent roads between the corresponding traffic network nodes; an image acquisition module 820, which is used to acquire satellite image data corresponding to the predetermined area; an attribute determination module 830, which is used to determine attribute information corresponding to a first graph node and an associated edge in the graph structure based on the satellite image data; a graph data construction module 840, which is used to construct first graph data corresponding to the graph structure based on the attribute information; a feature determination module 850, which is used to use a graph neural network to perform message transmission on the first graph data, determine a first graph feature corresponding to the first graph data, and determine a node feature of a second graph node of the graph structure based on the first graph feature, wherein the satellite image data does not include a traffic network node corresponding to the second graph node; and an anomaly determination module 860, which is used to determine whether there is an anomaly in the traffic network node corresponding to the second graph node based on the node feature of the second graph node.

[0072] Optionally, the feature determination module 850 is used to use multiple pre-set graph neural networks to perform message transmission on the first graph data respectively, to obtain multiple second graph features corresponding to each graph neural network respectively; based on the second graph features, determine the weight values ​​corresponding to the multiple graph neural networks respectively; and perform weighted summation of the multiple second graph features according to the weight values ​​to determine the first graph feature.

[0073] Optionally, the feature determination module 850 is used to extract at least one third graph node from the first graph node, mask the attribute information corresponding to the third graph node in the first graph data, construct the second graph data, and use multiple graph neural networks to perform message transmission on the second graph data to obtain multiple third first graph features corresponding to each graph neural network; determine multiple second node features corresponding to the multiple second graph features and the third graph node, and determine multiple third node features corresponding to the multiple third graph features and the third graph node; and determine the weight values ​​corresponding to the multiple graph neural networks based on the similarity between the second node features and the third node features corresponding to each graph neural network.

[0074] Optionally, before determining the attribute information corresponding to the first graph node and the associated edges in the graph structure based on the satellite image data, the attribute determination module 830 is also used to perform anomaly identification on the satellite image data for the traffic network nodes and the roads between the traffic network nodes to obtain anomaly identification results; based on the anomaly identification results, determine the first traffic network node, the first traffic network node refers to the traffic network node that can be identified whether there is an anomaly through the satellite image data; and determine the first graph node corresponding to the first traffic network node in the graph structure.

[0075] Optionally, the attribute determination module 830 is used to determine, based on satellite image data, whether the traffic network node corresponding to the first graph node or the road corresponding to the associated edge is damaged; in the case that the corresponding traffic network node or the corresponding road is damaged, determine the attribute information of the first graph node corresponding to the corresponding traffic network node or the edge corresponding to the corresponding road based on a predetermined minimum value of the minimum vehicle spacing and / or the minimum vehicle speed; in the case that the corresponding traffic network node or the corresponding road is not damaged and there are vehicles on the corresponding traffic network node or the corresponding road, determine the actual minimum vehicle spacing and / or the actual minimum vehicle speed in the corresponding traffic network node and the corresponding road based on the satellite image data, and determine the attribute information of the first graph node corresponding to the corresponding traffic network node or the edge corresponding to the corresponding road based on the actual minimum vehicle spacing and / or the actual minimum vehicle speed; and in the case that the corresponding traffic network node or the corresponding road is not damaged and there are no vehicles on the corresponding traffic network node or the corresponding road, determine the attribute information of the first graph node corresponding to the corresponding traffic network node or the edge corresponding to the corresponding road based on a predetermined maximum value of the minimum vehicle spacing and / or the maximum vehicle speed.

[0076] Optionally, the attribute determination module 830 is used to determine the actual minimum speed of vehicles in the corresponding traffic network node and the corresponding road based on the satellite image data and satellite image data with a historical acquisition time interval not exceeding a preset time period from the satellite image data.

[0077] Therefore, according to this embodiment, by combining the graph structure and satellite image data, it is possible to predict whether an anomaly has occurred at a traffic network node where anomalies are unknown, by referring to traffic network nodes where anomalies are known to exist. Furthermore, on this basis, the weights corresponding to each graph neural network are determined to weight the second graph features derived from each graph neural network to obtain the first graph features. This allows the graph neural network, which performs better feature modeling through message passing, to play a greater role in determining the first graph features, thereby improving the accuracy of anomaly judgments made using the node features of the second graph nodes contained in the first graph features.

[0078] Example 3 Figure 9 FIG. 1 shows an abnormality monitoring device 900 for a road traffic network according to the first aspect of this embodiment, which corresponds to the method according to the first aspect of embodiment 1. Figure 9As shown, the device 900 includes: a processor 910; and a memory 920, connected to the processor 910, for providing the processor 910 with instructions for processing the following processing steps: determining a graph structure corresponding to a road traffic network in a predetermined area, the graph structure including graph nodes and edges connecting two graph nodes, wherein the graph nodes are used to represent traffic network nodes of the road traffic network, and the edges connecting two graph nodes are used to represent roads between the corresponding traffic network nodes; obtaining satellite image data corresponding to the predetermined area; determining attribute information corresponding to a first graph node and associated edges in the graph structure based on the satellite image data; constructing first graph data corresponding to the graph structure based on the attribute information; using a graph neural network to perform message transmission on the first graph data, determine a first graph feature corresponding to the first graph data, and determine a node feature of a second graph node of the graph structure based on the first graph feature, wherein the satellite image data does not include a traffic network node corresponding to the second graph node; and determining whether there is an abnormality in the traffic network node corresponding to the second graph node based on the node feature of the second graph node.

[0079] Optionally, the operation of using a graph neural network to perform message passing on the first graph data to determine the first graph feature corresponding to the first graph data includes: using multiple pre-set graph neural networks to perform message passing on the first graph data respectively to obtain multiple second graph features corresponding to each graph neural network respectively; determining weight values ​​corresponding to the multiple graph neural networks respectively based on the second graph features; and performing weighted summation of the multiple second graph features based on the weight values ​​to determine the first graph feature.

[0080] Optionally, the operation of determining weight values ​​corresponding to multiple graph neural networks respectively based on the second graph feature includes: extracting at least one third graph node from the first graph node, masking the attribute information corresponding to the third graph node in the first graph data, constructing the second graph data, and using multiple graph neural networks to perform message passing on the second graph data respectively to obtain multiple third first graph features corresponding to each graph neural network respectively; determining multiple second node features corresponding to the multiple second graph features and the third graph node respectively, and determining multiple third node features corresponding to the multiple third graph features and the third graph node respectively; and determining the weight values ​​corresponding to the multiple graph neural networks respectively based on the similarity between the second node features and the third node features corresponding to each graph neural network.

[0081] Optionally, before determining the attribute information corresponding to the first graph node and the associated edges in the graph structure based on the satellite image data, the memory 920 is also used to provide the processor 910 with instructions for processing the following processing steps: performing anomaly identification on the satellite image data for the traffic network nodes and the roads between the traffic network nodes to obtain anomaly identification results; determining the first traffic network node based on the anomaly identification results, the first traffic network node refers to the traffic network node that can be identified whether there is an anomaly through the satellite image data; and determining the first graph node corresponding to the first traffic network node in the graph structure.

[0082] Optionally, the operation of determining the attribute information corresponding to the first graph node and the associated edge in the graph structure based on the satellite image data includes: determining whether the traffic network node corresponding to the first graph node or the road corresponding to the associated edge is damaged based on the satellite image data; in the case that the corresponding traffic network node or the corresponding road is damaged, determining the attribute information of the first graph node corresponding to the corresponding traffic network node or the edge corresponding to the corresponding road based on the minimum value of the pre-set minimum vehicle spacing and / or minimum vehicle speed; in the case that the corresponding traffic network node or the corresponding road is not damaged and there is a vehicle on the corresponding traffic network node or the corresponding road In this case, the actual minimum vehicle spacing and / or the actual minimum vehicle speed in the corresponding traffic network node and the corresponding road are determined based on the satellite image data, and the attribute information of the first graph node corresponding to the corresponding traffic network node or the edge corresponding to the corresponding road is determined based on the actual minimum vehicle spacing and / or the actual minimum vehicle speed; and in the case that the corresponding traffic network node or the corresponding road is not damaged and there is no vehicle on the corresponding traffic network node or the corresponding road, the attribute information of the first graph node corresponding to the corresponding traffic network node or the edge corresponding to the corresponding road is determined based on the pre-set maximum value of the minimum vehicle spacing and / or the maximum vehicle speed.

[0083] Optionally, the operation of determining the actual minimum speed of vehicles in the corresponding traffic network nodes and the corresponding roads based on the satellite image data includes: determining the actual minimum speed of vehicles in the corresponding traffic network nodes and the corresponding roads based on the satellite image data and satellite image data whose collection time interval with the satellite image data in history does not exceed a preset time period.

[0084] Therefore, according to this embodiment, by combining the graph structure and satellite image data, it is possible to predict whether an anomaly has occurred at a traffic network node where anomalies are unknown, by referring to traffic network nodes where anomalies are known to exist. Furthermore, on this basis, the weights corresponding to each graph neural network are determined to weight the second graph features derived from each graph neural network to obtain the first graph features. This allows the graph neural network, which performs better feature modeling through message passing, to play a greater role in determining the first graph features, thereby improving the accuracy of anomaly judgments made using the node features of the second graph nodes contained in the first graph features.

[0085] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0086] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0088] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0089] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0090] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk.

[0091] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for monitoring abnormalities in a road traffic network, characterized in that: include: Determining a graph structure corresponding to a road traffic network in a predetermined area, the graph structure comprising graph nodes and edges connecting two graph nodes, wherein the graph nodes are used to represent traffic network nodes of the road traffic network, and the edges connecting two graph nodes are used to represent roads between the corresponding traffic network nodes; Acquiring satellite image data corresponding to the predetermined area; Determining attribute information corresponding to a first graph node and an associated edge in the graph structure based on the satellite image data; constructing first graph data corresponding to the graph structure according to the attribute information; performing message passing on the first graph data using a graph neural network to determine a first graph feature corresponding to the first graph data, and determining a node feature of a second graph node of the graph structure based on the first graph feature, wherein the satellite image data does not include a transportation network node corresponding to the second graph node; and It is determined whether there is an abnormality in the traffic network node corresponding to the second graph node according to the node feature of the second graph node.

2. The method according to claim 1, characterized in that The operation of performing message transmission on the first graph data using a graph neural network to determine a first graph feature corresponding to the first graph data includes: Utilizing a plurality of pre-set graph neural networks to perform message transmission on the first graph data respectively, to obtain a plurality of second graph features corresponding to each graph neural network respectively; Determining weight values ​​corresponding to a plurality of graph neural networks, respectively, based on the second graph feature; and The first graph feature is determined by performing weighted summation on the plurality of second graph features according to the weight value.

3. The method according to claim 2, characterized in that The operation of determining weight values ​​corresponding to the plurality of graph neural networks, respectively, according to the second graph feature, includes: Extracting at least one third graph node from the first graph node, and masking the attribute information corresponding to the third graph node in the first graph data to construct second graph data; Using the multiple graph neural networks to perform message passing on the second graph data respectively, to obtain multiple third graph features corresponding to the respective graph neural networks; respectively determining a plurality of second node features corresponding to the plurality of second graph features and the third graph nodes, and respectively determining a plurality of third node features corresponding to the plurality of third graph features and the third graph nodes; and Based on the similarity between the second node features and the third node features corresponding to each graph neural network, the weight values ​​corresponding to the multiple graph neural networks are determined.

4. The method according to claim 1, wherein Before determining attribute information corresponding to the first graph node and the associated edge in the graph structure based on the satellite image data, the method further includes: Performing anomaly identification on traffic network nodes and roads between traffic network nodes on satellite image data to obtain anomaly identification results; Determining a first transportation network node based on the anomaly identification result, where the first transportation network node refers to a transportation network node that can be identified as having an anomaly based on the satellite image data; A first graph node in the graph structure corresponding to the first transportation network node is determined.

5. The method according to claim 1, characterized in that The operation of determining, based on the satellite image data, attribute information corresponding to a first graph node and an associated edge in the graph structure includes: determining, based on the satellite image data, whether a transportation network node corresponding to the first graph node or a road corresponding to the associated edge is damaged; In the event that the corresponding traffic network node or the corresponding road is damaged, determining attribute information of a first graph node corresponding to the corresponding traffic network node or an edge corresponding to the corresponding road according to a predetermined minimum value of a minimum vehicle spacing and / or a minimum vehicle speed; In a case where the corresponding transportation network node or the corresponding road is not damaged and there are vehicles on the corresponding transportation network node or the corresponding road, determining, based on the satellite image data, an actual minimum vehicle spacing and / or an actual minimum vehicle speed in the corresponding transportation network node and the corresponding road, and determining, based on the actual minimum vehicle spacing and / or the actual minimum vehicle speed, attribute information of a first graph node corresponding to the corresponding transportation network node or an edge corresponding to the corresponding road; and When the corresponding traffic network node or the corresponding road is not damaged and there is no vehicle on the corresponding traffic network node or the corresponding road, attribute information of the first graph node corresponding to the corresponding traffic network node or the edge corresponding to the corresponding road is determined based on the maximum value of the pre-set minimum vehicle spacing and / or minimum vehicle speed.

6. The method according to claim 5, characterized in that The operation of determining the actual minimum speed of vehicles in the corresponding traffic network node and the corresponding road based on the satellite image data includes: The actual minimum speed of vehicles in the corresponding traffic network node and the corresponding road is determined based on the satellite image data and the historical satellite image data whose collection time interval with the satellite image data does not exceed a preset time period.

7. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 6.

8. A device for monitoring abnormalities in a road traffic network, characterized in that: include: a graph structure determination module, configured to determine a graph structure corresponding to a road traffic network in a predetermined area, wherein the graph structure includes graph nodes and edges connecting two graph nodes, wherein the graph nodes are used to represent traffic network nodes of the road traffic network, and the edges connecting two graph nodes are used to represent roads between the corresponding traffic network nodes; An image acquisition module, configured to acquire satellite image data corresponding to the predetermined area; an attribute determination module, configured to determine attribute information corresponding to a first graph node and an associated edge in the graph structure based on the satellite image data; A graph data construction module, configured to construct first graph data corresponding to the graph structure according to the attribute information; a feature determination module, configured to perform message transmission on the first graph data using a graph neural network to determine a first graph feature corresponding to the first graph data, and determine a node feature of a second graph node of the graph structure based on the first graph feature, wherein the satellite image data does not include a transportation network node corresponding to the second graph node; and An anomaly determination module is used to determine whether there is an anomaly in the traffic network node corresponding to the second graph node based on the node characteristics of the second graph node.

9. A device for monitoring abnormalities in a road traffic network, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Determining a graph structure corresponding to a road traffic network in a predetermined area, the graph structure comprising graph nodes and edges connecting two graph nodes, wherein the graph nodes are used to represent traffic network nodes of the road traffic network, and the edges connecting two graph nodes are used to represent roads between the corresponding traffic network nodes; Acquiring satellite image data corresponding to the predetermined area; Determining attribute information corresponding to a first graph node and an associated edge in the graph structure based on the satellite image data; constructing first graph data corresponding to the graph structure according to the attribute information; performing message passing on the first graph data using a graph neural network to determine a first graph feature corresponding to the first graph data, and determining a node feature of a second graph node of the graph structure based on the first graph feature, wherein the satellite image data does not include a transportation network node corresponding to the second graph node; and It is determined whether there is an abnormality in the traffic network node corresponding to the second graph node according to the node feature of the second graph node.

Images