Multi-source data linkage query method and visualization system

Abstract

The invention discloses a multi-source data linkage query method and a visualization system, and the query method comprises the steps: 1, receiving a monitoring video stream and data obtained by an Internet of Things sensor, and recording the positions and action ranges of a monitoring camera and the Internet of Things sensor corresponding to the data, and the timestamps of the data; 2, establishing an association relationship between the trigger event and the equipment, wherein the association relationship is a corresponding relationship among the trigger event, time, position, a monitoring camera and an Internet of Things sensor; 3, according to a query condition input by a user, querying and obtaining a query result in the association relationship, the query result comprising the monitoring video stream corresponding to the query condition and the data collected by the Internet of Things sensor; and 4, carrying out serialization display on the query result on a user interface. According to the method, data obtained by a plurality of data sources can be inquired in a linkage manner during inquiry, and the data is displayed in a serialized manner.

Description

TECHNICAL FIELD

[0001] The present application relates to a multi-source data linkage query method and a visualization system, belonging to the technical field of intelligent query. BACKGROUND

[0002] With the development of technology, video monitoring and deployment of Internet of Things equipment are increasing, and the data obtained in combination with algorithms can realize multi-angle monitoring. However, video monitoring systems, Internet of Things management systems and algorithm event processing systems often run independently, lacking an effective data integration mechanism. This independent running mode leads to information islands, and when an algorithm event is detected and cross-system data query and analysis are needed, the user needs to operate multiple systems, which is not only inefficient but also difficult to realize data linkage query and comprehensive analysis. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a multi-source data linkage query method, which can query data obtained from multiple data sources in linkage and display the data in sequence by establishing an association between trigger events and devices.

[0004] In order to solve the above technical problems, a multi-source data linkage query method is disclosed, comprising the steps of:

[0005] S1, receiving data obtained by monitoring video stream and Internet of Things sensors, recording the positions, action ranges of monitoring cameras and Internet of Things sensors corresponding to the data, and time stamps of the data;

[0006] S2, establishing an association between trigger events and devices, the association being a corresponding relationship between trigger events, time, position, monitoring cameras and Internet of Things sensors; the time and position in the association being the time and position of the occurrence of the trigger event, and the devices being monitoring cameras and Internet of Things sensors whose action ranges include the position of the trigger event;

[0007] S3, according to the query condition input by the user, querying and obtaining the query result in the association, the query result including data collected by monitoring video stream and Internet of Things sensors corresponding to the query condition;

[0008] S4, sequentially displaying the query result on the user interface.

[0009] Further, the step S1 further comprises: recording the positions and action ranges of the monitoring cameras and the Internet of Things sensors in the device registry;

[0010] The device registry is used to record the positions, action ranges, installation time and exit time of the monitoring cameras and the Internet of Things sensors;

[0011] In the step S2, the device registry is searched to obtain a first device set; the first device set comprises the monitoring camera and the Internet of Things sensor in the action range including the trigger event position;

[0012] The association between the trigger event and the monitoring camera and the Internet of Things sensor in the first device set is established.

[0013] Further, when there is displacement, deletion or addition of the monitoring camera or the Internet of Things sensor, the device registry is updated in real time.

[0014] Further, the step S2 further comprises: storing the association in a database and recording the establishment time, the modification history and the effective state.

[0015] Further, the query condition comprises: the trigger event type, the trigger event time range and the geographic space information.

[0016] The geographic space information comprises the point, line and surface positions or the road and community positions in the geographic space.

[0017] Further, when the query condition is a plurality of road queries and the plurality of roads are connected in sequence, the step S4 specifically comprises:

[0018] All the end points of the query roads and the connecting roads between the end points are traversed, the end point with only one connecting road is selected as a starting point, all the query roads are traversed along the roads in the connected order from the starting point to the end point, and the collection data of all the monitoring cameras and the Internet of Things sensors are displayed according to the path from the starting point to the end point.

[0019] Further, when the query condition is a road query, the road is a circular road or an arc-shaped curve, and the step S4 specifically comprises:

[0020] The latitude and longitude coordinates of at least three known points on the circular road or the arc-shaped curve are selected, and the UTM projection is used to convert the latitude and longitude coordinates into plane coordinates , The formula (1) is used to calculate the center coordinates of the circular road or the arc-shaped curve :

[0021] (1)

[0022] Wherein, R is the radius of the circular road or the arc-shaped curve;

[0023] For each device point position, the latitude and longitude coordinates are converted into plane coordinates by the UTM projection, and the intersection point of the connecting line of the plane coordinates of each device point position and the circle is calculated ​As the projection point of the equipment location on the circumference; calculate its relationship with the center of the circle. polar angle ,according to Sort the equipment locations in descending order;

[0024] (2)

[0025] The system displays the data collected by all the surveillance cameras and IoT sensors found in the query, sorted by device location.

[0026] On the other hand, the present invention also discloses a multi-source data linkage query visualization system, comprising:

[0027] The multi-source data receiving module (1) is used to receive monitoring video streams and data acquired by IoT sensors, and to record the location, range of action and timestamp of the monitoring camera and IoT sensor corresponding to the data.

[0028] The rule engine module (2) is used to establish the association between triggering events and devices. The association is the correspondence between triggering events, time, location, surveillance cameras, and IoT sensors. In the association, time and location refer to the time and location of the triggering event, and devices refer to surveillance cameras and IoT sensors whose scope includes the location of the triggering event.

[0029] The query module (3) is used to query and obtain query results in the association relationship according to the user's query conditions. The query results include the monitoring video stream and the data collected by the Internet of Things sensor corresponding to the query conditions.

[0030] The display module (4) is used to display the query results.

[0031] Furthermore, it also includes a device registry management module (5) for managing the device registry; the device registry is used to record the location and operating range of the surveillance camera and the IoT sensor;

[0032] After the multi-source data receiving module (1) receives the data, the device registry management module (5) records the location and range of the monitoring camera and IoT sensor in the device registry.

[0033] The rule engine module (2) first retrieves the device registry to obtain the first device set; the first device set is the surveillance camera and IoT sensor whose scope includes the location of the triggering event.

[0034] Then establish the association between the triggering event and the surveillance cameras and IoT sensors in the first set of devices.

[0035] Furthermore, the rules engine module (2) also stores the relationships in the database and records the creation time, modification history and effective status.

[0036] Beneficial effects: The multi-source data linkage query method and visualization system disclosed in this invention establishes the association between triggering events and devices, enabling the linkage query of data obtained from multiple data sources during the query process, and serializing and displaying the data; in addition, in order to meet the requirements for sorting device locations around circular roads or curved curves, a query sorting algorithm based on coordinates and polar angles is adopted to improve the accuracy and computational efficiency of spatial queries. Attached Figure Description

[0037] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0038] Figure 1 This is a flowchart of the multi-source data linkage query method disclosed in this invention;

[0039] Figure 2 A schematic diagram showing the sorting of equipment locations for online queries;

[0040] Figure 3 A schematic diagram showing the arrangement of equipment locations when multiple roads are connected end to end;

[0041] Figure 4 This is a schematic diagram showing the arrangement of equipment locations when multiple roads are not connected end to end.

[0042] Figure 5 A schematic diagram showing the arrangement of equipment locations when multiple roads share a common starting point;

[0043] Figure 6 A schematic diagram showing the arrangement of equipment locations on a circular road or curved road.

[0044] Figure 7 This is a schematic diagram of the composition of the multi-source data linkage query visualization system disclosed in this invention. Detailed Implementation

[0045] This invention discloses a multi-source data linkage query method, such as... Figure 1 As shown, the steps include:

[0046] S1. Receive surveillance video streams and data acquired by IoT sensors, and record the location, effective range, and timestamp of the surveillance camera and IoT sensor corresponding to the data.

[0047] To record data from diverse sources, this embodiment establishes a cross-platform interface to access data collected from different video and IoT devices. The data is cleaned and normalized into a standard format, a data management system is established, and the data is stored in a database to ensure data quality and searchability. Different devices have different effective ranges. For example, a camera's data collection area can range from a few meters to tens of meters in diameter, or it can be a conical section with the camera's position as the apex, depending on the camera's monitoring range and installation angle. Temperature sensors typically have a point-based effective range, while smoke sensors have a limited spatial range. Each device's data is recorded along with its effective range and a timestamp.

[0048] To facilitate data and device management, this embodiment records the location, status, and operating range of surveillance cameras and IoT sensors in a device registry. The device registry records the location, operating range, installation time, and deactivation time of the surveillance cameras and IoT sensors; thus, a mapping relationship is established between the devices and the data. When a surveillance camera or IoT sensor experiences displacement, deletion, or addition, the device registry is updated in real time.

[0049] S2. Establish the association between the triggering event and the device. The association is the correspondence between the triggering event, time, location, surveillance camera, and IoT sensor. In the association, time and location refer to the time and location of the triggering event, and the device refers to the surveillance camera and IoT sensor whose scope includes the location of the triggering event.

[0050] If a device registry has already been established, first search the device registry to obtain the first set of devices; the first set of devices consists of surveillance cameras and IoT sensors whose scope includes the location of the triggering event.

[0051] Then, the association between the triggering event and the surveillance cameras and IoT sensors in the first set of devices is established, thus establishing the association between the triggering event and the devices.

[0052] For ease of management, this embodiment stores the association relationships in a database, recording the creation time, modification history, and effective status. The database allows for viewing, modifying, or deleting these relationships, meeting flexible configuration requirements. Furthermore, when devices within the association relationships are updated, the relationships can also be updated to ensure they are up-to-date and adapt to dynamic changes.

[0053] S3. Based on the query conditions input by the user, query and obtain query results in the association relationship. The query results include the monitoring video stream and data collected by IoT sensors corresponding to the query conditions.

[0054] The query criteria include: trigger event type, trigger event time range, and geospatial information;

[0055] The geospatial information includes the locations of points, lines, and areas in geospatial space, or the locations of roads and communities.

[0056] For example, if a user can query video data of the vicinity before and after a traffic accident to view traffic conditions, the query conditions would be: Triggering event type: traffic accident, Triggering event time range: a certain range before and after the time of the traffic accident, and Geospatial information: the road segment of interest near the location of the traffic accident.

[0057] If the geospatial information in the query conditions is a point location, then a point query is performed. The device located at a point is queried based on the coordinates of that point. The method is to generate a circular surface with the point as the center and a specified radius (such as 0.5 meters), and then query the device whose coordinates are within that surface.

[0058] If the geospatial information in the query criteria is a line, such as a line other than a road specified by the user, then a line query will be performed. A line query refers to querying devices located near a straight line or polyline specified by the user. The implementation method is similar to that of a point query, that is, a surface is generated along both sides of the line according to a specified range (such as 5 meters), and the devices with coordinates within the surface are queried.

[0059] Road lookup works on a similar principle to line lookup, except that the road surface already exists, so there is no need to generate surface data based on the line first.

[0060] A surface query is a query that searches for devices whose coordinates are located within a surface range specified by the user. A road query is a surface query that uses road surface data.

[0061] The system retrieves the locations of cameras or IoT sensors by querying and returns data for each location within the time range specified in the query criteria.

[0062] S4. Serialize and display the query results on the user interface.

[0063] For line queries and road queries, the device locations need to be sorted and then displayed serially.

[0064] The results of a line query can be sorted by the order in which the line segments that make up the polyline are drawn, that is, sorted from the start point to the end point of the line, and the data is displayed according to the sorted equipment points. For example... Figure 2 As shown, there are five devices A, B, C, D, and E on both sides of line P1P2, and their effective ranges all include line P1P2. Their sorting method is as follows: calculate the perpendicular from the installation position of each device to the line. Figure 2The feet of the perpendiculars from the installation positions of devices A, B, and C to the line are all point P1. In three-dimensional space, the distances from the installation positions of devices A, B, and C to the foot of the perpendicular P1 are in ascending order as AP1 < BP1 < CP1; the feet of the perpendiculars from the installation positions of devices D and E to the line are all point P2. In three-dimensional space, the distances from the installation positions of devices D and E to the foot of the perpendicular P2 are in ascending order as DP2 < EP2; according to the rule of the line direction P1→P2 and the distances from the devices to the foot of the perpendicular being short first and then long, the sorting result is A→B→C→D→E, and the data collected by the devices are displayed in this order.

[0065] For a single road, sort the device points according to the road direction.

[0066] For multiple roads that are connected end to end in sequence, such as Figure 3 shown, the sorting method is as follows: Traverse the endpoints of all query roads and the connecting roads between the endpoints, select the endpoints with only one connecting road as the starting point, and start from the starting point to traverse all query roads in the connected order along the road until the end point; display the data collected by all monitored cameras and IoT sensors queried according to the path from the starting point to the end point. As Figure 2 shown, after traversing, it is found that both point A and point E are endpoints with only one connecting road, and the remaining points B, C, and D all have two connecting roads.任选点A或点E作为起点。本实施例选择纬度较小的E点作为起点,按照E→D→C→B→A的顺序进行序列化显示数据。

[0067] If there are multiple roads and the multiple roads are not connected end to end, the sorting steps are as follows: First, find the point with the smallest longitude among all road endpoints as the starting point. If there are multiple endpoints with the same longitude, take the point with the smallest latitude among these points with the same longitude as the starting point; then calculate the endpoint of the road that is closest to the current road among the remaining roads that have not been sorted as the starting point of the next road, and so on. As Figure 4 shown, the sorting result is that the west side endpoint of Road 1 is used as the starting point, and in the order of Road 1→Road 2→Road 3, the east side endpoint of Road 3 is used as the end point. As Figure 5 shown, multiple roads have a common starting point. The sorting result is that point A is used as the starting point, and the end points of the multiple roads are B, C, and D respectively. And the latitude of point B is the smallest, so Road AB comes first; AC and AD are not sorted, and Road AC is closer to the sorted Road AB, so the sorting is in the order of AB→AC→AD.

[0068] When the query condition is a road query and the road is a circular road (such as a roundabout) or an arc-shaped curve, the sorting method is as follows:

[0069] Select the longitude and latitude coordinates of at least three known points on the circular road or arc-shaped curve, and convert them into plane coordinates through UTM projection , Then, the coordinates of the center of the circle are calculated using formula (1). :

[0070] (1)

[0071] in The radius of a circular road or an arc-shaped curve;

[0072] For each device location found, its latitude and longitude coordinates are converted into planar coordinates using UTM projection. Calculate the intersection point of the line connecting the plane coordinates of each equipment point to the center of the circle and the circle. As the projection point of the equipment location on the circumference; calculate its relationship with the center of the circle. polar angle ,according to Sort the equipment locations in descending order; for example Figure 6 As shown.

[0073] (2)

[0074] If multiple device locations have the same coordinates when projected onto the circumference, then calculate the distance from the location to the projected point. The order of these devices is determined from smallest to largest, as shown in formula (3).

[0075] (3)

[0076] Therefore, cameras or IoT devices around circular roads or curved bends will be arranged in a counter-clockwise or clockwise order.

[0077] This invention also discloses a multi-source data linkage query visualization system, such as... Figure 7 As shown, it includes:

[0078] The multi-source data receiving module 1 is used to receive the monitoring video stream and the data acquired by the IoT sensor according to step S1, and record the location, status, range of action of the monitoring camera and IoT sensor corresponding to the data, as well as the timestamp of the data.

[0079] Rule engine module 2 is used to establish the association between triggering events and devices according to step S2. The association is the correspondence between triggering events, time, location, surveillance cameras, and IoT sensors. In the association, time and location refer to the time and location of the triggering event, and devices refer to surveillance cameras and IoT sensors whose scope includes the location of the triggering event.

[0080] Query module 3 is used to query and obtain query results in the association relationship according to the user's query conditions in step S3. The query results include the monitoring video stream and the data collected by the Internet of Things sensor corresponding to the query conditions.

[0081] Display module 4 is used to serialize and display the query results according to step S4.

[0082] In this embodiment, the display module 4 renders the locations of devices and events based on the Mapbox map framework. Users can view the locations of all video surveillance points and IoT devices within a certain area, as well as the distribution of events. When a user clicks on a device or event marker on the map, a prompt window displays detailed information related to that device or event, including real-time video streams, historical snapshots, and data curves of IoT devices.

[0083] The display module can automatically adjust the number of rows and columns in the grid according to the number of video query results, and switch between 4, 9 and 16 grids for playback. It can also use bar charts, pie charts and other formats to display statistical indicators such as device online rate and failure rate; display the spatiotemporal distribution characteristics of events in the form of heat maps; and support chart linkage, clicking on chart elements can link and update map markers, video streams and IoT information.

[0084] The visualization system also includes a device registry management module 5, which manages the device registry; the device registry is used to record the location, status and range of operation of the surveillance camera and the IoT sensor.

[0085] After the multi-source data receiving module 1 receives the data, the device registry management module 5 records the location, status and range of the monitoring camera and IoT sensor in the device registry.

[0086] Rule engine module 2 first searches the device registry to obtain a first set of devices; the first set of devices consists of surveillance cameras and IoT sensors whose scope includes the location of the triggering event.

[0087] Then establish the association between the triggering event and the surveillance cameras and IoT sensors in the first set of devices.

[0088] The rules engine module 2 also stores the relationships in the database and records the creation time, modification history, and effective status.

[0089] This invention provides a concept and method for multi-source data linkage query. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for multi-source data linkage query, characterized in that, Including the following steps: S1. Receive surveillance video streams and data acquired by IoT sensors, and record the location, effective range, and timestamp of the surveillance camera and IoT sensor corresponding to the data. S2. Establish the association between triggering events and devices, wherein the association is the correspondence between triggering events, time, location, surveillance cameras, and IoT sensors; In the aforementioned association, time and location refer to the time and location of the triggering event, and the device refers to a surveillance camera and an IoT sensor whose effective range includes the location of the triggering event. S3. Based on the query conditions input by the user, query and obtain query results in the association relationship. The query results include the monitoring video stream and data collected by IoT sensors corresponding to the query conditions. S4. Serialize and display the query results on the user interface.

2. The multi-source data linkage query method according to claim 1, characterized in that, Step S1 also includes: recording the location and effective range of the surveillance camera and IoT sensor in the device registry; The device registry is used to record the location, range of action, installation time, and removal time of surveillance cameras and IoT sensors; In step S2, the device registry is first searched to obtain a first set of devices; the first set of devices consists of surveillance cameras and IoT sensors whose scope includes the location of the triggering event. Then establish the association between the triggering event and the surveillance cameras and IoT sensors in the first set of devices.

3. The multi-source data linkage query method according to claim 2, characterized in that, When there are displacement, deletion, or addition events from surveillance cameras or IoT sensors, the device registry is updated in real time.

4. The multi-source data linkage query method according to claim 1, characterized in that, Step S2 further includes: storing the association in the database and recording the creation time, modification history and effective status.

5. The multi-source data linkage query method according to claim 1, characterized in that, The query criteria include: trigger event type, trigger event time range, and geospatial information; The geospatial information includes the locations of points, lines, and areas in geospatial space, or the locations of roads and communities.

6. The multi-source data linkage query method according to claim 5, characterized in that: When the query condition is a query for multiple roads, and the multiple roads are connected sequentially, step S4 specifically includes: Iterate through all endpoints of the queried roads and the connecting roads between them, select the endpoint with only one connecting road as the starting point, and traverse all queried roads in connected order from the starting point to the endpoint; display the data collected by all surveillance cameras and IoT sensors according to the path from the starting point to the endpoint.

7. The multi-source data linkage query method according to claim 5, characterized in that: When the query condition is a road query, and the road is a circular road or an arc-shaped curve, step S4 specifically includes: Select the latitude and longitude coordinates of at least three known points on the circular road or curved road, and convert them into planar coordinates using UTM projection. , Then, the coordinates of the center of the circle are calculated using formula (1). : （1） in The radius of a circular road or an arc-shaped curve; For each device location found, its latitude and longitude coordinates are converted into planar coordinates using UTM projection. Calculate the intersection point of the line connecting the plane coordinates of each equipment point to the center of the circle and the circle. As the projection point of the equipment location on the circumference; calculate its relationship with the center of the circle. polar angle ,according to Sort the equipment locations in descending order; （2） The system displays the data collected by all the surveillance cameras and IoT sensors found in the query, sorted by device location.

8. A multi-source data linkage query visualization system, characterized in that, include: The multi-source data receiving module (1) is used to receive monitoring video streams and data acquired by IoT sensors, and to record the location, range of action and timestamp of the monitoring camera and IoT sensor corresponding to the data. The rule engine module (2) is used to establish the association between triggering events and devices, wherein the association is the correspondence between triggering events, time, location, surveillance cameras, and IoT sensors; In the aforementioned association, time and location refer to the time and location of the triggering event, and the device refers to a surveillance camera and an IoT sensor whose effective range includes the location of the triggering event. The query module (3) is used to query and obtain query results in the association relationship according to the user's query conditions. The query results include the monitoring video stream and the data collected by the Internet of Things sensor corresponding to the query conditions. The display module (4) is used to serialize and display the query results.

9. The multi-source data linkage query visualization system according to claim 8, characterized in that, It also includes a device registry management module (5) for managing the device registry; the device registry is used to record the location and range of the surveillance camera and the IoT sensor. After the multi-source data receiving module (1) receives the data, the device registry management module (5) records the location and range of the monitoring camera and IoT sensor in the device registry. The rule engine module (2) first retrieves the device registry to obtain the first device set; the first device set is the surveillance camera and IoT sensor whose scope includes the location of the triggering event. Then establish the association between the triggering event and the surveillance cameras and IoT sensors in the first set of devices.

10. The multi-source data linkage query visualization system according to claim 8, characterized in that, The rules engine module (2) also stores the relationships in the database and records the creation time, modification history and effective status.

Images