Visual decision-making system and method

Abstract

The invention is suitable for the technical field of intelligent decision making, and provides a visual decision making system and method. The system is characterized in that a space-time big data center acquires first data, a situation awareness engine determines public security factors in a city area according to the first data, and a city total factor visualization engine performs three-dimensional visualization display on the public security factors in the city area; and the command and dispatch module receives the alarm information or the traffic accident information and obtains the real-time position of the policeman, determines a target policeman according to the alarm information or the traffic accident information and the real-time position of the policeman, and renders and displays the alarm information or the traffic accident information, the target policeman and the optimal path. According to the invention, information such as personnel resource distribution, personnel conditions, vehicle conditions, alarm information and commanding and dispatching in an urban area can be integrated in a three-dimensional scene for display, and fusion of a'condition-to-line 'service and the three-dimensional scene is realized.

Description

Technical Field

[0001] The present application belongs to the field of intelligent decision-making technology, and in particular, relates to a visual decision-making system and method. Background Art

[0002] In recent years, with the rapid development of computer software, hardware and communication technology, spatiotemporal information technology has gained a broader development and application space, becoming a new growth point for the development of information technology. Under such a background, the traditional geographic information system (PGIS) lacks complete dimensional information, does not have three-dimensional visualization expression and spatiotemporal analysis capabilities, and cannot adapt to the transformation and upgrading requirements of the police operation model under the new situation. Summary of the invention

[0003] To overcome the problems existing in the related art, the embodiments of the present application provide a visual decision-making system and method.

[0004] This application is implemented through the following technical solutions:

[0005] In a first aspect, an embodiment of the present application provides a visual decision-making system, including: a spatiotemporal big data center, a situation awareness engine, a city full-factor visualization engine, and a command and dispatch module;

[0006] The spatiotemporal big data center is used to obtain first data, where the first data includes geographic location, video data, and sensor device data;

[0007] The situation awareness engine is used to determine the public security factors in the urban area according to the first data, wherein the public security factors include the distribution of personnel resources, the situation of target persons and the situation of target vehicles;

[0008] The city full-factor visualization engine is used to perform three-dimensional visualization of public security factors within the city area;

[0009] The command and dispatch module is used to receive alarm information or traffic accident information and obtain the real-time location of the police officer, determine the target police officer based on the alarm information or traffic accident information and the real-time location of the police officer, and render and display the alarm information or traffic accident information, the target police officer and the optimal path.

[0010] In conjunction with the first aspect, in some embodiments, the spatiotemporal big data center is specifically used for:

[0011] Use the Flink CDC Connectors component to read full or incremental data from MySQL, PostgreSQL, Oracle, and MongoDB databases to obtain the first data.

[0012] The Flink real-time computing engine is used to deduplicate, format, add three-field identifiers, and perform coordinate conversion on the first data.

[0013] In combination with the first aspect, in some embodiments, the coordinate transformation is specifically:

[0014] Obtaining first coordinates of multiple target points in the video data, where the first coordinates are coordinate values ​​of the multiple target points in the camera coordinate system;

[0015] Determine the rotation matrix R and translation matrix T corresponding to the first coordinate converted to the second coordinate. The second coordinate is the coordinate value of the multiple target points in the earth coordinate system. The centroid surrounded by the first coordinates of the multiple target points is the Centroid. A The centroid formed by the second coordinates of the multiple target points is the Centroid B , rotation matrix R = VU T , translation matrix T = -R*Centroid A +Centroid B ;

[0016] The coordinates in the video data are converted into coordinates in the earth coordinate system according to the rotation matrix R and the translation matrix T.

[0017] In conjunction with the first aspect, in some embodiments, the situation awareness engine is specifically used to:

[0018] Detecting a target person or a target vehicle in the video data of the first data to obtain a Bounding Box of the target person or the target vehicle;

[0019] Determine the edge contour points of the target person or the target vehicle in the Bounding Box, and determine the average of the leftmost, rightmost, topmost and bottommost four coordinate values ​​of the edge contour points as the coordinates of the target person or the target vehicle;

[0020] The trajectory of the target person or target vehicle is predicted based on the historical coordinates of the target person or target vehicle.

[0021] In conjunction with the first aspect, in some embodiments, the situation awareness engine is further used to:

[0022] Detecting images of vehicles that meet preset conditions from video data; the preset conditions are that the number of times a vehicle appears at a preset geographical location at a preset time is greater than a preset number, or the driving trajectory meets a preset rule;

[0023] Extracting a vehicle qualification certificate image from the image;

[0024] Extracting image blocks from a plurality of preset areas in the vehicle qualification certificate image, respectively, each image block corresponding to an area where a piece of information in the vehicle qualification certificate is located;

[0025] Setting a position mark for each image block, wherein the position mark represents the position of each image block in the document image;

[0026] Perform OCR recognition on each image block to obtain a piece of information in the vehicle qualification certificate;

[0027] Generate vehicle qualification certificate information based on the position identifier of each image block and the corresponding extracted information;

[0028] The generated vehicle qualification certificate information is matched with the license plate to check the vehicle's qualifications.

[0029] In conjunction with the first aspect, in some embodiments, the city full-factor visualization engine is specifically used for:

[0030] Store vector data including water systems, settlements and facilities, transportation and place names into a PostGIS database cluster;

[0031] Use the oblique model slicing tool to process the urban oblique model data into 3D Tiles format and store it in the MinIO object storage cluster;

[0032] Receive tile requests sent by the client through the visualization engine according to the service address of vector tiles, and receive slice requests sent by the client through the visualization engine according to the service address of three-dimensional model slices;

[0033] The server obtains the service name, level, row and column number of the vector tile according to the tile request, and calculates the coordinate range of the corresponding tile according to the tile request;

[0034] The server determines whether there is corresponding tile data in the cache according to the service name and the coordinate range; if there is no corresponding tile data, the coordinate range is used as a spatial constraint to perform a spatial query, obtain the geometry and attribute information of the corresponding tile, and use the ST_AsMvtGeom and ST_AsMVT functions of PostGIS to convert the vector data corresponding to the tile into the mvt format of MapBox; if there is corresponding tile data, the tile data in the cache is returned to the client;

[0035] Use Geo3DtilesLayer to render the 3D model slices to obtain the 3D model layer;

[0036] Parse the vector tile data, obtain the spatial graphic data contained in the vector tile data, and convert the parsed spatial graphic data into spatial graphic data in Json format;

[0037] Use ColorMask to create a mask layer with spatial graphic data in Json format and bind mouse events;

[0038] The mask layer is bound to the 3D model layer, the 3D model bounding box is calculated using the spatial graphic data of the mask layer, and the 3D model layer is dynamically segmented based on the 3D model bounding box to achieve dynamic individualized rendering of the oblique photography layer.

[0039] In conjunction with the first aspect, in some embodiments, the command and dispatch module is specifically used to:

[0040] Extract the time, location and content of the incident from the alarm information or traffic accident information;

[0041] Get the real-time location of each police officer;

[0042] Determine the target police officer who meets the preset conditions according to the occurrence location and the real-time location of each police officer;

[0043] According to the occurrence location and the real-time location of the target police officer, an optimal path from the real-time location of the target police officer to the occurrence location is generated;

[0044] The alarm information or traffic accident information, target police officer and optimal path are rendered and displayed.

[0045] In a second aspect, an embodiment of the present application provides a visual decision-making method, including:

[0046] Acquire first data, where the first data includes geographic location, video data, and sensor device data;

[0047] Determine the public security factors in the urban area according to the first data, wherein the public security factors include the distribution of personnel resources, the situation of target persons and the situation of target vehicles;

[0048] Performing a three-dimensional visual display of public security elements within the urban area;

[0049] Receive alarm information or traffic accident information and obtain the real-time location of the police officer, determine the target police officer according to the alarm information or traffic accident information and the real-time location of the police officer, and render and display the alarm information or traffic accident information, the target police officer and the optimal path.

[0050] In conjunction with the second aspect, in some embodiments, the three-dimensionally visualizing the public security elements in the urban area includes:

[0051] Store vector data including water systems, settlements and facilities, transportation and place names into a PostGIS database cluster;

[0052] Use the oblique model slicing tool to process the urban oblique model data into 3D Tiles format and store it in the MinIO object storage cluster;

[0053] Receive tile requests sent by the client through the visualization engine according to the service address of vector tiles, and receive slice requests sent by the client through the visualization engine according to the service address of three-dimensional model slices;

[0054] The server obtains the service name, level, row and column number of the vector tile according to the tile request, and calculates the coordinate range of the corresponding tile according to the tile request;

[0055] The server determines whether there is corresponding tile data in the cache according to the service name and the coordinate range; if there is no corresponding tile data, the coordinate range is used as a spatial constraint to perform a spatial query, obtain the geometry and attribute information of the corresponding tile, and use the ST_AsMvtGeom and ST_AsMVT functions of PostGIS to convert the vector data corresponding to the tile into the mvt format of MapBox; if there is corresponding tile data, the tile data in the cache is returned to the client;

[0056] Use Geo3DtilesLayer to render the 3D model slices to obtain the 3D model layer;

[0057] Parse the vector tile data, obtain the spatial graphic data contained in the vector tile data, and convert the parsed spatial graphic data into spatial graphic data in Json format;

[0058] Use ColorMask to create a mask layer with spatial graphic data in Json format and bind mouse events;

[0059] The mask layer is bound to the 3D model layer, the 3D model bounding box is calculated using the spatial graphic data of the mask layer, and the 3D model layer is dynamically segmented based on the 3D model bounding box to achieve dynamic individualized rendering of the oblique photography layer.

[0060] In conjunction with the second aspect, in some embodiments, determining the situation of a target person or a target vehicle in a city area according to the first data is specifically as follows:

[0061] Detecting a target person or a target vehicle in the video data of the first data to obtain a Bounding Box of the target person or the target vehicle;

[0062] Determine the edge contour points of the target person or the target vehicle in the Bounding Box, and determine the average of the leftmost, rightmost, topmost and bottommost four coordinate values ​​of the edge contour points as the coordinates of the target person or the target vehicle;

[0063] The trajectory of the target person or target vehicle is predicted based on the historical coordinates of the target person or target vehicle.

[0064] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0065] In the embodiment of the present application, the spatiotemporal big data center obtains first data including geographic location, video data and sensing device data, and the situation awareness engine determines the distribution of personnel resources, the situation of target persons and the situation of target vehicles in the urban area based on the first data, and the city full-factor visualization engine performs three-dimensional visualization of the distribution of personnel resources, the situation of target persons and the situation of target vehicles in the urban area. Finally, the command and dispatch module is used to receive alarm information or traffic accident information and obtain the real-time location of the police officer, determine the target police officer based on the alarm information or traffic accident information and the real-time location of the police officer, and render and display the alarm information or traffic accident information, the target police officer and the optimal path, so that the distribution of personnel resources, personnel situation, vehicle situation, alarm information and command and dispatch information in the urban area can be integrated in the three-dimensional scene for display, realizing the integration of the "information, command and action" business and the three-dimensional scene.

[0066] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0068] Figure 1 It is a structural diagram of a visual decision-making system provided by an embodiment of the present application;

[0069] Figure 2 It is a flowchart of a visual decision-making method provided in one embodiment of the present application. DETAILED DESCRIPTION

[0070] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0071] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0072] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0073] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0074] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0075] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0076] Reference Figure 1The visualization decision-making system provided in the embodiment of the present application includes a spatiotemporal big data center 101, a situation awareness engine 102, a city full-factor visualization engine 103 and a command and dispatch module 104.

[0077] The spatiotemporal big data center 101 is used to obtain first data, which includes geographic location, video data and sensory device data.

[0078] The situation awareness engine 102 is used to determine the public security factors in the urban area based on the first data, and the public security factors include the distribution of personnel resources, the situation of target persons and the situation of target vehicles.

[0079] The city full-factor visualization engine 103 is used to visualize the public security factors in the city area in three dimensions.

[0080] The command and dispatch module 104 is used to receive alarm information or traffic accident information and obtain the real-time location of the police officer, determine the target police officer based on the alarm information or traffic accident information and the real-time location of the police officer, and render and display the alarm information or traffic accident information, the target police officer and the optimal path.

[0081] In the above-mentioned visualization decision system, the spatiotemporal big data center 101 obtains the first data including geographic location, video data and sensing device data, the situation awareness engine 102 determines the distribution of personnel resources, the situation of target persons and the situation of target vehicles in the urban area according to the first data, and the city full-factor visualization engine 103 performs three-dimensional visualization of the distribution of personnel resources, the situation of target persons and the situation of target vehicles in the urban area. Finally, the command and dispatch module 104 receives alarm information or traffic accident information and obtains the real-time location of the police officer, determines the target police officer according to the alarm information or traffic accident information and the real-time location of the police officer, and renders and displays the alarm information or traffic accident information, the target police officer and the optimal path, so that the distribution of personnel resources, the situation of personnel, the situation of vehicles, the alarm information and the command and dispatch information in the urban area can be integrated in the three-dimensional scene for display, realizing the integration of the "information, command and action" business and the three-dimensional scene.

[0082] In some embodiments, the spatiotemporal big data center 101 is specifically used to: use the Flink CDC Connectors component to read the full data or incremental data in the MySQL, PostgreSQL, Oracle, and MongoDB databases to obtain the first data; use the Flink real-time computing engine to deduplicate, format, add three-domain identifiers, and perform coordinate conversion on the first data.

[0083] Exemplarily, if the spatiotemporal big data center 101 reads data from a database for the first time, it reads all the data in the database; if the spatiotemporal big data center 101 reads data from a database again, it reads incremental data in the database.

[0084] For example, data generated after the time of initial reading may be read as incremental data.

[0085] For another example, it is possible to detect whether the data at the current moment has changed compared to the data at the previous moment. If the data has changed, the data is used as incremental data; if the data has not changed, the data is not used as incremental data.

[0086] For example, the first data in the database is read for the first time at the first moment, and all the first data stored in the database is read, including the geographic location, video data, and sensor device data. When reading data at the second moment, if the geographic location at the second moment has not changed compared to the geographic location at the first moment, the sensor device at the second moment has not changed compared to the sensor device data at the first moment, and the video data at the second moment has changed compared to the video data at the first moment, then the incremental data for this data reading is the video data generated between the first moment and the second moment, and there is no need to read the temporal and geographical location and sensor device data between the first moment and the second moment, and the temporal and geographical location and sensor device data at the first moment can continue to be used.

[0087] In some embodiments, whether the first data at two moments has changed may be determined by means of a text vector.

[0088] In some scenarios, after receiving the first data, the database generates a first text vector for each data in the received first data, and each data corresponds to a first text vector. When the spatiotemporal big data center 101 reads the first data from the database at the first moment, it also reads the first text vector of each data in the first data. After a certain period of time, the database receives new first data, and the database calculates the second text vector of each data in the new first data. When the spatiotemporal big data center 101 reads certain data in the first data from the database at the second moment (such as the geographic location of a police officer, video data and perception data of a certain area, etc.), it first sends the first text vector of the data to be read to the database. The database calculates the distance between the first text vector and the second text vector of the data. If the distance is less than the threshold, it is determined that the data has not changed, and the database reports the information that the data has not changed to the spatiotemporal big data center 101. The spatiotemporal big data center 101 continues to use the data read at the first moment.

[0089] In some other scenarios, after receiving the first data, the database generates a first text vector for each data in the received first data, and each data corresponds to a first text vector. When the spatiotemporal big data center 101 reads the first data from the database at the first moment, it also reads the first text vector of each data in the first data. After a certain period of time, the database receives new first data, and the database calculates the second text vector of each data in the new first data. The database calculates the distance between the first text vector and the second text vector of the data. If the distance is less than a threshold (for example, the distance is 0 or close to 0), it is determined that the data has not changed, and the database reports the information that the data has not changed to the spatiotemporal big data center 101. The spatiotemporal big data center 101 continues to use the data read at the first moment.

[0090] In the embodiment of the present application, since the first data comes from different databases, different databases have different data storage formats, in order to facilitate the subsequent processing of the first data, the first data can be formatted and converted into a unified format. The three-domain identifier is added to add time, location and attributes to the first data, which is a technology well known to those skilled in the art and will not be described in the embodiment of the present application.

[0091] In addition, the video data in the first data usually involves the coordinates of the target person or target vehicle. Since different video acquisition devices usually use their own coordinate systems to represent coordinates, the coordinates of these video data need to be converted into the same coordinate system, such as the earth coordinate system.

[0092] Specifically, the coordinate conversion is as follows: obtaining the first coordinates of multiple target points in the video data, where the first coordinates are the coordinate values ​​of the multiple target points in the camera coordinate system; determining the rotation matrix R and the translation matrix T corresponding to the first coordinates converted to the second coordinates, where the second coordinates are the coordinate values ​​of the multiple target points in the earth coordinate system, and the centroid enclosed by the first coordinates of the multiple target points is v A The centroid formed by the second coordinates of the multiple target points is the Centroid B , rotation matrix R = VU T , translation matrix T = -R*Centroid A +Centroid B ; According to the rotation matrix R and the translation matrix T, the coordinates in the video data are converted into coordinates in the earth coordinate system.

[0093] In addition, the spatiotemporal big data center 101 can also provide geocoding services, which can convert the alarm address (such as the intersection of XX Road and XX Street in XX District) into location coordinates.

[0094] In some embodiments, the situation awareness engine 102 can be specifically used to: detect a target person or a target vehicle in the video data of the first data to obtain a Bounding Box of the target person or the target vehicle; determine the edge contour points of the target person or the target vehicle in the Bounding Box, and determine the average of the four leftmost, rightmost, topmost and bottommost coordinate values ​​of the edge contour points as the coordinates of the target person or the target vehicle; and predict the trajectory of the target person or the target vehicle based on the historical coordinates of the target person or the target vehicle.

[0095] Exemplarily, the historical coordinates of the target person or target vehicle may be input into a related trajectory prediction model for trajectory prediction, so as to better detect and track the target task or target vehicle.

[0096] In some embodiments, the situation awareness engine 102 can also be used to: detect images of vehicles from video data that appear at a preset geographic location more than a preset number of times at a preset time; extract a vehicle qualification certificate image from the image; extract image blocks from multiple preset areas in the vehicle qualification certificate image, each preset area being an area where a piece of information in the vehicle qualification certificate is located; perform OCR recognition on each image block to obtain a piece of information in the vehicle qualification certificate; generate vehicle qualification certificate information based on the information extracted from each image block and the preset area corresponding to each image block; match the generated vehicle qualification certificate information with the license plate to check the vehicle qualifications of the vehicle.

[0097] Specifically, if the vehicle qualifications found from the vehicle qualifications database based on the license plate correspond to the generated vehicle qualifications certificate information, the vehicle's vehicle qualifications inspection passes; if the vehicle qualifications found from the vehicle qualifications database based on the license plate do not correspond to the generated vehicle qualifications certificate information, the vehicle's vehicle qualifications inspection fails. After a vehicle fails the vehicle qualifications inspection, the vehicle can be detected and tracked through video data. Among them, the vehicle qualifications can be transportation qualifications, etc., and the vehicle qualifications database stores relevant information of all vehicles with transportation qualifications.

[0098] In the embodiment of the present application, the vehicle qualifications of the vehicle are checked, and vehicles that do not have certain vehicle qualifications can be detected and tracked, and the driving trajectories of these vehicles in urban areas can be monitored. When necessary, certain safety protection measures can be implemented on the vehicle to ensure the safety of urban residents.

[0099] In some embodiments, the city full-factor visualization engine 103 combines the public security factors in the city area with the city three-dimensional model for three-dimensional visualization, which can be specifically used for:

[0100] Store vector data including water systems, settlements and facilities, transportation and place names into a PostGIS database cluster;

[0101] Use the oblique model slicing tool to process the urban oblique model data into 3D Tiles format and store it in the MinIO object storage cluster;

[0102] Receive tile requests sent by the client through the city full-factor visualization engine according to the service address of vector tiles, and receive slice requests sent by the client through the city full-factor visualization engine according to the service address of 3D model slices; the service address of vector tiles can be the address of the PostGIS database cluster, and the service address of 3D model slices can be the address corresponding to the MinIO object storage cluster;

[0103] The server obtains the service name, level, row and column number of the vector tile according to the tile request, and calculates the coordinate range of the corresponding tile according to the tile request;

[0104] The server determines whether there is corresponding tile data in the cache according to the service name and the coordinate range; if there is no corresponding tile data, the server uses the coordinate range as a spatial constraint to perform a spatial query in the storage space corresponding to the service address of the vector tile, obtains the geometry and attribute information of the corresponding tile, and uses the ST_AsMvtGeom and ST_AsMVT functions of PostGIS to convert the vector data corresponding to the tile into the mvt format of MapBox; if there is corresponding tile data, the server returns the tile data in the cache to the client;

[0105] Use Geo3DtilesLayer to render the 3D model slices to obtain the 3D model layer;

[0106] Parse the tile data, obtain the spatial graphic data contained in the tile data, and convert the parsed spatial graphic data into spatial graphic data in Json format;

[0107] Use ColorMask to create a mask layer with spatial graphic data in Json format and bind mouse events;

[0108] The mask layer is bound to the 3D model layer, the 3D model bounding box is calculated using the spatial graphic data of the mask layer, and the 3D model layer is dynamically segmented based on the 3D model bounding box to achieve dynamic individualized rendering of the oblique photography layer.

[0109] In some embodiments, the command and dispatch module 104 can be specifically used to: extract the occurrence time, occurrence location and event content from the alarm information or traffic accident information; obtain the real-time location of each police officer; determine the target police officer who meets the preset conditions based on the occurrence location and the real-time location of each police officer; generate the optimal path from the real-time location of the target police officer to the occurrence location based on the occurrence location and the real-time location of the target police officer; render and display the alarm information or traffic accident information, the target police officer and the optimal path.

[0110] Exemplarily, the target police officers who meet the preset conditions may be one or more police officers whose distance from the location of occurrence in the alarm information or traffic accident information is less than a distance threshold or the closest police officers.

[0111] Specifically, the command and dispatch module 104 can be used to: obtain the 110 alarm, 119 fire alarm, and 122 traffic accident information from the alarm receiving platform in real time; obtain the location information of the law enforcement recorder and 350M wireless cluster in real time to obtain the real-time location of the police officer; use the geocoding service provided by the spatiotemporal big data center 101 to convert the alarm address into geographic coordinates; symbolize the location, overview, real-time location of the police officer, name, unit, terminal individual call, group call number and other information in a three-dimensional scene; perform automatic resource search according to different categories of situations, such as: surrounding police stations, group defense forces, fire hydrants, checkpoints, video surveillance, etc., conduct personnel resource accessibility and optimal path analysis, and render available resources in real time in a three-dimensional scene.

[0112] Based on the above-mentioned visualization decision system, an embodiment of the present application also provides a visualization decision method.

[0113] See also Figure 2 , the visual decision-making method may include the following steps:

[0114] Step 201, obtaining first data, wherein the first data includes geographic location, video data, and sensor device data.

[0115] Exemplarily, the Flink CDC Connectors component can be used to read full data or incremental data from MySQL, PostgreSQL, Oracle, and MongoDB databases to obtain first data; and then the Flink real-time computing engine is used to deduplicate, format, add three-domain identifiers, and perform coordinate conversion on the first data.

[0116] Exemplarily, the coordinate transformation is specifically as follows: obtaining first coordinates of multiple target points in the video data, where the first coordinates are the coordinate values ​​of the multiple target points in the camera coordinate system; determining the rotation matrix R and translation matrix T corresponding to the first coordinates converted to the second coordinates, where the second coordinates are the coordinate values ​​of the multiple target points in the earth coordinate system, and the centroid surrounded by the first coordinates of the multiple target points is the Centroid A The centroid formed by the second coordinates of the multiple target points is the Centroid B , rotation matrix R = VU T , translation matrix T = -R*Centroid A +Centroid B ; According to the rotation matrix R and the translation matrix T, the coordinates in the video data are converted into coordinates in the earth coordinate system.

[0117] Step 202, determining the public security factors in the urban area based on the first data, wherein the public security factors include the distribution of personnel resources, the situation of target persons, and the situation of target vehicles.

[0118] In some embodiments, for a target person or target vehicle, determining the situation of the target person or target vehicle in an urban area based on the first data may include: detecting the target person or target vehicle in the video data of the first data to obtain a Bounding Box of the target person or target vehicle; determining the edge contour points of the target person or target vehicle in the Bounding Box, and determining the average of the four leftmost, rightmost, topmost and bottommost coordinate values ​​of the edge contour points as the coordinates of the target person or target vehicle; and predicting the trajectory of the target person or target vehicle based on the historical coordinates of the target person or target vehicle.

[0119] Exemplarily, for a target vehicle, determining the situation of the target vehicle in an urban area based on the first data may include: detecting from video data an image of a vehicle that appears at a preset geographic location greater than a preset number of times within a preset time; extracting a vehicle qualification certificate image from the image; extracting image blocks from multiple preset areas in the vehicle qualification certificate image, each preset area being an area where a piece of information in the vehicle qualification certificate is located; performing OCR recognition on each image block to obtain a piece of information in the vehicle qualification certificate; generating vehicle qualification certificate information based on the information extracted from each image block and the preset area corresponding to each image block; matching the generated vehicle qualification certificate information with the license plate to check the vehicle qualifications of the vehicle.

[0120] Step 203: Perform a three-dimensional visualization of the public security elements in the urban area.

[0121] Exemplarily, the three-dimensional visualization of the public security elements in the urban area may specifically include:

[0122] Store vector data including water systems, settlements and facilities, transportation and place names into a PostGIS database cluster;

[0123] Use the oblique model slicing tool to process the urban oblique model data into 3D Tiles format and store it in the MinIO object storage cluster;

[0124] Receive tile requests sent by the client through the visualization engine according to the service address of vector tiles, and receive slice requests sent by the client through the visualization engine according to the service address of three-dimensional model slices;

[0125] The server obtains the service name, level, row and column number of the vector tile according to the tile request, and calculates the coordinate range of the corresponding tile according to the tile request;

[0126] The server determines whether there is corresponding tile data in the cache according to the service name and the coordinate range; if there is no corresponding tile data, the coordinate range is used as a spatial constraint to perform a spatial query, obtain the geometry and attribute information of the corresponding tile, and use the ST_AsMvtGeom and ST_AsMVT functions of PostGIS to convert the vector data corresponding to the tile into the mvt format of MapBox; if there is corresponding tile data, the tile data in the cache is returned to the client;

[0127] Use Geo3DtilesLayer to render the 3D model slices to obtain the 3D model layer;

[0128] Parse the tile data, obtain the spatial graphic data contained in the tile data, and convert the parsed spatial graphic data into spatial graphic data in Json format;

[0129] Use ColorMask to create a mask layer with spatial graphic data in Json format and bind mouse events;

[0130] The mask layer is bound to the 3D model layer, the 3D model bounding box is calculated using the spatial graphic data of the mask layer, and the 3D model layer is dynamically segmented based on the 3D model bounding box to achieve dynamic individualized rendering of the oblique photography layer.

[0131] Step 204, receiving alarm information or traffic accident information and obtaining the real-time location of the police officer, determining the target police officer according to the alarm information or traffic accident information and the real-time location of the police officer, and rendering and displaying the alarm information or traffic accident information, the target police officer and the optimal path.

[0132] Exemplarily, step 204 may specifically include: extracting the occurrence time, occurrence location and event content from the alarm information or traffic accident information; obtaining the real-time location of each police officer; determining the target police officer who meets the preset conditions based on the occurrence location and the real-time location of each police officer; generating the optimal path from the real-time location of the target police officer to the occurrence location based on the occurrence location and the real-time location of the target police officer; rendering and displaying the alarm information or traffic accident information, the target police officer and the optimal path.

[0133] The above-mentioned visualization decision-making method obtains first data including geographic location, video data and sensing device data, determines the distribution of human resources, target person conditions and target vehicle conditions in the urban area based on the first data, and integrates the distribution of human resources, target person conditions and target vehicle conditions in the urban area into the urban three-dimensional model for three-dimensional visualization. Finally, it receives alarm information or traffic accident information and obtains the real-time location of the police officer, determines the target police officer based on the alarm information or traffic accident information and the real-time location of the police officer, and renders and displays the alarm information or traffic accident information, the target police officer and the optimal path, so that the distribution of human resources, personnel conditions, vehicle conditions, alarm information and command and dispatch information in the urban area can be integrated in the three-dimensional scene for display, realizing the integration of "information, command and action" business and three-dimensional scene.

[0134] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0135] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A visual decision system, characterized in that: include: Spatiotemporal big data center, situation awareness engine, city full-factor visualization engine and command and dispatch module; The spatiotemporal big data center is used to obtain first data, where the first data includes geographic location, video data, and sensor device data; The situation awareness engine is used to determine the public security factors in the urban area according to the first data, wherein the public security factors include the distribution of personnel resources, the situation of target persons and the situation of target vehicles; The city full-factor visualization engine is used to perform three-dimensional visualization of public security factors within the city area; The command and dispatch module is used to receive alarm information or traffic accident information and obtain the real-time location of the police officer, determine the target police officer based on the alarm information or traffic accident information and the real-time location of the police officer, and render and display the alarm information or traffic accident information, the target police officer and the optimal path.

2. The visual decision system according to claim 1, characterized in that: The spatiotemporal big data center is specifically used for: Use the Flink CDC Connectors component to read full or incremental data from MySQL, PostgreSQL, Oracle, and MongoDB databases to obtain the first data. The Flink real-time computing engine is used to deduplicate, format, add three-field identifiers, and perform coordinate conversion on the first data.

3. The visual decision system according to claim 2, characterized in that: The coordinate transformation is specifically: Obtaining first coordinates of multiple target points in the video data, where the first coordinates are coordinate values ​​of the multiple target points in the camera coordinate system; Determine the rotation matrix R and translation matrix T corresponding to the conversion of the first coordinate to the second coordinate. The second coordinate is the coordinate value of the multiple target points in the earth coordinate system. The centroid surrounded by the first coordinates of the multiple target points is the Centroid. A The centroid formed by the second coordinates of the multiple target points is the Centroid B , rotation matrix R = VU T , translation matrix T = -R*Centroid A +Centroid B ; The coordinates in the video data are converted into coordinates in the earth coordinate system according to the rotation matrix R and the translation matrix T.

4. The visual decision system according to claim 1, characterized in that: The situation awareness engine is specifically used for: Detecting a target person or a target vehicle in the video data of the first data to obtain a Bounding Box of the target person or the target vehicle; Determine the edge contour points of the target person or the target vehicle in the Bounding Box, and determine the average of the leftmost, rightmost, topmost and bottommost four coordinate values ​​of the edge contour points as the coordinates of the target person or the target vehicle; The trajectory of the target person or target vehicle is predicted based on the historical coordinates of the target person or target vehicle.

5. The visual decision system according to claim 1, characterized in that: The situational awareness engine is also used to: Detecting images of vehicles that appear at a preset geographic location more than a preset number of times at a preset time from the video data; Extracting a vehicle qualification certificate image from the image; Extracting image blocks from a plurality of preset areas in the vehicle qualification certificate image, respectively, each preset area being an area where a piece of information in the vehicle qualification certificate is located; Perform OCR recognition on each image block to obtain a piece of information in the vehicle qualification certificate; Generate vehicle qualification certificate information based on the information extracted from each image block and the preset area corresponding to each image block; The generated vehicle qualification certificate information is matched with the license plate to check the vehicle's qualifications.

6. The visual decision system according to claim 1, characterized in that: The city full-factor visualization engine is specifically used for: Store vector data including water systems, settlements and facilities, transportation and place names into a PostGIS database cluster; Use the oblique model slicing tool to process the urban oblique model data into 3D Tiles format and store it in the MinIO object storage cluster; Receive tile requests sent by the client through the city full-factor visualization engine according to the service address of vector tiles, and receive slice requests sent by the client through the city full-factor visualization engine according to the service address of three-dimensional model slices; The server obtains the service name, level, row and column number of the vector tile according to the tile request, and calculates the coordinate range of the corresponding tile according to the tile request; The server determines whether corresponding tile data exists in the cache according to the service name and the coordinate range; If the corresponding tile data does not exist, the coordinate range is used as a spatial constraint to perform a spatial query to obtain the geometry and attribute information of the corresponding tile, and the ST_AsMvtGeom and ST_AsMVT functions of PostGIS are used to convert the vector data corresponding to the tile into the mvt format of MapBox; if the corresponding tile data exists, the tile data in the cache is returned to the client; Use Geo3DtilesLayer to render the 3D model slices to obtain the 3D model layer; Parse the tile data, obtain the spatial graphic data contained in the tile data, and convert the parsed spatial graphic data into spatial graphic data in Json format; Use ColorMask to create a mask layer with spatial graphic data in Json format and bind mouse events; The mask layer is bound to the 3D model layer, the 3D model bounding box is calculated using the spatial graphic data of the mask layer, and the 3D model layer is dynamically segmented based on the 3D model bounding box to achieve dynamic individualized rendering of the oblique photography layer.

7. The visual decision system according to claim 1, characterized in that: The command and dispatch module is specifically used for: Extract the time, location and content of the incident from the alarm information or traffic accident information; Get the real-time location of each police officer; Determine the target police officer who meets the preset conditions according to the occurrence location and the real-time location of each police officer; According to the occurrence location and the real-time location of the target police officer, an optimal path from the real-time location of the target police officer to the occurrence location is generated; The alarm information or traffic accident information, target police officer and optimal path are rendered and displayed.

8. A visual decision-making method, characterized in that: include: Acquire first data, where the first data includes geographic location, video data, and sensor device data; Determine the public security factors in the urban area according to the first data, wherein the public security factors include the distribution of personnel resources, the situation of target persons and the situation of target vehicles; Performing a three-dimensional visual display of public security elements within the urban area; Receive alarm information or traffic accident information and obtain the real-time location of the police officer, determine the target police officer according to the alarm information or traffic accident information and the real-time location of the police officer, and render and display the alarm information or traffic accident information, the target police officer and the optimal path.

9. The visual decision-making method according to claim 8, characterized in that: The three-dimensional visualization of the public security elements in the urban area includes: Store vector data including water systems, settlements and facilities, transportation and place names into a PostGIS database cluster; Use the oblique model slicing tool to process the urban oblique model data into 3D Tiles format and store it in the MinIO object storage cluster; Receive tile requests sent by the client through the visualization engine according to the service address of vector tiles, and slice requests sent by the client through the visualization engine according to the service address of three-dimensional model slices; The server obtains the service name, level, row and column number of the vector tile according to the tile request, and calculates the coordinate range of the corresponding tile according to the tile request; The server determines whether there is corresponding tile data in the cache according to the service name and the coordinate range; if there is no corresponding tile data, the coordinate range is used as a spatial constraint to perform a spatial query, obtain the geometry and attribute information of the corresponding tile, and use the ST_AsMvtGeom and ST_AsMVT functions of PostGIS to convert the vector data corresponding to the tile into the mvt format of MapBox; if there is corresponding tile data, the tile data in the cache is returned to the client; Use Geo3DtilesLayer to render the 3D model slices to obtain the 3D model layer; Parse the vector tile data, obtain the spatial graphic data contained in the vector tile data, and convert the parsed spatial graphic data into spatial graphic data in Json format; Use ColorMask to create a mask layer with spatial graphic data in Json format and bind mouse events; The mask layer is bound to the 3D model layer, the 3D model bounding box is calculated using the spatial graphic data of the mask layer, and the 3D model layer is dynamically segmented based on the 3D model bounding box to achieve dynamic individualized rendering of the oblique photography layer.

10. The visual decision-making method according to claim 8, characterized in that: Determine the target person or vehicle situation in the urban area according to the first data, specifically: Detecting a target person or a target vehicle in the video data of the first data to obtain a Bounding Box of the target person or the target vehicle; Determine the edge contour points of the target person or the target vehicle in the Bounding Box, and determine the average of the leftmost, rightmost, topmost and bottommost four coordinate values ​​of the edge contour points as the coordinates of the target person or the target vehicle; The trajectory of the target person or target vehicle is predicted based on the historical coordinates of the target person or target vehicle.

Images