Intelligent building dynamic display system based on AI video recognition and digital twin

Abstract

The present invention provides a dynamic display system for intelligent buildings based on AI video recognition and digital twins, wherein the system includes: a video recognition model training subsystem for training video recognition models; a real-time video data collection subsystem for collecting real-time video data; a dangerous event identification subsystem for identifying dangerous events based on the video recognition model, according to real-time video data and video recognition requirements; a dynamic display subsystem for building a dynamic display model of an intelligent building and marking dangerous events according to real-time video data. The dynamic display system for intelligent buildings based on AI video recognition and digital twins of the present invention trains a video recognition model, and identifies dangerous events according to real-time video data, video recognition requirements and video recognition models. By integrating real-time video data and digital twin models to obtain a dynamic display model of an intelligent building, and then marking dangerous areas, the recognition efficiency of dangerous events is higher and the positioning is more timely.

Description

Technical Field

[0001] The present invention relates to the field of digital twin technology, and in particular to an intelligent building dynamic display system based on AI video recognition and digital twin. Background Art

[0002] AI video recognition is a process that uses artificial intelligence technology to detect, analyze, and process video content. It involves a variety of technologies, including but not limited to object detection, object recognition, target tracking, real-time video analysis, and triggering real-time alarms. At the same time, digital twin is a concept that uses digital technology to create a virtual copy of a physical entity. This virtual copy can simulate the state, behavior, and performance of the physical entity for a variety of purposes, such as analysis, prediction, optimization, and maintenance. Digital twin technology has applications in smart buildings, industrial automation, urban planning, medical equipment, and many other fields. Using digital twin technology to dynamically display smart buildings, the design, function, operation, and other information of smart buildings are presented to users in a dynamic and interactive way, which can not only provide users with visual intuition, but also display the real-time status and performance of the building through the integration of real-time data.

[0003] The invention patent with application number: CN202210131244.8 discloses a machine vision behavior intention prediction method applied to intelligent buildings, wherein the method includes the following steps: pedestrian detection, pedestrian tracking, establishment of spatiotemporal operators for action description, action detection and time boundary definition, environmental state change detection, key action clustering and behavior prediction; the above invention learns from historical video data, does not rely on manually set rules, and has fewer limitations on use. By clustering the action set before the environmental state changes, the key action vector is obtained, the human behavior and environmental state changes in the video are analyzed and connected, and the environmental state changes that need to be executed when the key action occurs are predicted. When the key action occurs in the video, the corresponding prediction signal is output for automatic execution. It has high accuracy and meets the real-time requirements. It can improve the intelligence level of intelligent buildings, reduce manual operations to a certain extent, and bring convenience to people's daily activities.

[0004] However, the above-mentioned prior art determines environmental variables based on video data. Video data is two-dimensional data and contains less information. Therefore, there is a certain lag when performing intent recognition and the intent recognition result cannot be responded to in time.

[0005] In view of this, there is an urgent need for an intelligent building dynamic display system based on AI video recognition and digital twins to at least solve the above-mentioned shortcomings. Summary of the invention

[0006] One of the purposes of the present invention is to provide an intelligent building dynamic display system based on AI video recognition and digital twins, train a video recognition model, and identify dangerous events based on real-time video data, video recognition requirements, and video recognition models. At the same time, the digital twin model of the building that integrates real-time video data and dynamic display obtains the intelligent building dynamic display model and marks the dangerous area, which improves the recognition efficiency of dangerous events and makes the location of dangerous events more timely.

[0007] The intelligent building dynamic display system based on AI video recognition and digital twins provided in an embodiment of the present invention includes:

[0008] Video recognition model training subsystem, used to train video recognition models;

[0009] A real-time video data collection subsystem, used for collecting real-time video data;

[0010] The dangerous event identification subsystem is used to obtain video identification requirements and identify dangerous events based on the video identification model, real-time video data and video identification requirements;

[0011] The dynamic display subsystem is used to build a dynamic display model of an intelligent building based on digital twin technology and real-time video data, and mark the dangerous areas corresponding to dangerous events in the dynamic display model of the intelligent building.

[0012] Preferably, the video recognition model training subsystem includes:

[0013] The video annotation module is used to annotate historical video data and obtain annotated data. The annotation contents of the historical video data include: objects, actions and scenes.

[0014] An extraction feature determination module is used to determine the extraction features based on the deep learning network and the labeled data;

[0015] A target video recognition requirement acquisition module is used to obtain target video recognition requirements;

[0016] The target model construction module is used to construct the target model according to the target video recognition requirements. The target model construction includes: screening the target model type and setting the network structure. The target model types include: target detection model and behavior recognition model;

[0017] A corresponding information acquisition module is used to obtain corresponding information of extracted features and labeled data;

[0018] The model training module is used to input the corresponding information into the target model and train the video recognition model.

[0019] Preferably, the real-time video data collection subsystem includes:

[0020] A focus area determination module, used to determine the focus area of ​​a target building;

[0021] A security information acquisition module is used to acquire security information of security devices associated with the area of ​​interest;

[0022] The real-time video data acquisition module is used to analyze the security information of the security equipment and obtain real-time video data.

[0023] Preferably, the region of interest determination module includes:

[0024] The regional attribute label acquisition submodule is used to determine the building partition of the target building and obtain the regional attribute label of the building partition;

[0025] A first management area attribute label acquisition submodule, used to determine a display object of the display content, and try to acquire a first management area attribute label of the display object;

[0026] A first concerned area determination submodule, configured to determine the concerned area according to the first management area attribute tag and the area attribute tag if the acquisition attempt is successful;

[0027] A management attribute extraction record acquisition submodule is used to acquire the management attribute extraction record of the display object if the acquisition attempt fails;

[0028] A second management area attribute label generation submodule, used for extracting records according to management attributes and generating a second management area attribute label;

[0029] The concerned area determination submodule is used to determine the concerned area according to the second management area attribute tag and the area attribute tag.

[0030] Preferably, the management attribute extraction record acquisition submodule includes:

[0031] A publishing task queue acquisition unit is used to acquire a publishing task queue of a display object;

[0032] A first partition keyword determination unit is used to parse each target publishing task in the publishing task queue from beginning to end, determine the first partition keyword, and use the target publishing task corresponding to the first partition keyword as the first publishing task;

[0033] The second release task determination unit is used to determine the completion status of the first release task, and if the completion status is that the task is not completed, the first release task is used as the second release task;

[0034] The third release task determination unit is used to traverse from the queue position of the second release task to the head of the release task queue, determine the second partition keyword, and use the first release task corresponding to the second partition keyword as the third release task;

[0035] The management attribute extraction record determination unit is used to set the first release task between the second release task and the third release task as the fourth release task, and set the second release task, the third release task and the fourth release task together as the management attribute extraction record.

[0036] Preferably, the second management area attribute label generation submodule includes:

[0037] A third partition keyword determination unit, used for extracting records according to management attributes and determining a third partition keyword;

[0038] A first target area determining unit, configured to determine the first target area according to the third partition keyword;

[0039] A task completion time difference acquisition unit, used to acquire a task completion time difference of the first published task for extracting the third partition keyword;

[0040] A region information acquisition unit, used for acquiring region information of a first target region;

[0041] The area adjustment determination unit is used to determine whether the first target area needs to be adjusted according to the task completion time difference and the area information;

[0042] A second target area determining unit, configured to use the first target area as the second target area if there is no need to adjust the first target area;

[0043] An auxiliary information acquisition unit, used for parsing the first release task between the second release task and the third release task to acquire auxiliary information if the first target area needs to be adjusted;

[0044] A third target region determining unit, configured to adjust the first target region according to the auxiliary information to obtain an adjusted third target region;

[0045] The second management area attribute label generating unit is used to label the second target area or the third target area to generate a second management area attribute label.

[0046] Preferably, the dangerous event identification subsystem includes:

[0047] The recognition feature input determination module is used to determine the recognition feature input according to the video recognition requirements;

[0048] The dangerous event determination module is used to determine dangerous events based on recognition feature input and video recognition model.

[0049] Preferably, the dynamic display subsystem includes:

[0050] The intelligent building dynamic display model acquisition module is used to fuse the real-time video data with the digital twin model of the target building to obtain the intelligent building dynamic display model;

[0051] A danger feature acquisition module is used to acquire danger features of dangerous events;

[0052] A module for determining the scope of hazard impact, used to determine the scope of hazard impact according to hazard characteristics;

[0053] The danger zone determination module is used to determine and mark the danger zone according to the danger impact range.

[0054] Preferably, the intelligent building dynamic display model acquisition module includes:

[0055] The operating platform acquisition submodule is used to obtain the operating platform of the digital twin model;

[0056] The element conversion submodule is used to convert real-time video data into UI elements by running the platform's onboard controls;

[0057] The real-time display submodule is used to display the UI elements in real time at the top layer of the virtual scene viewport of the running platform.

[0058] The method for dynamic display of intelligent buildings based on AI video recognition and digital twins provided in an embodiment of the present invention includes:

[0059] Step 1: Train the video recognition model;

[0060] Step 2: Collect real-time video data;

[0061] Step 3: Obtain video recognition requirements, and identify dangerous events based on the video recognition model, real-time video data and video recognition requirements;

[0062] Step 4: Based on digital twin technology and real-time video data, a dynamic display model of the intelligent building is constructed, and the dangerous areas corresponding to the dangerous events are marked in the dynamic display model of the intelligent building.

[0063] The beneficial effects of the present invention are:

[0064] The present invention trains a video recognition model to identify dangerous events based on real-time video data, video recognition requirements and video recognition models. At the same time, the digital twin model of the real-time video data and the dynamically displayed building is integrated to obtain the dynamic display model of the intelligent building and mark the dangerous area, which improves the recognition efficiency of dangerous events and makes the location of dangerous events more timely.

[0065] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the present application documents.

[0066] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0068] Figure 1 Schematic diagram of an intelligent building dynamic display system based on AI video recognition and digital twins in an embodiment of the present invention;

[0069] Figure 2 It is a schematic diagram of a method for dynamic display of intelligent buildings based on AI video recognition and digital twins in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0071] The embodiment of the present invention provides a dynamic display system of intelligent buildings based on AI video recognition and digital twins, such as Figure 1 As shown, including:

[0072] The video recognition model training subsystem 1 is used to train the video recognition model; wherein the video recognition model is: an AI model for analyzing and processing video data;

[0073] The real-time video data collection subsystem 2 is used to collect real-time video data; wherein the real-time video data is: video information captured by a camera installed in a building performing dynamic display of an intelligent building;

[0074] Dangerous event identification subsystem 3 is used to obtain video identification requirements, identify dangerous events based on video identification models, according to real-time video data and video identification requirements; wherein, video identification requirements are: the content that managers hope the video identification model can identify, such as: detecting the wearing of safety helmets by workers in the building; dangerous events are: the identification results identified by the video identification model that meet the video identification requirements, such as: the situation where workers are not wearing safety helmets;

[0075] The dynamic display subsystem 4 is used to construct a dynamic display model of an intelligent building based on the digital twin technology and real-time video data, and mark the dangerous area corresponding to the dangerous event in the dynamic display model of the intelligent building. The dynamic display model of the intelligent building is: a virtual model that integrates real-time video data and a digital twin model of a building for dynamic display of the intelligent building, so that the real-time state of the building can be intuitively seen in the digital twin model; the dangerous area is: the range that the dangerous event can affect is in the area corresponding to the dynamic display model of the intelligent building.

[0076] The working principle and beneficial effects of the above technical solution are:

[0077] This application trains a video recognition model to identify dangerous events based on real-time video data, video recognition requirements, and video recognition models. At the same time, the real-time video data and the digital twin model of the dynamically displayed building are integrated to obtain the dynamic display model of the intelligent building and mark the dangerous areas, which improves the recognition efficiency of dangerous events and makes the location of dangerous events more timely.

[0078] In one embodiment, the video recognition model training subsystem includes:

[0079] The video annotation module is used to annotate historical video data and obtain annotated data. The annotation content of the historical video data includes: objects, actions and scenes; wherein the historical video data is: videos of the interior and exterior of the target building recorded in history;

[0080] The extraction feature determination module is used to determine the extraction features based on the deep learning network and the labeled data; wherein the extraction features are useful features in the labeled data, such as color, texture, shape, and motion;

[0081] The target video recognition requirement acquisition module is used to acquire the target video recognition requirement; wherein the target video recognition requirement is: a manually set video recognition task;

[0082] The target model construction module is used to construct the target model according to the target video recognition requirements. The target model construction includes: screening the target model types and setting the network structure. The target model types include: target detection model and behavior recognition model; the network structure is: the specific architecture of the target model, including: the number of network layers and the number of neurons in each layer of the network and other information;

[0083] A corresponding information acquisition module is used to obtain corresponding information of the extracted features and the annotated data; wherein the corresponding information is: information about where in the annotated data the extracted features are extracted and what kind of extracted features are extracted;

[0084] The model training module is used to input the corresponding information into the target model and train the video recognition model.

[0085] The working principle and beneficial effects of the above technical solution are:

[0086] This application obtains labeled data with objects, actions and scenes, and introduces a deep learning network to determine the extracted features. The target video recognition requirements (manually set video recognition tasks) are obtained, and according to the target video recognition requirements, the target model type and the network structure of the target model are screened. The corresponding information of the extracted features and the labeled data is input into the target model to train the video recognition model, which improves the suitability of the model training.

[0087] In one embodiment, the real-time video data collection subsystem includes:

[0088] The focus area determination module is used to determine the focus area of ​​the target building; wherein the target building is: a building that needs to be dynamically displayed; the focus area is: an area that needs to be managed in the target building;

[0089] The security information acquisition module is used to acquire security information of security equipment associated with the area of ​​interest; wherein the security equipment associated with the area of ​​interest is: a camera device that captures the area of ​​interest; the security information is: data collected by the security equipment, such as: captured surveillance video;

[0090] The real-time video data acquisition module is used to analyze the security information of the security equipment and obtain real-time video data.

[0091] The working principle and beneficial effects of the above technical solution are:

[0092] The present application determines the security equipment in the focus area of ​​the target building, analyzes the security information of the security equipment to obtain real-time video data, and improves the rationality of real-time video data acquisition.

[0093] In one embodiment, the region of interest determination module includes:

[0094] The regional attribute label acquisition submodule is used to determine the building partition of the target building and obtain the regional attribute label of the building partition; wherein the building partition is the smallest division area obtained by the management personnel by dividing the internal and external areas of the target building; the regional attribute label is the regional identification, which includes regional information, such as: regional shape, location and size;

[0095] The first management area attribute label acquisition submodule is used to determine the display object of the display content and try to obtain the first management area attribute label of the display object; wherein the display content is: the content of the dynamic display of the intelligent building; the display object is: the viewer of the display content; the first management area attribute label is: the preset area identifier of the display object that can manage the target area;

[0096] The first determination submodule of the area of ​​interest is used to determine the area of ​​interest according to the first management area attribute tag and the regional attribute tag if the acquisition attempt is successful; wherein, when determining the area of ​​interest according to the first management area attribute tag and the regional attribute tag, the first management area attribute tag and the regional attribute tag are matched, and the building partition corresponding to the successfully matched regional attribute tag is used as the area of ​​interest;

[0097] The management attribute extraction record acquisition submodule is used to acquire the management attribute extraction record of the display object if the acquisition attempt fails; wherein the management attribute extraction record is used to extract the information of the building partition managed by the display object;

[0098] A second management area attribute label generation submodule is used to generate a second management area attribute label according to the management attribute extraction record; wherein the second management area attribute label is: an area identifier of a display object management target area extracted according to the management attribute extraction record;

[0099] The focus area determination submodule is used to determine the focus area according to the second management area attribute tag and the regional attribute tag. When determining the focus area according to the second management area attribute tag and the regional attribute tag, the second management area attribute tag and the regional attribute tag are matched, and the building partition corresponding to the successfully matched regional attribute tag is used as the focus area.

[0100] The working principle and beneficial effects of the above technical solution are:

[0101] The present application determines the regional attribute tag of the smallest divided area of ​​the target building, and at the same time, obtains the display object of the display content, and attempts to obtain the first management regional attribute tag of the display object. If the attempt to obtain is successful, the first management regional attribute tag and the regional attribute tag are directly matched, and the building partition corresponding to the successfully matched regional attribute tag is used as the focus area. If the attempt to obtain fails, the management attribute extraction record for extracting information about the building partition managed by the display object is obtained. A second management regional attribute tag corresponding to the management attribute extraction record is generated, and the focus area is determined based on the second management regional attribute tag and the regional attribute tag, and the determination of the focus area is more appropriate.

[0102] In one embodiment, the management attribute extraction record acquisition submodule includes:

[0103] The publishing task queue acquisition unit is used to acquire the publishing task queue of the display object; wherein the publishing task queue is a queue sorted according to the publishing time of the publishing task, and the publishing task is a task that the display object needs to perform, such as checking the solid waste treatment status of area A;

[0104] The first partition keyword determination unit is used to parse each target publishing task in the publishing task queue from beginning to end, determine the first partition keyword, and use the target publishing task corresponding to the first partition keyword as the first publishing task; wherein the target publishing task is: the publishing task being parsed in the publishing task queue; the first partition keyword is: the keyword describing the building partition extracted from the publishing task;

[0105] The second release task determination unit is used to determine the completion status of the first release task, and if the completion status is that the task is not completed, the first release task is used as the second release task;

[0106] The third release task determination unit is used to traverse from the queue position of the second release task to the head of the release task queue, determine the second partition keyword, and use the first release task corresponding to the second partition keyword as the third release task; wherein the second partition keyword is: the first partition keyword corresponding to the first release task with the first partition keyword when traversing from the queue position of the second release task to the head of the release task queue;

[0107] The management attribute extraction record determination unit is used to set the first release task between the second release task and the third release task as the fourth release task, and set the second release task, the third release task and the fourth release task together as the management attribute extraction record.

[0108] The working principle and beneficial effects of the above technical solution are:

[0109] When determining the management attribute extraction record, the task assigned to the management personnel can be obtained, and the relevant information about the regional management can be extracted from the task. Therefore, the present application obtains the release task queue of the display object, parses each target release task in the release task queue from beginning to end, determines the first partition keyword, and takes the target release task corresponding to the first partition keyword as the first release task. Generally, the completed first release task display object will not be viewed immediately, but the most recently completed first release task can provide the location information of the first release task. At the same time, the optimal execution strategy is generally considered when the task is executed, so the location information can provide the orientation guidance for the building area that is about to be executed and has the intention to be viewed. The first release task is traversed from the queue position of the second release task to the head of the release task queue, and the first release task with the first partition keyword is taken as the third release task. At the same time, the first partition keyword corresponding to the third release task is taken as the second partition keyword. The second release task, the third release task and the fourth release task are taken together as the management attribute extraction record, which improves the rationality of the subsequent extraction of the area of ​​interest based on the management attribute extraction record.

[0110] In one embodiment, the second management area attribute label generation submodule includes:

[0111] A third partition keyword determination unit is used to extract records according to management attributes and determine a third partition keyword; wherein the third partition keyword is: the first partition keyword and the second partition keyword;

[0112] The first target area determination unit is used to determine the first target area according to the third partition keyword; wherein, when determining the first target area according to the third partition keyword, a distribution map of the target building is obtained, and according to the third partition keyword, the building partition corresponding to the third partition keyword in the distribution map of the target building is determined and marked, and according to the marked building partition, the smallest enclosing area in the distribution map of the target building that can contain the marked building partition is determined and used as the first target area;

[0113] The task completion time difference acquisition unit is used to acquire the task completion time difference of the first published task for extracting the third partition keyword; wherein the task completion time difference is: the maximum time difference value of the task release time of the first published task for extracting the third partition keyword;

[0114] The area information acquisition unit is used to acquire the area information of the first target area; wherein the area information includes: the number and area of ​​the building partitions included in the first target area and the estimated execution time of the tasks executed in the building partitions;

[0115] The area adjustment determination unit is used to determine whether the first target area needs to be adjusted according to the task completion time difference and the area information; wherein, when determining whether the first target area needs to be adjusted according to the task completion time difference and the area information, the area residence time is estimated according to the area information, and if the task completion time difference is less than the area residence time, the number of building partitions included in the first target area needs to be reduced;

[0116] A second target area determining unit, configured to use the first target area as the second target area if there is no need to adjust the first target area;

[0117] The auxiliary information acquisition unit is used to parse the first release task between the second release task and the third release task to acquire auxiliary information if the first target area needs to be adjusted; wherein the auxiliary information is: the task semantics of the first release task between the second release task and the third release task;

[0118] A third target area determination unit is used to adjust the first target area according to the auxiliary information to obtain the adjusted third target area; wherein when adjusting the first target area according to the auxiliary information, the conflicting building partitions of the first target area are determined according to the task semantics, and all conflicting building partitions are eliminated to obtain the third target area;

[0119] The second management area attribute label generating unit is used to label the second target area or the third target area to generate the second management area attribute label. The labeling of the second target area or the third target area includes: determining the area attribute label of the building partition included in the second target area or the third target area.

[0120] The working principle and beneficial effects of the above technical solution are:

[0121] The present application determines the smallest enclosing area that can contain the marked building partition in the distribution map of the target building, that is, the first target area, based on the third partition keyword extracted from the management attribute extraction record. The regional information of the first target area is obtained, including the number and area of ​​the building partitions and the estimated execution time of the tasks executed in the building partitions. According to the regional information, the regional residence time is estimated. In addition, the task completion time difference of the first published task that extracts the third partition keyword is obtained, and the task completion time difference is compared with the estimated regional residence time. If the task completion time difference is greater than or equal to the regional residence time, the corresponding first target area is used as the second target area. If the task completion time difference is less than the regional residence time, the task semantics is determined according to the first published task between the second published task and the third published task. According to the task semantics, the conflicting building partitions are determined and removed from the first target area to obtain the third target area. The regional attribute label of the building partition contained in the second target area or the third target area is determined and used as the second management area attribute label. The generation efficiency of the second management area attribute label is higher and more appropriate.

[0122] In one embodiment, the dangerous event identification subsystem includes:

[0123] The recognition feature input determination module is used to determine the recognition feature input according to the video recognition requirement; wherein the recognition feature input is: the recognition direction constraint of the video recognition model determined according to the video recognition requirement;

[0124] The dangerous event determination module is used to determine dangerous events based on recognition feature input and video recognition model.

[0125] The working principle and beneficial effects of the above technical solution are:

[0126] The present application characterizes the video recognition requirements to obtain recognition feature inputs that characterize the recognition direction constraints of the video recognition model, and determines dangerous events based on the recognition feature inputs and the video recognition model, thereby improving the accuracy of dangerous event recognition.

[0127] In one embodiment, the dynamic display subsystem includes:

[0128] A three-dimensional information acquisition module is used to acquire three-dimensional information of a target building; wherein the three-dimensional information is: three-dimensional scanning information of the target building;

[0129] Digital twin model module, used to build a digital twin model based on three-dimensional information;

[0130] The intelligent building dynamic display model acquisition module is used to fuse the real-time video data with the digital twin model of the target building to obtain the intelligent building dynamic display model;

[0131] The danger feature acquisition module is used to acquire the danger features of the danger event; wherein the danger feature is: a characteristic representation of the event information of the danger event, such as: the affected object, affected scope and affected duration of the danger event;

[0132] A module for determining the scope of hazard impact, used to determine the scope of hazard impact according to hazard characteristics;

[0133] The danger zone determination module is used to determine and mark the danger zone according to the danger impact range.

[0134] The working principle and beneficial effects of the above technical solution are:

[0135] This application obtains the three-dimensional information (three-dimensional scanning information) of the target building, builds a digital twin model based on the three-dimensional information, and integrates the real-time video data with the digital twin model to obtain a dynamic display model of the intelligent building. Characterize the dangerous events to obtain the dangerous features, determine the dangerous impact range based on the dangerous features, and mark the dangerous areas in the dynamic display model of the intelligent building accordingly, so as to locate the dangerous areas in time and facilitate subsequent measures.

[0136] In one embodiment, the intelligent building dynamic display model acquisition module includes:

[0137] The operating platform acquisition submodule is used to obtain the operating platform of the digital twin model; wherein the operating platform is: a software environment that supports the operation of the digital twin model;

[0138] The element conversion submodule is used to convert the real-time video data into UI elements by running the onboard controls of the platform; wherein the onboard controls are user interface components used to process and display real-time video data; and UI elements are user interface elements, such as text, images, animations or other visual objects.

[0139] The real-time display submodule is used to display the UI elements in real time on the top layer of the virtual scene viewport of the running platform. The virtual scene viewport is the window that displays the digital twin model on the running platform; the top layer real-time display is: the UI elements are displayed on the top of the virtual scene viewport in a way that covers other elements.

[0140] The working principle and beneficial effects of the above technical solution are:

[0141] This application converts real-time video data into user interface elements through a user interface component that processes and displays real-time video data on the running platform of the digital twin model, and displays the user interface elements on the top layer of the window that displays the digital twin model on the running platform, thereby improving the timeliness of UI display.

[0142] The embodiment of the present invention provides a method for dynamic display of intelligent buildings based on AI video recognition and digital twins, such as Figure 2 As shown, including:

[0143] Step 1: Train the video recognition model;

[0144] Step 2: Collect real-time video data;

[0145] Step 3: Obtain video recognition requirements, and identify dangerous events based on the video recognition model, real-time video data and video recognition requirements;

[0146] Step 4: Based on digital twin technology and real-time video data, a dynamic display model of the intelligent building is constructed, and the dangerous areas corresponding to the dangerous events are marked in the dynamic display model of the intelligent building.

[0147] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. Intelligent building dynamic display system based on AI video recognition and digital twin, characterized by: include: Video recognition model training subsystem, used to train video recognition models; A real-time video data collection subsystem, used for collecting real-time video data; The dangerous event identification subsystem is used to obtain video identification requirements and identify dangerous events based on the video identification model, real-time video data and video identification requirements; The dynamic display subsystem is used to build a dynamic display model of an intelligent building based on digital twin technology and real-time video data, and mark the dangerous areas corresponding to dangerous events in the dynamic display model of the intelligent building; The real-time video data collection subsystem comprises: A focus area determination module, used to determine the focus area of ​​a target building; A security information acquisition module is used to acquire security information of security devices associated with the area of ​​interest; A real-time video data acquisition module is used to analyze the security information of security equipment and obtain real-time video data; The focus area determination module includes: The regional attribute label acquisition submodule is used to determine the building partition of the target building and obtain the regional attribute label of the building partition; A first management area attribute label acquisition submodule, used to determine a display object of the display content, and try to acquire a first management area attribute label of the display object; A first concerned area determination submodule, configured to determine the concerned area according to the first management area attribute tag and the area attribute tag if the acquisition attempt is successful; A management attribute extraction record acquisition submodule is used to acquire the management attribute extraction record of the display object if the acquisition attempt fails; A second management area attribute label generation submodule, used for extracting records according to management attributes and generating a second management area attribute label; The concerned area determination submodule is used to determine the concerned area according to the second management area attribute tag and the area attribute tag.

2. The intelligent building dynamic display system based on AI video recognition and digital twins according to claim 1, characterized in that: Video recognition model training subsystem, including: The video annotation module is used to annotate historical video data and obtain annotated data. The annotation contents of the historical video data include: objects, actions and scenes. An extraction feature determination module is used to determine the extraction features based on the deep learning network and the labeled data; A target video recognition requirement acquisition module is used to obtain target video recognition requirements; The target model construction module is used to construct the target model according to the target video recognition requirements. The target model construction includes: screening the target model type and setting the network structure. The target model types include: target detection model and behavior recognition model; A corresponding information acquisition module is used to obtain corresponding information of extracted features and labeled data; The model training module is used to input the corresponding information into the target model and train the video recognition model.

3. The intelligent building dynamic display system based on AI video recognition and digital twins according to claim 1, characterized in that: Management attribute extraction record acquisition submodule, including: A publishing task queue acquisition unit is used to acquire a publishing task queue of a display object; A first partition keyword determination unit is used to parse each target publishing task in the publishing task queue from beginning to end, determine the first partition keyword, and use the target publishing task corresponding to the first partition keyword as the first publishing task; The second release task determination unit is used to determine the completion status of the first release task, and if the completion status is that the task is not completed, the first release task is used as the second release task; The third release task determination unit is used to traverse from the queue position of the second release task to the head of the release task queue, determine the second partition keyword, and use the first release task corresponding to the second partition keyword as the third release task; The management attribute extraction record determination unit is used to set the first release task between the second release task and the third release task as the fourth release task, and set the second release task, the third release task and the fourth release task together as the management attribute extraction record.

4. The intelligent building dynamic display system based on AI video recognition and digital twins according to claim 1, characterized in that: The second management area attribute label generation submodule includes: A third partition keyword determination unit, used for extracting records according to management attributes and determining a third partition keyword; A first target area determining unit, configured to determine the first target area according to the third partition keyword; A task completion time difference acquisition unit, used to acquire a task completion time difference of the first published task for extracting the third partition keyword; A region information acquisition unit, used for acquiring region information of a first target region; The area adjustment determination unit is used to determine whether the first target area needs to be adjusted according to the task completion time difference and the area information; A second target area determining unit, configured to use the first target area as the second target area if there is no need to adjust the first target area; An auxiliary information acquisition unit, used for parsing the first release task between the second release task and the third release task to acquire auxiliary information if the first target area needs to be adjusted; A third target region determining unit, configured to adjust the first target region according to the auxiliary information to obtain an adjusted third target region; The second management area attribute label generating unit is used to label the second target area or the third target area to generate a second management area attribute label.

5. The intelligent building dynamic display system based on AI video recognition and digital twins according to claim 1, characterized in that: Dangerous event identification subsystem, including: The recognition feature input determination module is used to determine the recognition feature input according to the video recognition requirements; The dangerous event determination module is used to determine dangerous events based on recognition feature input and video recognition model.

6. The intelligent building dynamic display system based on AI video recognition and digital twins according to claim 1, characterized in that: Dynamic display subsystem, including: The intelligent building dynamic display model acquisition module is used to fuse the real-time video data with the digital twin model of the target building to obtain the intelligent building dynamic display model; A danger feature acquisition module is used to acquire danger features of dangerous events; A module for determining the scope of hazard impact, used to determine the scope of hazard impact according to hazard characteristics; The danger zone determination module is used to determine and mark the danger zone according to the danger impact range.

7. The intelligent building dynamic display system based on AI video recognition and digital twins according to claim 6, characterized in that: Intelligent building dynamic display model acquisition module, including: The operating platform acquisition submodule is used to obtain the operating platform of the digital twin model; The element conversion submodule is used to convert real-time video data into UI elements by running the platform's onboard controls; The real-time display submodule is used to display the UI elements in real time at the top layer of the virtual scene viewport of the running platform.

8. A method for dynamic display of intelligent buildings based on AI video recognition and digital twins, characterized in that: include: Step 1: Train the video recognition model; Step 2: Collect real-time video data; Step 3: Obtain video recognition requirements, and identify dangerous events based on the video recognition model, real-time video data and video recognition requirements; Step 4: Based on the digital twin technology and real-time video data, a dynamic display model of the intelligent building is constructed, and the dangerous areas corresponding to the dangerous events are marked in the dynamic display model of the intelligent building; The collecting of real-time video data comprises: Identify areas of concern in the target building; Obtain security information of security equipment associated with the area of ​​interest; Analyze the security information of security equipment and obtain real-time video data; The determining of the focus area of ​​the target building includes: Determine the building partition of the target building and obtain the regional attribute label of the building partition; Determine a display object of the display content, and try to obtain a first management area attribute tag of the display object; If the acquisition attempt is successful, determining the area of ​​interest based on the first management area attribute tag and the area attribute tag; If the attempt to obtain fails, obtain the management attribute extraction record of the display object; Extracting records according to management attributes and generating a second management area attribute tag; The concerned area is determined according to the second management area attribute label and the area attribute label.

Images