Dangerous article storage management method and system based on digital twinning and application of dangerous article storage management method and system

By constructing a digital twin model and using real-time monitoring technology, the safety issues caused by environmental factors in the storage management of dangerous goods at port terminals have been resolved, achieving efficient anomaly response and vehicle scheduling control, and improving the safety management level of port terminals.

CN118966966APending Publication Date: 2024-11-15JIANGSU PORT GRP CONTAINER CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410988493.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing technologies are insufficient for effectively monitoring and preventing hazardous materials accidents caused by environmental factors in port terminal hazardous materials storage management. The lack of comprehensive cross-monitoring methods leads to inadequate safety management levels.

Method used

A hazardous materials storage management system based on digital twins is constructed. By establishing a digital twin model of the site, environmental parameters and target feature data are monitored in real time, image monitoring data is generated, data of objects of interest are extracted, and abnormal conditions are visualized in the digital twin model to realize vehicle scheduling and control.

Benefits of technology

It improves the safety and visualization capabilities of dangerous goods storage management at port terminals, enables rapid response to abnormal situations, provides reliable route planning and early warning, and ensures the safety and efficiency of dangerous goods storage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966966A_ABST
    Figure CN118966966A_ABST
Patent Text Reader

Abstract

The invention discloses a dangerous goods warehouse management method and system based on digital twinning and application thereof.According to the scheme, after a site digital twinning model is constructed, a carrier, a living body and dangerous goods in a site serve as concerned objects, and after image monitoring is conducted on a storage area where dangerous goods are located, the dangerous goods are stored in the storage area; according to the method, focus object data about carriers, living bodies and dangerous goods in a site are extracted from generated image monitoring data and are associated to a site digital twinborn model, so that moving objects, fixed objects and terrains in a storage area can be well and visually displayed, background management personnel can clearly and intuitively know site conditions, and the management efficiency is improved. On this basis, according to the scheme, the field environment data and the target feature data are further associated into the field digital twinborn model, so that when the dangerous goods or the field environment is abnormal, the dangerous goods or the field environment can be visually displayed in the field monitoring model based on digital twinborn more quickly and intuitively.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of digital twin technology, warehouse management technology and Internet of Things information technology, and in particular to a dangerous goods warehouse management method, system and application thereof based on digital twin. Background Art

[0002] Port terminals are important places for the distribution of goods. The goods distributed through port terminals include not only processed raw materials but also finished products. With the development of international trade and the growth of port container throughput, the transportation and operation volume of dangerous goods containers are also showing a rapid growth trend. Due to the dangerous characteristics of dangerous goods containers such as flammable, explosive, toxic and corrosive, higher requirements are placed on the safety management and environmental protection of port terminals.

[0003] Currently, most of the published literature on cargo safety management at ports and terminals achieves safety management through cross-patrol by personnel and dual monitoring by cameras on site. However, for some hazardous chemical raw materials, the factors that induce safety risks are not limited to external factors (such as impact on containers or cargoes themselves, combustion, leakage, and leaks caused by heat transfer of high-temperature objects, etc.). Hazardous goods accidents caused by environmental factors such as humidity and temperature in the storage environment combined with the characteristics of the hazardous goods themselves are not uncommon. Therefore, for the storage management of hazardous goods, comprehensive cross-monitoring management is very necessary.

[0004] Based on this, how to prevent, effectively guard against, effectively sense feedback and reduce the occurrence of dangerous goods container accidents and improve the safety management level of port terminals are research topics and projects with very positive and practical significance. Summary of the invention

[0005] In view of this, the purpose of the present invention is to propose a dangerous goods warehouse management method, system and application based on digital twins, which are reliable in implementation, flexible in application and have high feedback response efficiency.

[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0007] A dangerous goods storage management method based on digital twins, comprising:

[0008] Establish a spatial reference coordinate system, collect information on the terrain and fixed objects within the preset range of the storage area where the dangerous goods are stored, and build a digital twin model of the storage site;

[0009] Collect environmental parameters at multiple locations in the storage area where dangerous goods are stored at a preset frequency to generate site environmental data;

[0010] Collect characteristic data of target objects in the storage area where dangerous goods are located at a preset time frequency to generate target characteristic data;

[0011] Conduct image monitoring of the storage area where dangerous goods are stored, generate image monitoring data, and then extract data on objects of concern about vehicles, living objects and dangerous goods in the site from the image monitoring data;

[0012] The site environment data, target feature data, and object of interest data are mapped to the site digital twin model to obtain a digital twin-based on-site monitoring model, which is then visualized;

[0013] Based on one or more of the site environment data, target feature data, and object of interest data, determine whether there are abnormal conditions in the storage area where the dangerous goods are located. When there are abnormal conditions, output the abnormal information and associate it with the on-site monitoring model for visual display.

[0014] As a possible implementation method, further, the dangerous goods described in this solution are placed in a container;

[0015] The cargo stacking area and the walking passage area in the storage area are both marked, and when collecting terrain information, the location and specification information of the cargo stacking area and the walking passage area are also collected accordingly;

[0016] The environmental parameters include one or more of the temperature, humidity, gas composition, wind speed, and wind direction corresponding to multiple locations in the storage area where the dangerous goods are located;

[0017] The multiple locations include at least the upper space area, the middle space area and at least two side areas of the storage area where the dangerous goods are located, wherein each area has more than one collection point, and the collection points in the same area are spaced more than 3m apart.

[0018] As a preferred implementation option, preferably, the target object in this solution is the dangerous goods themselves or a container containing dangerous goods;

[0019] The characteristic data collection of the target object includes one or more of the following:

[0020] (1) Monitor the temperature of the target object through a thermal imaging device to obtain temperature distribution data of the target object;

[0021] (2) performing a morphological scan on the target object by means of a contour scanning device to obtain contour data of the target object;

[0022] (3) Monitor the area where the target object is located through smoke sensing equipment to obtain smoke perception data;

[0023] (4) Monitor the area where the target object is located through a gas sensor to obtain gas composition perception data;

[0024] (5) Monitor the area where the target object is located through a humidity sensor to obtain humidity perception data;

[0025] The target characteristic data includes one or more of temperature distribution data, contour data, smoke perception data, gas composition perception data, and humidity perception data.

[0026] As a preferred implementation option, preferably, this solution performs image monitoring on the storage area where dangerous goods are located, generates image monitoring data, and then extracts data on objects of concern about vehicles, living objects and dangerous goods in the site from the image monitoring data, including:

[0027] A01. Perform image monitoring of the storage area where dangerous goods are stored and generate image monitoring data, where the image monitoring area at least covers the cargo storage area and the walking passage area of ​​the storage area where dangerous goods are stored, as well as the entrance and exit of the storage area;

[0028] A02. Obtain image monitoring data, extract frames according to the time series, and extract no less than 3 image frames from the image monitoring data per second to obtain a number of image frames arranged in a time series;

[0029] A03, locating, extracting and identifying elements in the image frame, and obtaining positioning data and identification results corresponding to the elements in the image frame;

[0030] A04. Determine the vehicles, living beings and dangerous goods in the image frame based on the recognition results, classify them, extract them in combination with the positioning data, and then aggregate the extracted local images, as well as their corresponding positioning data and recognition results, to obtain the data of the objects of interest regarding the vehicles, living beings and dangerous goods in the venue.

[0031] As a better implementation option, preferably, this solution will visualize the location where the site environment data is collected and the location of the corresponding object in the target feature data within the site when visually displaying the on-site monitoring model based on the digital twin.

[0032] As a preferred implementation option, preferably, this solution determines whether there is an abnormal situation in the storage area where the dangerous goods are located based on one or more of the site environment data, target feature data, and object of concern data, including the following parallel steps:

[0033] B01. Data judgment

[0034] (1) comparing at least one of the temperature, humidity, gas composition, and wind speed corresponding to the sampling location in the site environment data with a corresponding preset threshold value according to the site environment data to obtain a comparison result, and then determining whether there is an abnormality in the site environment data according to the comparison result to generate site monitoring information;

[0035] or,

[0036] According to the site environment data, at least one of the temperature, humidity, gas composition, and wind speed corresponding to the sampling position in the site environment data is compared with the corresponding preset threshold value, and then a comprehensive weight is calculated according to the comparison result, and whether there is an abnormality in the site environment data is determined according to the comprehensive weight to generate site monitoring information;

[0037] (2) judging whether the temperature distribution data, contour data, smoke sensing data, gas composition sensing data and / or humidity sensing data of the target object meet preset requirements based on the target characteristic data, and then determining whether the target object is abnormal based on the judgment result, and generating target object monitoring information;

[0038] (3) Based on the data of the object of interest, determine whether the location of the vehicle, living person and / or dangerous goods in the venue is abnormal, or whether the vehicle and / or living person meets the conditions for entering the venue, and generate monitoring information of the object of interest.

[0039] B02. Obtain site monitoring information, target object monitoring information and object of interest monitoring information. When it indicates an abnormality, output the abnormal information and associate it with the on-site monitoring model for visual display.

[0040] Based on the above, the present invention also provides a method for dispatching and controlling vehicles in a port hazardous goods storage area, which includes the above-mentioned method for managing hazardous goods storage based on digital twins; it also includes:

[0041] Initiate route planning request information to the server through the vehicle terminal associated with the vehicle, wherein the route planning request information includes communication docking information, current location information, target location information and work content information of the corresponding vehicle, as well as vehicle specification information and vehicle work parameter information;

[0042] The server responds to the route planning request information initiated by the vehicle terminal, determines the current position and target position of the vehicle according to the request information, and plans the route of the vehicle corresponding to the request information in combination with the data of the object of interest, and generates route planning information. At the same time, the right of way of the vehicle is defined according to the work content information, and the right of way priority information is generated;

[0043] Sending route planning information and right-of-way priority information to the vehicle terminal or vehicle corresponding to the route planning request information;

[0044] The location information of the vehicle in the storage area is collected in real time, and then combined with the route planning information corresponding to the vehicle, it is determined whether the vehicle deviates from its corresponding route planning information and a route judgment result is generated;

[0045] Obtain the route judgment result, and when it deviates from the route planning information, generate and output a prompt message.

[0046] As a preferred implementation option, preferably, this solution also includes:

[0047] Obtain the data of the object of interest, determine the distribution density of vehicles and / or living things in the area where the vehicle passes when it moves according to the route planning information, and adjust the thresholds corresponding to the vehicle's travel speed and acceleration based on the distribution density.

[0048] As a preferred implementation option, preferably, there are a plurality of vehicles in the storage area of ​​the present solution; the vehicle scheduling control method further includes:

[0049] The real-time position and speed of all vehicles in the storage area are monitored, and combined with the route planning information corresponding to the vehicle, it is determined whether different vehicles will cause mobile interference when moving at the future time point T1. When there is mobile interference, the route of the low priority vehicle is re-planned and updated, or its operation is suspended, to avoid mobile interference according to the road right priority information corresponding to the vehicle; when the priority level corresponding to the road right priority information is consistent, one of the vehicles is randomly determined to re-plan the route and update it, or suspend its operation to avoid mobile interference.

[0050] Based on the above, this solution also provides a dangerous goods warehouse management system based on digital twins, which includes:

[0051] The basic information collection unit is used to collect information about the terrain and fixed objects within a preset range of the storage area where the dangerous goods are stored, and generate basic data;

[0052] A server, which is deployed with a digital twin unit, wherein the digital twin unit is used to establish a spatial reference coordinate system and construct a site digital twin model of the storage site based on basic data;

[0053] An environmental parameter collection unit is used to collect environmental parameters at multiple locations in the storage area where dangerous goods are stored at a preset time frequency to generate site environmental data;

[0054] The target object monitoring unit is used to collect characteristic data of the target objects in the storage area where the dangerous goods are located at a preset time frequency to generate target characteristic data;

[0055] The image monitoring unit is deployed in the storage area and performs image monitoring on the storage area where the dangerous goods are located to generate image monitoring data;

[0056] An image processing unit, used to extract data on objects of interest such as vehicles, living objects and dangerous goods in the site from the image monitoring data;

[0057] The data association unit is used to associate and map the site environment data, target feature data, and object of interest data into the site digital twin model, obtain a site monitoring model based on the digital twin, and then visualize it;

[0058] A data judgment unit is used to judge whether there is an abnormal condition in the storage area where the dangerous goods are stored based on one or more of the site environment data, target feature data, and attention object data. When an abnormal condition exists, the abnormal information is output and associated with the site monitoring model for visual display;

[0059] A visualization unit for displaying the site digital twin model and / or the on-site monitoring model;

[0060] A storage unit is deployed in the server, and is used to store basic data, site environment data, target feature data, image monitoring data and object of interest data.

[0061] By adopting the above-mentioned technical scheme, the present invention has the following beneficial effects compared with the prior art: after constructing the digital twin model of the site, the present scheme takes the vehicles, living objects and dangerous goods in the site as the objects of concern, and after performing image monitoring on the storage area where the dangerous goods are located, the data of the objects of concern about the vehicles, living objects and dangerous goods in the site are extracted from the generated image monitoring data, and the data are associated with the digital twin model of the site, so that the moving objects and fixed objects in the storage area, as well as the terrain can be better visualized, so that the back-end management personnel can clearly and intuitively understand the on-site conditions. On this basis, the present scheme further associates the site environment data and target feature data (data concentrated on dangerous goods and / or objects in their close range after monitoring and collection) with the digital twin model of the site, so that when abnormalities occur in dangerous goods or the site environment, they can be more quickly and intuitively visualized in the on-site monitoring model based on the digital twin.

[0062] The warehouse management method of this solution can also be applied to the dispatching control of vehicles in the storage area. Based on the on-site monitoring model, various monitoring data are linked to provide reliable and safe route planning, early warning and adaptive adjustment of control parameters for the movement of vehicles in the field, providing practical and reliable protection for the transshipment of goods in hazardous goods storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 This is a brief implementation flow chart of this program management method;

[0065] Figure 2 It is one of the brief schematic diagrams for establishing a field monitoring model for visual display in this solution management method;

[0066] Figure 3 This is the second brief schematic diagram of establishing a field monitoring model for visual display in this solution management method;

[0067] Figure 4 This is the third brief schematic diagram of establishing a field monitoring model for visual display in this solution management method;

[0068] Figure 5 It is one of the brief schematic diagrams showing abnormal information when establishing a field monitoring model for visual display in this solution management method;

[0069] Figure 6 This is the second simplified schematic diagram showing abnormal information when establishing a field monitoring model for visual display in the solution management method;

[0070] Figure 7 This is the third simplified schematic diagram showing abnormal information when establishing a field monitoring model for visual display in the solution management method;

[0071] Figure 8 It is a brief schematic diagram of the outline positioning of the living body in the field when establishing the field monitoring model for visual display in the management method of this solution;

[0072] Fig. 9 It is a brief implementation flow diagram of the vehicle dispatching control method of this scheme;

[0073] Fig.10 It is a schematic diagram of possible path interference when vehicle A and the vehicle move in the field in the vehicle scheduling control method of this scheme;

[0074] Fig.11 It is a schematic diagram of the path adjustment after the vehicle A and the vehicle A move in the field in the vehicle scheduling control method of this scheme, where path interference may occur;

[0075] Fig.12 This is a schematic diagram of the unit module connection of this solution management system. DETAILED DESCRIPTION

[0076] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0077] like Figure 1 As shown, this embodiment provides a dangerous goods storage management method based on digital twins, which includes:

[0078] S01. Establish a spatial reference coordinate system, collect information on the terrain and fixed objects within the preset range of the storage area where the dangerous goods are stored, and build a digital twin model of the storage site (reference Figure 2 shown);

[0079] S02. Collect environmental parameters at multiple locations in the storage area where dangerous goods are stored at a preset time frequency to generate site environmental data;

[0080] S03, collecting characteristic data of target objects in the storage area where dangerous goods are located at a preset time frequency to generate target characteristic data;

[0081] S04, performing image monitoring of the storage area where the dangerous goods are located, generating image monitoring data, and then extracting data on objects of interest regarding the vehicles, living objects, and dangerous goods in the site from the image monitoring data;

[0082] S05. Associate and map the site environment data, target feature data, and object of interest data into the site digital twin model to obtain a site monitoring model based on the digital twin, and then visualize it (refer to Figure 3 or Figure 4 shown);

[0083] S06. Based on one or more of the site environment data, target feature data, and object of interest data, determine whether there is an abnormal situation in the storage area where the dangerous goods are located. When an abnormal situation exists, output the abnormal information and associate it with the on-site monitoring model for visual display (reference Figures 5 to 7 one of them).

[0084] After constructing a digital twin model of the site, this solution focuses on the vehicles, living things and dangerous goods in the site. After image monitoring of the storage area where the dangerous goods are located, the data on the objects of interest about the vehicles, living things and dangerous goods in the site are extracted from the generated image monitoring data, and associated with the digital twin model of the site, so that the moving objects and fixed objects, as well as the terrain in the storage area can be better visualized, so that the back-end management personnel can clearly and intuitively understand the on-site conditions. On this basis, this solution further associates the site environment data and target feature data (data concentrated on dangerous goods and / or objects in their close range after monitoring and collection) with the digital twin model of the site, so that when abnormalities occur in dangerous goods or the site environment, they can be more quickly and intuitively visualized in the on-site monitoring model based on the digital twin.

[0085] As a possible implementation method, further, the dangerous goods described in this solution are placed in a container, wherein: Figure 7 It shows the situation when dangerous goods are placed in a container and a fire or local high heat occurs.

[0086] In order to improve the precision of the digital twin and facilitate the monitoring of public areas within the site, the cargo stacking area and walking channel area in the storage area described in this plan are marked. When collecting terrain information, the location and specification information of the cargo stacking area and the walking channel area are also collected accordingly.

[0087] In terms of monitoring of environmental parameters, the environmental parameters described in this plan include one or more of the temperature, humidity, gas composition, wind speed, and wind direction corresponding to multiple locations in the storage area where the dangerous goods are located.

[0088] In order to improve the reliability of obtaining environmental parameters and avoid the back-end management personnel's misjudgment of the overall situation in the field due to local conditions, in this solution, multiple locations include at least the upper space area, the middle space area and at least two side areas of the storage area where the dangerous goods are located, wherein each area has more than one collection point, and the interval between the collection points in the same area is greater than 3m.

[0089] In addition, this solution can also be further connected to the Internet to obtain weather information and issue early warnings for the corresponding weather at the storage site (such as Figure 6 shown).

[0090] Since this solution is aimed at the management of dangerous goods, as a better implementation option, preferably, the target object described in this solution is the dangerous goods themselves or the container containing the dangerous goods.

[0091] When collecting feature data of a target object, it may include one or more of the following:

[0092] (1) Monitor the temperature of the target object through a thermal imaging device to obtain temperature distribution data of the target object;

[0093] (2) performing a morphological scan on the target object by means of a contour scanning device to obtain contour data of the target object;

[0094] (3) Monitor the area where the target object is located through smoke sensing equipment to obtain smoke perception data;

[0095] (4) Monitor the area where the target object is located through a gas sensor to obtain gas composition perception data;

[0096] (5) Monitor the area where the target object is located through a humidity sensor to obtain humidity perception data;

[0097] Among them, the target characteristic data includes one or more of temperature distribution data, contour data, smoke perception data, gas composition perception data, and humidity perception data. In order to improve the time consistency of the collected data, this solution also synchronizes the internal clocks of all data acquisition devices or sensors at a preset time frequency (for example, once every 12 hours), and synchronizes them to the same clock source, thereby improving the time correspondence capability of data acquisition.

[0098] In terms of data collection of objects of interest, as a preferred implementation option, preferably, this solution conducts image monitoring of the storage area where dangerous goods are located, generates image monitoring data, and then extracts the data of objects of interest about vehicles, living objects and dangerous goods in the site from the image monitoring data, including:

[0099] A01. Perform image monitoring of the storage area where dangerous goods are stored and generate image monitoring data, where the image monitoring area at least covers the cargo storage area and the walking passage area of ​​the storage area where dangerous goods are stored, as well as the entrance and exit of the storage area;

[0100] A02. Obtain image monitoring data, extract frames according to the time series, and extract no less than 3 image frames from the image monitoring data per second to obtain a number of image frames arranged in a time series;

[0101] A03, locating, extracting and identifying elements in the image frame, and obtaining positioning data and identification results corresponding to the elements in the image frame;

[0102] A04. Determine the vehicles, living beings and dangerous goods in the image frame based on the recognition results, classify them, extract them in combination with the positioning data, and then aggregate the extracted local images, as well as their corresponding positioning data and recognition results, to obtain the data of the objects of interest regarding the vehicles, living beings and dangerous goods in the venue.

[0103] As an implementation example, in this scheme, after the image monitoring data is extracted, the image frame can be further denoised, enhanced and normalized to improve the quality of the image frame, and the positioning of the elements in the image frame can be assisted by an edge detection algorithm, and then combined with a support vector machine SVM or a detection neural network to identify the elements located by the edge detection algorithm.

[0104] As an example, this may include the following:

[0105] 1. Image frame data preprocessing

[0106] The extracted image frames are denoised, enhanced, and normalized to improve their content quality.

[0107] This solution uses a Gaussian filter or a median filter to smooth the image frame so that the noise inside it can be removed to a certain extent. The filtering formula model is defined as follows:

[0108]

[0109] Where G(i, j) is the Gaussian kernel, σ is the standard deviation, which is used to control the width of the function, I(x, y) is the pixel value of the original image, and I"(x, y) is the pixel value of the filtered image.

[0110] The image after filtering and denoising is enhanced by histogram equalization, and its formula model is defined as follows:

[0111]

[0112] Among them, I enhanced (x, y) is the enhanced image, min(I) is the minimum value of the image pixel value, max(I) is the maximum value of the image pixel value, and I(x, y) is the image pixel value after filtering.

[0113] The pixel values ​​of the enhanced image are normalized to the range of [0, 1], and the formula model is defined as follows:

[0114]

[0115] Among them, I normalized(x, y) is the pixel value of the image after pixel value normalization, and I(x, y) is the pixel value of the image after enhancement.

[0116] 2. Object Detection

[0117] This implementation example uses the Canny edge detector to perform edge detection on elements in the image frame to achieve element positioning. The formula model is defined as follows:

[0118] E(x, y) = Canny(I normalized (x, y)

[0119] A sliding window is applied to the image for scanning, and a convolutional neural network is used to classify each window. The formula model is defined as follows:

[0120] P obj =CNN(W(x,y))

[0121] Among them, W(x, y) is the graphic block of the sliding window, P obj is the existence probability of the classified object.

[0122] Non-maximum suppression (NMS) is performed by merging overlapping detection windows: NMS(B,P), where B is the set of bounding boxes and P is the corresponding probability.

[0123] In addition, this solution can also detect and locate the preprocessed image frame data through other forms of deep learning networks or other algorithm tools to obtain element objects.

[0124] 3. Feature extraction

[0125] The detected and located element objects are extracted through scale-invariant feature transform SIFT and accelerated robust feature SURF.

[0126] Scale-invariant feature transform SIFT extracts key points and descriptors, and its formula models are defined as follows:

[0127] D=SIFT(I obj )

[0128] Among them, I obj is the object area, and D is the set of feature descriptors.

[0129] Accelerated robust feature SURF extracts accelerated feature points, and its formula models are defined as follows:

[0130] D=SURF(I obj )

[0131] Among them, D is the feature descriptor set, I obj This is the target area.

[0132] 4. Object Recognition

[0133] The extracted features are classified using machine learning algorithms (including support vector machines (SVMs) or neural networks) to identify objects.

[0134] Among them, the model definition of the support vector machine SVM for classifying the extracted features is as follows:

[0135] y=SVM(D)

[0136] Among them, y is the classification result and D is the feature descriptor set.

[0137] The model definition of the neural network for classifying the extracted features is as follows:

[0138] y=NM(D)

[0139] Among them, y is the classification result and D is the feature descriptor set.

[0140] In this solution, when a neural network is used to detect, extract and classify an object in an image frame to determine whether it is a vehicle, a living object or dangerous goods, a detection neural network can be formed by pre-training it and then applied, for example, Figure 8 The diagram shows a schematic diagram of contour positioning of a living body (human) in an image frame extracted from image monitoring data in a venue.

[0141] In addition, in this scheme, when dangerous goods are taken as the object of concern, the spacing between them in the image frame can be judged to indirectly know whether the stacking density of dangerous goods in the container meets the requirements, or whether the spacing between different dangerous goods in different containers does not meet the safety distance, so as to output the corresponding warning information (reference Figure 5 ).

[0142] In this scheme, the purpose of focusing on living things is that people and animals in the hazardous materials storage area may pose a safety risk because their whereabouts are difficult to be dispatched or controlled by back-end management personnel. For example, unauthorized personnel enter the storage area, or foreign animals enter, causing vehicles in the area to be disturbed during movement.

[0143] Therefore, based on the above, when a living body is identified through image monitoring data, this solution can also track the whereabouts of the living body (human or animal) to provide protection for background management.

[0144] As an example of an implementation scheme, the steps of monitoring the whereabouts of a living body (set as an object) are as follows:

[0145] 1. De-noising, enhancement and normalization of object data, that is, filtering, denoising, enhancement and normalization of image frames extracted from image monitoring data;

[0146] 2. Object detection: detect and locate living objects in the normalized image frame data, and combine the site digital twin model to obtain the coordinates of the living object in the site, and then annotate it so that it has unique definition information in all image monitoring data, such as a unique ID;

[0147] 3. Object tracking: Track the detected and located living objects through Kalman filter or multi-target tracking algorithm.

[0148] Among them, the Kalman filter can track the object by updating its state and measurement through the corresponding information of the detected object in the image frame. Specifically, its formula model includes:

[0149] (1) Status update

[0150] x k|k-1 =Fx k-1|k-1 +Bu k-1

[0151] P k|k-1 =FP k-1|k-1 F T +Q

[0152] Among them, x k|k-1 is the predicted state vector, which indicates the state at time k-1 predicted to the state estimate at time k, x k-1|k-1 is the updated state vector, representing the state estimate at time k-1; F is the state transfer matrix, B is the control input matrix, u k-1 is the control input vector, P k|k-1 is the predicted covariance matrix, P k-1|k-1 is the updated covariance matrix, and Q is the process noise covariance matrix.

[0153] (2) Measurement update

[0154] K k =P k|k-1 H T (HP k|k-1 H T +R) -1

[0155] x k|k =x k|k-1 +K k (z k -Hx k|k-1 )

[0156] Pk|k =(IK k H)P k|k-1

[0157] Among them, K k is the Kalman gain, H is the observation matrix, R is the observation noise covariance matrix, z k is the observation vector and I is the identity matrix.

[0158] 4. Object movement trajectory prediction

[0159] After completing object tracking, you can further use time series models such as long short-term memory networks (LSTMs) for prediction. Taking LSTM as an example, it specifically includes:

[0160] 1. Input sequence

[0161] X t =[x t ,y t , v x , v y , a x , a y ]

[0162] Among them, x t ,y t are the positions of the object in the x and y directions at time t, respectively. x , v y are the speed of the object in the x and y directions at time t, respectively, x , a y are the accelerations of the object in the x and y directions at time t, respectively.

[0163] 2. LSTM update formula

[0164] i t =σ(W i x t +U i h t-1 +b i )

[0165] f t =σ(W f x t +U f h t-1 +b f )

[0166] c t =f t ⊙c t-1 +i t ⊙tanh(W c x t +U ch t-1 +b c )

[0167] o t =σ(W o x t +U o h t-1 +b o )

[0168] h t =o t ⊙tanh(c t )

[0169] Among them, i t is the input gate, f t is the forget gate, c t is the cell state, o t is the output gate, h t is the hidden state, W i , W f , U c , W o are the weight matrices input to each gate; U i , U f , U c , U o is the weight matrix from hidden state to each gate; b i 、b f 、b c 、b o is the bias vector of each gate; σ is the sigmoid activation function, tanh is the hyperbolic tangent activation function, and ⊙ is the element-by-element multiplication;

[0170] 3. Predict future locations

[0171]

[0172] in, It is the position of the predicted object at the future time t+k.

[0173] In order to facilitate the retrieval of intuitive associations, as a better implementation option, preferably, this solution will visualize the location where the site environment data is collected, and the location of the corresponding object in the target feature data within the site when visually displaying the on-site monitoring model based on the digital twin.

[0174] Based on the above, as a preferred implementation option, preferably, this solution determines whether there is an abnormal situation in the storage area where the dangerous goods are located based on one or more of the site environment data, target feature data, and object of interest data, including the following parallel steps:

[0175] B01. Data judgment

[0176] (1) comparing at least one of the temperature, humidity, gas composition, and wind speed corresponding to the sampling location in the site environment data with a corresponding preset threshold value according to the site environment data to obtain a comparison result, and then determining whether there is an abnormality in the site environment data according to the comparison result to generate site monitoring information;

[0177] or,

[0178] According to the site environment data, at least one of the temperature, humidity, gas composition, and wind speed corresponding to the sampling position in the site environment data is compared with the corresponding preset threshold value, and then a comprehensive weight is calculated according to the comparison result, and whether there is an abnormality in the site environment data is determined according to the comprehensive weight to generate site monitoring information;

[0179] (2) judging whether the temperature distribution data, contour data, smoke sensing data, gas composition sensing data and / or humidity sensing data of the target object meet preset requirements based on the target characteristic data, and then determining whether the target object is abnormal based on the judgment result, and generating target object monitoring information;

[0180] (3) Based on the data of the object of interest, determine whether the location of the vehicle, living person and / or dangerous goods in the venue is abnormal, or whether the vehicle and / or living person meets the conditions for entering the venue, and generate monitoring information of the object of interest.

[0181] B02. Obtain site monitoring information, target object monitoring information and object of interest monitoring information. When it indicates an abnormality, output the abnormal information and associate it with the on-site monitoring model for visual display.

[0182] As an example, the process of calculating the weight of site environment data can be as follows:

[0183] (1) Determine whether the temperature, humidity, gas composition, and wind speed corresponding to the sampling location in the site environmental data are greater than the preset value or exceed the preset conditions;

[0184]

[0185] (2) Calculation of comprehensive weight

[0186] A=A1+A2+…A n

[0187] Among them, A nIt is the judgment definition value of temperature, humidity, gas composition and wind speed. For example, when the temperature is greater than the preset threshold of 90 (m) degrees, the weight corresponding to the temperature is 1, otherwise it is 0. When the gas composition contains components that should not exist, its corresponding weight is 1, otherwise it is 0; A is the comprehensive weight value, which is the sum of various parameters. By the amplitude of A being greater than the preset threshold, the abnormal situation can be further known.

[0188] As an example, the monitoring information of the object of interest can be reused as follows:

[0189] (1) When the monitoring information of the object of interest indicates that there is a vehicle in the venue, it can be further verified whether the vehicle is authorized to enter. If not, an abnormal prompt is generated and output.

[0190] (2) When the monitoring information of the object of concern points to the displacement of dangerous goods in the site, it can be further verified whether the dangerous goods are dumped or damaged. If so, an abnormal prompt is generated and output.

[0191] (3) When the monitoring information of the object of interest indicates that there is a living body in the venue, it can be further determined whether the living body is an animal or a human. If it is an animal, an abnormal prompt is generated and output. If it is a human, the facial positioning is further performed, and then the person is verified to see whether the person is authorized to enter. If not, an abnormal prompt is generated and output.

[0192] The abnormal prompts output as above can be further used as references by backstage management personnel, and then generate relevant expulsion information or intervention information for on-site security personnel to carry out security work.

[0193] Combination Fig. 9 As shown, based on the above, this embodiment also provides a method for dispatching and controlling vehicles in a dangerous goods storage area in a port, which includes the dangerous goods storage management method based on digital twins described above; it also includes:

[0194] C01. Initiate route planning request information to the server through the vehicle terminal associated with the vehicle, wherein the route planning request information includes communication docking information, current location information, target location information and work content information of the corresponding vehicle, as well as vehicle specification information and vehicle work parameter information;

[0195] C02. The server responds to the route planning request information initiated by the vehicle terminal, determines the current position and target position of the vehicle according to the request information, and plans the route of the vehicle corresponding to the request information in combination with the data of the object of interest, and generates route planning information. At the same time, the right of way of the vehicle is defined according to the work content information, and the right of way priority information is generated;

[0196] C03. Sending the route planning information and the road right priority information to the vehicle terminal or vehicle corresponding to the route planning request information;

[0197] C04. Real-time collection of the location information of the vehicle in the storage area, and then combined with the route planning information corresponding to the vehicle, to determine whether the vehicle deviates from its corresponding route planning information, and generate a route determination result;

[0198] C05. Obtain the route judgment result, and when it deviates from the route planning information, generate and output a prompt message.

[0199] In order to facilitate the linkage between the on-site situation and the working parameters of the vehicle, as a better implementation option, preferably, this solution also includes:

[0200] Obtain the data of the object of interest, determine the distribution density of vehicles and / or living things in the area where the vehicle passes when it moves according to the route planning information, and adjust the thresholds corresponding to the vehicle's travel speed and acceleration based on the distribution density.

[0201] In this solution, real-time positioning can be performed on the vehicle by deploying a positioning unit, which can be a unit module such as RFID, GPS or UWB.

[0202] In path planning, the Dijistra algorithm can be used to plan the shortest path for the vehicle. At the same time, the A* algorithm and obstacle avoidance algorithm can be combined to optimize the path planned by the Dijistra algorithm when information such as the site terrain and obstacles is known.

[0203] In addition to the above, this solution can also directly assist in route planning by using existing navigation software or equipment.

[0204] In order to avoid interference in movement between different vehicles when they are traveling according to route planning information, as a preferred implementation option, preferably, there are several vehicles in the storage area of ​​the solution; the vehicle scheduling control method further includes:

[0205] Monitor the real-time position and speed of all vehicles in the storage area, and combine the route planning information corresponding to the vehicles to determine whether different vehicles will cause mobile interference when traveling at the future time point T1. Fig.10 As shown, it shows the situation when vehicle A and vehicle B have mobile interference at time point T1. In this case, the route of the vehicle with lower priority can be replanned and updated according to the right of way priority information corresponding to the vehicle (assuming that the priority of vehicle B is disclosed) ( Fig.11It shows that after the route of vehicle B is replanned, at time point T1, the positions of vehicle A and vehicle B in the venue area are different, and at this time the two will not cause mobile interference problems). In addition, this solution is also able to avoid mobile interference by pausing the operation of vehicle B.

[0206] In addition to the above situation, when the priorities corresponding to the road right priority information are consistent, one of the vehicles can be randomly determined to re-plan the route and update it, or its operation can be suspended to avoid mobile interference.

[0207] Combination Fig.12 As shown, based on the above, this solution also provides a dangerous goods warehouse management system based on digital twins, which includes:

[0208] The basic information collection unit is used to collect information about the terrain and fixed objects within a preset range of the storage area where the dangerous goods are stored, and generate basic data;

[0209] A server, which is deployed with a digital twin unit, wherein the digital twin unit is used to establish a spatial reference coordinate system and construct a site digital twin model of the storage site based on basic data;

[0210] An environmental parameter collection unit is used to collect environmental parameters at multiple locations in the storage area where dangerous goods are stored at a preset time frequency to generate site environmental data;

[0211] The target object monitoring unit is used to collect characteristic data of the target objects in the storage area where the dangerous goods are located at a preset time frequency to generate target characteristic data;

[0212] The image monitoring unit is deployed in the storage area and performs image monitoring on the storage area where the dangerous goods are located to generate image monitoring data;

[0213] An image processing unit, used to extract data on objects of interest such as vehicles, living objects and dangerous goods in the site from the image monitoring data;

[0214] The data association unit is used to associate and map the site environment data, target feature data, and object of interest data into the site digital twin model, obtain a site monitoring model based on the digital twin, and then visualize it;

[0215] A data judgment unit is used to judge whether there is an abnormal condition in the storage area where the dangerous goods are stored based on one or more of the site environment data, target feature data, and attention object data. When an abnormal condition exists, the abnormal information is output and associated with the site monitoring model for visual display;

[0216] A visualization unit for displaying the site digital twin model and / or the on-site monitoring model;

[0217] A storage unit is deployed in the server, and is used to store basic data, site environment data, target feature data, image monitoring data and object of interest data.

[0218] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0219] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0220] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A dangerous goods storage management method based on digital twins, characterized in that: It includes: Establish a spatial reference coordinate system, collect information on the terrain and fixed objects within the preset range of the storage area where the dangerous goods are stored, and build a digital twin model of the storage site; Collect environmental parameters at multiple locations in the storage area where dangerous goods are stored at a preset frequency to generate site environmental data; Collect characteristic data of target objects in the storage area where dangerous goods are located at a preset time frequency to generate target characteristic data; Conduct image monitoring of the storage area where dangerous goods are stored, generate image monitoring data, and then extract data on objects of concern about vehicles, living objects and dangerous goods in the site from the image monitoring data; The site environment data, target feature data, and object of interest data are mapped to the site digital twin model to obtain a digital twin-based on-site monitoring model, which is then visualized; Based on one or more of the site environment data, target feature data, and object of interest data, determine whether there are abnormal conditions in the storage area where the dangerous goods are located. When there are abnormal conditions, output the abnormal information and associate it with the on-site monitoring model for visual display.

2. The dangerous goods storage management method based on digital twins according to claim 1 is characterized in that: The dangerous goods are placed in a container; The cargo stacking area and the walking passage area in the storage area are both marked, and when collecting terrain information, the location and specification information of the cargo stacking area and the walking passage area are also collected accordingly; The environmental parameters include one or more of the temperature, humidity, gas composition, wind speed, and wind direction corresponding to multiple locations in the storage area where the dangerous goods are located; The multiple locations include at least the upper space area, the middle space area and at least two side areas of the storage area where the dangerous goods are located, wherein each area has more than one collection point, and the collection points in the same area are spaced more than 3m apart.

3. The dangerous goods storage management method based on digital twins according to claim 2 is characterized in that: The target object is the dangerous goods themselves or a container containing the dangerous goods; The characteristic data collection of the target object includes one or more of the following: (1) Monitor the temperature of the target object through a thermal imaging device to obtain temperature distribution data of the target object; (2) performing a morphological scan on the target object by means of a contour scanning device to obtain contour data of the target object; (3) Monitor the area where the target object is located through smoke sensing equipment to obtain smoke perception data; (4) Monitor the area where the target object is located through a gas sensor to obtain gas composition perception data; (5) Monitor the area where the target object is located through a humidity sensor to obtain humidity perception data; The target characteristic data includes one or more of temperature distribution data, contour data, smoke perception data, gas composition perception data, and humidity perception data.

4. The dangerous goods storage management method based on digital twins according to claim 3 is characterized in that: Perform image monitoring of the storage area where dangerous goods are stored to generate image monitoring data, and then extract data on the objects of concern about the vehicles, living objects and dangerous goods in the site from the image monitoring data, including: A01. Perform image monitoring of the storage area where dangerous goods are stored and generate image monitoring data, where the image monitoring area at least covers the cargo storage area and the walking passage area of ​​the storage area where dangerous goods are stored, as well as the entrance and exit of the storage area; A02. Obtain image monitoring data, extract frames according to the time series, and extract no less than 3 image frames from the image monitoring data per second to obtain a number of image frames arranged in a time series; A03, locating, extracting and identifying elements in the image frame, and obtaining positioning data and identification results corresponding to the elements in the image frame; A04. Determine the vehicles, living beings and dangerous goods in the image frame based on the recognition results, classify them, extract them in combination with the positioning data, and then aggregate the extracted local images, as well as their corresponding positioning data and recognition results, to obtain the data of the objects of interest regarding the vehicles, living beings and dangerous goods in the venue.

5. The dangerous goods storage management method based on digital twins according to claim 4 is characterized in that: When visualizing the digital twin-based on-site monitoring model, the location where the site environment data is collected and the location of the corresponding object in the target feature data within the site are also visualized.

6. The dangerous goods storage management method based on digital twins according to claim 4 or 5, characterized in that: Based on one or more of the site environment data, target characteristic data, and object of concern data, judging whether there is an abnormal situation in the storage area where the dangerous goods are located includes the following parallel steps: B01. Data judgment (1) comparing at least one of the temperature, humidity, gas composition, and wind speed corresponding to the sampling location in the site environment data with a corresponding preset threshold value according to the site environment data to obtain a comparison result, and then determining whether there is an abnormality in the site environment data according to the comparison result to generate site monitoring information; or, According to the site environment data, at least one of the temperature, humidity, gas composition, and wind speed corresponding to the sampling position in the site environment data is compared with the corresponding preset threshold value, and then a comprehensive weight is calculated according to the comparison result, and whether there is an abnormality in the site environment data is determined according to the comprehensive weight to generate site monitoring information; (2) judging whether the temperature distribution data, contour data, smoke sensing data, gas composition sensing data and / or humidity sensing data of the target object meet preset requirements based on the target characteristic data, and then determining whether the target object is abnormal based on the judgment result, and generating target object monitoring information; (3) Based on the data of the object of interest, determine whether the location of the vehicle, living person and / or dangerous goods in the venue is abnormal, or whether the vehicle and / or living person meets the conditions for entering the venue, and generate monitoring information of the object of interest. B02. Obtain site monitoring information, target object monitoring information and object of interest monitoring information. When it indicates an abnormality, output the abnormal information and associate it with the on-site monitoring model for visual display.

7. A method for dispatching and controlling vehicles in a port hazardous goods storage area, characterized in that: It includes the dangerous goods storage management method based on digital twins as described in any one of claims 1 to 6; It also includes: Initiate route planning request information to the server through the vehicle terminal associated with the vehicle, wherein the route planning request information includes communication docking information, current location information, target location information and work content information of the corresponding vehicle, as well as vehicle specification information and vehicle work parameter information; The server responds to the route planning request information initiated by the vehicle terminal, determines the current position and target position of the vehicle according to the request information, and plans the route of the vehicle corresponding to the request information in combination with the data of the object of interest, and generates route planning information. At the same time, the right of way of the vehicle is defined according to the work content information, and the right of way priority information is generated; Sending route planning information and right-of-way priority information to the vehicle terminal or vehicle corresponding to the route planning request information; The location information of the vehicle in the storage area is collected in real time, and then combined with the route planning information corresponding to the vehicle, it is determined whether the vehicle deviates from its corresponding route planning information and a route judgment result is generated; Obtain the route judgment result, and when it deviates from the route planning information, generate and output a prompt message.

8. The method for controlling vehicle dispatching in a port hazardous goods storage area according to claim 7, characterized in that: It also includes: Obtain the data of the object of interest, determine the distribution density of vehicles and / or living things in the area where the vehicle passes when it moves according to the route planning information, and adjust the thresholds corresponding to the vehicle's travel speed and acceleration based on the distribution density.

9. The method for controlling vehicle dispatching in a port hazardous goods storage area according to claim 8, characterized in that: There are several vehicles in the storage area; the vehicle scheduling control method also includes: The real-time position and speed of all vehicles in the storage area are monitored, and combined with the route planning information corresponding to the vehicle, it is determined whether different vehicles will cause mobile interference when moving at the future time point T1. When there is mobile interference, the route of the low priority vehicle is re-planned and updated, or its operation is suspended, to avoid mobile interference according to the road right priority information corresponding to the vehicle; when the priority level corresponding to the road right priority information is consistent, one of the vehicles is randomly determined to re-plan the route and update it, or suspend its operation to avoid mobile interference.

10. The dangerous goods warehouse management system based on digital twin is characterized by: It includes: The basic information collection unit is used to collect information about the terrain and fixed objects within a preset range of the storage area where the dangerous goods are stored, and generate basic data; A server, which is deployed with a digital twin unit, wherein the digital twin unit is used to establish a spatial reference coordinate system and construct a site digital twin model of the storage site based on basic data; An environmental parameter collection unit is used to collect environmental parameters at multiple locations in the storage area where dangerous goods are stored at a preset time frequency to generate site environmental data; The target object monitoring unit is used to collect characteristic data of the target objects in the storage area where the dangerous goods are located at a preset time frequency to generate target characteristic data; The image monitoring unit is deployed in the storage area and performs image monitoring on the storage area where the dangerous goods are located to generate image monitoring data; An image processing unit, used to extract data on objects of interest such as vehicles, living objects and dangerous goods in the site from the image monitoring data; The data association unit is used to associate and map the site environment data, target feature data, and object of interest data into the site digital twin model, obtain a site monitoring model based on the digital twin, and then visualize it; A data judgment unit is used to judge whether there is an abnormal condition in the storage area where the dangerous goods are stored based on one or more of the site environment data, target feature data, and attention object data. When an abnormal condition exists, the abnormal information is output and associated with the site monitoring model for visual display; A visualization unit for displaying the site digital twin model and / or the on-site monitoring model; A storage unit is deployed in the server, and is used to store basic data, site environment data, target feature data, image monitoring data and object of interest data.

Citation Information

Cited By

  • Digital country processing method and system based on digital twinning

    CN119579839A

  • Digital Village Processing Method and System Based on Digital Twin

    CN119579839B