A safety control method and system based on GIS+BIM+U3D
Through GIS+BIM+U3D technology, real-time monitoring and analysis of the movement status of the factory's aerial objects is solved, and the problem of traditional sensors being difficult to monitor the movement of high-temperature molten metals and other items is achieved, and safety is improved.
Patent Information
- Application Number
- CN202411089243.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2044-08-09
AI Technical Summary
During factory production, especially in scenarios such as lifting or air transport of high-temperature molten metals, traditional sensors find it difficult to monitor the movement status of items in real time, resulting in difficulty in early warning of safety hazards and easily lead to accidents.
The safety control method based on GIS+BIM+U3D is adopted to identify and track the speed, acceleration and vibration amplitude of air objects through video analysis, generate a virtual workshop, monitor it in real time and issue early warnings to ensure safety.
Real-time monitoring and early warning of the movement status of air objects is achieved, which improves the safety of the factory production process and reduces the risk of accidents.
Smart Images
Figure CN118608571B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of safe production technology, and in particular to a safety management and control method and management and control system based on GIS+BIM+U3D. Background Art
[0002] The whole process safety management and control of the factory refers to the participation of all employees, full process control, and all-round management during the production process. It changes from the past post-inspection and control to prevention and improvement, and from managing results to managing factors, so as to find out the factors that affect safety issues.
[0003] The maturity of technologies such as high-definition video surveillance, neural network analysis, and big data models has continuously reduced the cost of implementing safety control throughout the factory process, and has also enabled continuous monitoring in some places where sensors cannot be deployed. Taking a steel plant as an example, based on technologies such as real-time dynamic digital twin construction, real-time cloud rendering of 3D models, the Internet of Things, and integrated indoor and outdoor positioning, it is possible to reconstruct the entire production workshop in 3D space and dynamically manage personnel and equipment.
[0004] In actual use, it was found that it is difficult to solve the problem of some hoisting or items that need to be transported in the air by deploying sensors, such as high-temperature molten metal transfer and item hoisting. However, in such scenarios, once a shaking amplitude exceeds the limit, it can easily cause a safety accident. Summary of the invention
[0005] The present application provides a safety management and control method and system based on GIS+BIM+U3D, which can analyze the speed, acceleration and vibration amplitude of an object in the air during its movement through video analysis, so as to realize the pre-discovery of safety hazards.
[0006] The above-mentioned purpose of the present application is achieved through the following technical solutions:
[0007] In the first aspect, this application provides a safety management and control method based on GIS+BIM+U3D, including:
[0008] Identify moving objects in the image and associate the objects with object models in a model library;
[0009] When the association is successful, the moving object is marked as a tracking object and marked in the virtual workshop generated based on GIS+BIM+U3D;
[0010] Continuously track the tracking object;
[0011] determining characteristic indicators according to an object model associated with the tracked object, the characteristic indicators including velocity, acceleration, and vibration amplitude;
[0012] Acquire characteristic indicators of the tracking object and determine the danger value of the tracking object according to the characteristic indicators of the tracking object;
[0013] According to the danger value of the tracked object, an early warning is issued to the personnel object associated with the tracked object in the virtual workshop.
[0014] In a possible implementation manner of the first aspect, identifying an object in a moving state in an image includes:
[0015] Acquire multiple images in time series, each of which is generated at a different time;
[0016] Overlap multiple images to obtain a state analysis image;
[0017] Separate the part of the state analysis image with the object, and record it as a state analysis sub-image;
[0018] Establishing a detection region on the edge of the state analysis sub-image and generating a detection curve based on the detection region;
[0019] Count the number of peaks or troughs appearing on the detection curve to obtain a reference number;
[0020] When the reference number is the same as the image number, it is determined that the object included in the image is in a moving state.
[0021] In a possible implementation manner of the first aspect, the maximum width of a peak or a trough is calculated, and when the maximum width of a peak or a trough is less than or equal to a set width value, the time interval between two images that are adjacent in the time series is increased.
[0022] In a possible implementation manner of the first aspect, when acquiring the characteristic indication of the tracked object, it is necessary to first clarify the outline of the tracked object. Clarifying the outline of the tracked object includes:
[0023] Obtain edge curves of objects in motion in an image;
[0024] Segment the edge curve to obtain clear edge curve segments and fuzzy edge curve segments;
[0025] The clarity of the blurred edge curve segment is increased by using a super-resolution processing method or the blurred edge curve segment is replaced by a clear edge curve segment in an edge curve of another image in a time series.
[0026] In a possible implementation manner of the first aspect, using a super-resolution processing manner to increase the clarity of the blurred edge curve segment includes:
[0027] According to the color difference on both sides of the fuzzy edge curve segment, a corresponding clear edge curve segment is selected and recorded as a replacement curve segment;
[0028] Establishing a first perpendicular line on the replacement curve segment;
[0029] Constructing a first analysis curve based on the pixel points in the first vertical direction;
[0030] The first analysis curve is transferred into the frequency domain for decomposition to obtain a low-frequency curve group, a medium-frequency curve group and a high-frequency curve group;
[0031] Establishing a second perpendicular line on the fuzzy edge curve segment;
[0032] Constructing a second analysis curve based on the pixel points in the second vertical direction;
[0033] The second curve is transferred into the frequency domain for decomposition to obtain a curve group;
[0034] Filter the curves in the curve group and retain the curves whose frequencies and amplitudes are the same or similar to those in the medium frequency curve group and the high frequency curve group;
[0035] Replaces the blurred edge curve segments with the composite edge curve segments of the preserved curve composite.
[0036] In a possible implementation of the first aspect, when the first analysis curve is transferred to the frequency domain for decomposition, a time domain feature is simultaneously introduced so that the curves in the low-frequency curve group, the medium-frequency curve group, and the high-frequency curve group have both frequency domain features and time domain features;
[0037] When the second curve is transferred to the frequency domain for decomposition, the time domain feature is introduced at the same time, so that the curves in the curve group have both frequency domain features and time domain features.
[0038] In a possible implementation manner of the first aspect, replacing the blurred edge curve segment with a clear edge curve segment in an edge curve in another image in the time series includes:
[0039] Determine the corresponding clear edge curve segments on two images in the time series and calculate the adjustment parameter set;
[0040] Determine the clear edge curve segment at the corresponding position in the edge curve of other images in the time series, and record it as the clear edge curve segment to be adjusted;
[0041] Using the adjustment parameter set to adjust the clear edge curve segment to be adjusted, to obtain an adjusted clear edge curve segment;
[0042] Use the adjusted clear edge curve segment to replace the blurred edge curve segment at the corresponding position;
[0043] When there is an unreplaced portion on the blurred edge curve segment, a super-resolution processing method is used to increase the clarity of the unreplaced portion on the blurred edge curve segment.
[0044] In the second aspect, the present application provides a safety management and control device based on GIS+BIM+U3D, including:
[0045] A recognition and association unit, used for recognizing objects in a moving state in the image and associating the objects with object models in a model library;
[0046] The marking unit is used to mark the moving object as a tracking object and mark it in the virtual workshop generated based on GIS+BIM+U3D when the association is successful;
[0047] A continuous tracking unit, used for continuously tracking a tracking object;
[0048] a characteristic indicator determination unit, configured to determine characteristic indicators according to an object model associated with the tracked object, the characteristic indicators including velocity, acceleration and vibration amplitude;
[0049] A characteristic indication acquisition unit, used for acquiring characteristic indications of the tracked object and determining a danger value of the tracked object according to the characteristic indications of the tracked object;
[0050] The early warning unit is used to issue an early warning to a personnel object associated with the tracking object in the virtual workshop according to the danger value of the tracking object.
[0051] In a third aspect, the present application provides a safety management and control system based on GIS+BIM+U3D, the system comprising:
[0052] one or more memories for storing instructions; and
[0053] One or more processors, used to call and run the instructions from the memory to execute the method as described in the first aspect and any possible implementation of the first aspect.
[0054] In a fourth aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium comprising:
[0055] Program, when the program is executed by a processor, the method described in the first aspect and any possible implementation of the first aspect is executed.
[0056] In a fifth aspect, the present application provides a computer program product, comprising program instructions. When the program instructions are executed by a computing device, the method described in the first aspect and any possible implementation of the first aspect is executed.
[0057] In a sixth aspect, the present application provides a chip system, which includes a processor for implementing the functions involved in the above aspects, for example, generating, receiving, sending, or processing the data and / or information involved in the above methods.
[0058] The chip system may be composed of chips, or may include chips and other discrete devices.
[0059] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wired or wireless means, or the processor and the memory can also be coupled on the same device. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1 This is a schematic flowchart of the steps of a full-process safety management and control method based on GIS+BIM+U3D provided in this application.
[0061] Figure 2 This is a schematic diagram of a process for determining that an object is in a stationary state provided by the present application.
[0062] Figure 3 This is a schematic diagram of a process for determining that an object is in a moving state provided by the present application.
[0063] Figure 4 It is a schematic diagram of the principle of calculating speed provided by this application.
[0064] Figure 5 It is a schematic diagram of the principle of calculating acceleration provided by this application.
[0065] Figure 6 It is a principle schematic diagram of calculating vibration amplitude speed provided by this application.
[0066] Figure 7 It is a schematic diagram of a detection curve provided in this application.
[0067] Figure 8 It is a schematic diagram provided by the present application for establishing multiple detection areas and generating detection curves.
[0068] Fig. 9 This is a schematic diagram of segmenting an edge curve provided by the present application.
[0069] Fig.10 It is a schematic diagram of constructing an analysis curve (a first analysis curve, a second analysis curve) provided in the present application. DETAILED DESCRIPTION
[0070] The technical solution in this application is further described in detail below in conjunction with the accompanying drawings.
[0071] For some examples, see Figure 1, this application discloses a safety management and control method based on GIS+BIM+U3D, including the following steps:
[0072] S101, identifying a moving object in an image and associating the object with an object model in a model library;
[0073] S102, when the association is successful, the object in the moving state is marked as a tracking object and marked in the virtual workshop generated based on GIS+BIM+U3D;
[0074] S103, continuously tracking the tracking object;
[0075] S104, determining characteristic indicators according to an object model associated with the tracked object, the characteristic indicators including velocity, acceleration and vibration amplitude;
[0076] S105, obtaining characteristic indicators of the tracking object and determining the danger value of the tracking object according to the characteristic indicators of the tracking object;
[0077] S106: issuing an early warning to a person object associated with the tracked object in the virtual workshop according to the danger value of the tracked object.
[0078] In conjunction with the related technology, objects inside a workshop are created in a three-dimensional virtual space, including equipment and personnel, etc., and then the working status of the equipment and the movement of personnel are monitored and displayed in the three-dimensional virtual space at the same time.
[0079] To obtain the working status of the device, data can be directly obtained from the control system of the device. For the location and movement status of the personnel, indoor positioning technology (such as UWB) can be used to achieve it. In step S101, the moving objects in the image are identified and associated with the object models in the model library. Here, the moving objects are a broad description, and all the moving contents in the image are within this description.
[0080] Then use the model library to screen the objects in a moving state. The object models in the model library are the objects that need to be analyzed in detail. If the object in a moving state is successfully associated with the object model in the model library, then the object in a moving state will be marked as a tracking object and the tracking object will be continuously tracked, which is the content of step S102 and step S103.
[0081] In some possible implementations, a method used to identify an object in a moving state in an image is to overlap two images generated at different times and then determine the overlapping area. If all areas of the object overlap, it means that the object is stationary. Figure 2 As shown, otherwise it means the object is in a moving state. Figure 2 shown.
[0082] When the above method is difficult to judge, you can choose to increase the number of images, such as using three or four images, or increase the time interval between two generation times.
[0083] The virtual workshop is generated based on GIS+BIM+U3D, and the details are as follows:
[0084] The basic principle of GIS is to digitize objects and form three-dimensional models to reflect the real world through models. BIM (Building Information Modeling) is a method of using digital technology to express the physical and functional characteristics of construction projects. The combination of GIS and BIM can effectively improve the degree of visualization and enhance the ability of spatial analysis and decision support.
[0085] U3D is a development platform with an intuitive user interface and a powerful scripting system. It also supports publishing on multiple platforms. Through U3D, you can easily create a virtual workshop based on GIS+BIM and display it.
[0086] Taking this application as an example, a production workshop can be created in a virtual world generated based on GIS. The location of the production workshop is the same as the actual location. At the same time, equipment and personnel are placed in the production workshop in the virtual world with the help of BIM technology. Here, a variety of data collection methods and sensors are combined to display the operating status of equipment and the movement of personnel.
[0087] The above method can also participate in the safety management process. For example, the technical solution in the present application can issue an early warning to the personnel object associated with the tracking object in the virtual workshop according to the danger value of the tracking object. This method can realize the early warning.
[0088] In step S104, characteristic indicators are determined based on the object model associated with the tracked object. There are three characteristic indicators, namely, speed, acceleration and vibration amplitude. The characteristic indicators are associated with the object model. After the associated object model is determined, the characteristic indicators are automatically determined.
[0089] In step S105 and step S106, characteristic indicators of the tracking object are obtained and the danger value of the tracking object is determined according to the characteristic indicators of the tracking object, and then an early warning is issued to the personnel object associated with the tracking object in the virtual workshop according to the danger value of the tracking object.
[0090] Here, the person associated with the tracking object is generally determined based on the distance. The distance is a set value. Different characteristic indicators of the tracking object correspond to different distance values. The positioning of people indoors is generally accurately positioned using U3D technology.
[0091] For example, the transportation of high-temperature molten metal requires a constant speed and no shaking. For another example, the lifting of objects can have an unstable speed and a certain amount of shaking, but both need to be within the allowable range.
[0092] This is also the reason why the object model and characteristic indication are associated in this application. For different object models, the corresponding characteristic indications (the risk value is bound to the characteristic indication, and the risk value is given simultaneously when the characteristic indication is given) are fixed directly, rather than leaving it to the analysis system to judge.
[0093] The speed is calculated as Figure 4 As shown, S in the figure is the distance.
[0094] The acceleration is calculated as Figure 5 As shown, S1 and S2 in the figure are distances, V1 and V2 are distances, and the acceleration is the difference between V2 and V1, which is the accumulation of acceleration over time.
[0095] The vibration amplitude is calculated as Figure 6 As shown, T in the figure is the maximum distance between corresponding positions when two objects overlap.
[0096] In some examples, the specific method of identifying objects in motion in an image is:
[0097] S201, acquiring multiple images in a time series, where each image is generated at a different time;
[0098] S202, performing overlap processing on a plurality of images to obtain a state analysis image;
[0099] S203, separating the portion of the state analysis image having the object, and recording it as a state analysis sub-image;
[0100] S204, establishing a detection area on the edge of the state analysis sub-image and generating a detection curve based on the detection area;
[0101] S205, counting the number of peaks or troughs appearing on the detection curve to obtain a reference number;
[0102] S206: When the reference number is the same as the image number, it is determined that the object included in the image is in a moving state.
[0103] The contents of step S201 to step S206 provide a specific method for determining the state of an object, which is to obtain a state analysis sub-image by means of overlap processing, and then establish a detection area on the edge of the state analysis sub-image and generate a detection curve based on the detection area.
[0104] For the detection curve, the specific explanation is as follows:
[0105] The state analysis sub-image is composed of pixels arranged in an array. Each pixel has a value. A line segment is created on the detection area. The pixels that touch this line segment are converted into numerical representations to obtain a set of discrete points. The detection curve can be obtained by connecting this set of discrete points with a curve. Figure 7 shown.
[0106] The peaks appearing on the detection curve correspond to the dark-colored areas, and the troughs appearing on the detection curve correspond to the light-colored areas. The peaks and troughs appearing on the detection curve are both located above the X-axis.
[0107] The judgment basis is that when the reference number is the same as the image number, it is determined that the object included in the image is in a moving state.
[0108] In some possible implementations, the maximum width of a peak or a trough is calculated. Figure 7 S MAX When the maximum width of the peak or trough is less than or equal to the set width value, the time interval between two images in the time series is increased. This is mainly due to the fact that there is a certain detection error, which needs to be avoided through technical means.
[0109] For some possible implementations, see Figure 8 When establishing a detection area on the edge of the state analysis sub-image, multiple detection areas will be established at the same time. Each detection area generates a detection curve. Then, a statistical method is used to determine whether the object is in a moving state, and then the results are counted. Finally, the ratio of the number of objects in the moving state is calculated. The formula is:
[0110] The ratio of the number in the moving state = the number in the moving state / the total number of detection curves.
[0111] When the ratio of the number of objects in the moving state is greater than a set value, such as 0.7 or 0.8, the object is considered to be in the moving state.
[0112] In some examples, when obtaining the characteristic indications of the tracked object, it is necessary to first clarify the outline of the tracked object. There are two purposes here. One is to reduce the difficulty of associating the object with the object model in the model library. The other is to obtain more accurate calculation results when calculating the characteristic indications.
[0113] The specific way to clearly track the outline of the object is:
[0114] S301, obtaining an edge curve of an object in a moving state in an image;
[0115] S302, segmenting the edge curve to obtain a clear edge curve segment and a fuzzy edge curve segment. Fig. 9 As shown;
[0116] S303: Use a super-resolution processing method to increase the clarity of the blurred edge curve segment or use a clear edge curve segment in an edge curve of another image in the time series to replace the blurred edge curve segment.
[0117] In some examples, using super-resolution processing to increase the clarity of blurred edge curve segments includes:
[0118] According to the color difference on both sides of the fuzzy edge curve segment, a corresponding clear edge curve segment is selected and recorded as a replacement curve segment;
[0119] Establishing a first perpendicular line on the replacement curve segment;
[0120] Constructing a first analysis curve based on the pixel points in the first vertical direction;
[0121] The first analysis curve is transferred into the frequency domain for decomposition to obtain a low-frequency curve group, a medium-frequency curve group and a high-frequency curve group;
[0122] Establishing a second perpendicular line on the fuzzy edge curve segment;
[0123] Constructing a second analysis curve based on the pixel points in the second vertical direction;
[0124] The second curve is transferred into the frequency domain for decomposition to obtain a curve group;
[0125] Filter the curves in the curve group and retain the curves whose frequencies and amplitudes are the same or similar to those in the medium frequency curve group and the high frequency curve group;
[0126] Replaces the blurred edge curve segments with the composite edge curve segments of the preserved curve composite.
[0127] In the above method, the parameters of the clear edge curve segment are used to adjust the blurred edge curve segment. The clear edge curve segment is selected based on the color difference on both sides of the blurred edge curve segment. The color difference refers to randomly selecting multiple groups of pixel points on the left and right sides of the blurred edge curve segment. Each group includes two pixel points, and the minimum distance between these two pixel points and the blurred edge curve segment is the same.
[0128] Next, the color difference between two pixels in a group is calculated, and then the color difference is averaged to obtain the color difference average value, and then the color difference average value of the clear edge curve segment is calculated. When selecting the corresponding clear edge curve segment, the color difference average value of the clear edge curve segment needs to be the same or similar to the color difference average value of the fuzzy edge curve segment. If there is a difference, it needs to be controlled between 1% and 2%.
[0129] See also Fig.10, then establish a first perpendicular line on the replacement curve segment (with a fixed length and equal lengths on both sides of the replacement curve segment), and then construct a first analysis curve based on the pixel points in the direction of the first perpendicular line. The method of constructing the first analysis curve is the same as the method of constructing the detection curve, which will not be repeated here.
[0130] Then the first analysis curve is transferred into the frequency domain for decomposition to obtain a low-frequency curve group, a medium-frequency curve group and a high-frequency curve group.
[0131] Then, a second perpendicular line is established on the blurred edge curve segment (the length is the same as that of the first perpendicular line, and the lengths on both sides of the replacement curve segment are equal), and a second analysis curve is constructed based on the pixel points in the direction of the second perpendicular line. The method of constructing the second analysis curve is the same as the method of constructing the detection curve, which will not be repeated here.
[0132] After the second curve is transferred to the frequency domain and decomposed to obtain a curve group, the curves in the curve group are screened to retain the curves with the same or similar frequency and amplitude as the curves in the intermediate frequency curve group and the high frequency curve group. The purpose of this step is to remove interference in the curve group and only retain the curves that are the same or similar to the clear edge curve segment.
[0133] For the same or similar frequency and amplitude, it is generally necessary to determine it based on the required calculation accuracy, which is generally controlled within 2%. For example, if two frequencies are 0.98-1.02 times the other frequency, then the two frequencies are considered to be similar and the amplitudes are processed in the same way.
[0134] Furthermore, when the first analysis curve is transferred to the frequency domain for decomposition, the time domain feature is introduced at the same time, so that the curves in the low-frequency curve group, the medium-frequency curve group and the high-frequency curve group have both frequency domain features and time domain features;
[0135] When the second curve is transferred to the frequency domain for decomposition, the time domain feature is introduced at the same time, so that the curves in the curve group have both frequency domain features and time domain features.
[0136] The introduction of time domain features requires the use of wavelet transform. The advantage of wavelet transform is that while obtaining frequency domain features, it can also determine the appearance time and disappearance time of the waveform. In this way, when screening curves in the curve group, a more accurate contour can be obtained after replacing the fuzzy edge curve segments with the synthetic edge curve segments synthesized by the retained curves.
[0137] In some examples, the manner of replacing blurred edge curve segments with sharp edge curve segments in edge curves of other images in the time series is as follows:
[0138] Determine the corresponding clear edge curve segments on two images in the time series and calculate the adjustment parameter set;
[0139] Determine the clear edge curve segment at the corresponding position in the edge curve of other images in the time series, and record it as the clear edge curve segment to be adjusted;
[0140] Using the adjustment parameter set to adjust the clear edge curve segment to be adjusted, to obtain an adjusted clear edge curve segment;
[0141] Use the adjusted clear edge curve segment to replace the blurred edge curve segment at the corresponding position;
[0142] When there is an unreplaced portion on the blurred edge curve segment, a super-resolution processing method is used to increase the clarity of the unreplaced portion on the blurred edge curve segment.
[0143] In this method, an adjustment parameter set is required to correct the clear edge curve segment, because the positions of the corresponding clear edge curve segments on the two images are different, and the fuzzy edge curve segment cannot be directly replaced by the clear edge curve segment in the edge curve of other images in the time series. After determining the clear edge curve segment at the corresponding position in the edge curve of other images in the time series, the adjustment parameter set needs to be used to adjust the clear edge curve segment before replacement.
[0144] The specific implementation is as follows:
[0145] After determining the corresponding clear edge curve segments on the two images in the time series, the fixed parts of the two images (objects that are always stationary or temporarily stationary) are adjusted to overlap, and then the relative positions of the corresponding clear edge curve segments on the two images are determined, and then their movement process is calculated. The movement process includes movement in the X-axis direction, movement in the Y-axis direction, movement in the Z-axis direction, rotation in the X-axis direction, rotation in the Y-axis direction, and rotation in the Z-axis direction.
[0146] The calculation process can be realized with the help of the object model in the model library, driving the object model in the model library to move, and obtaining another clear edge curve segment from one clear edge curve segment. During the process, the movement process of the object model in the model library is recorded (movement in the X-axis direction, movement in the Y-axis direction, movement in the Z-axis direction, rotation in the X-axis direction, rotation in the Y-axis direction, and rotation in the Z-axis direction).
[0147] The adjustment parameter set is used to adjust the clear edge curve segment to be adjusted, and the adjusted clear edge curve segment is obtained, and finally the adjusted clear edge curve segment is used to replace the fuzzy edge curve segment at the corresponding position. Of course, in this process, there may be a situation where there is an unreplaced part of the fuzzy edge curve segment. In this case, the super-resolution processing method is used to increase the clarity of the unreplaced part on the fuzzy edge curve segment.
[0148] This application also provides a safety control device based on GIS+BIM+U3D, including:
[0149] A recognition and association unit, used for recognizing objects in a moving state in the image and associating the objects with object models in a model library;
[0150] The marking unit is used to mark the moving object as a tracking object and mark it in the virtual workshop generated based on GIS+BIM+U3D when the association is successful;
[0151] A continuous tracking unit, used for continuously tracking a tracking object;
[0152] a characteristic indicator determination unit, configured to determine characteristic indicators according to an object model associated with the tracked object, the characteristic indicators including velocity, acceleration and vibration amplitude;
[0153] A characteristic indication acquisition unit, used for acquiring characteristic indications of the tracked object and determining a danger value of the tracked object according to the characteristic indications of the tracked object;
[0154] The early warning unit is used to issue an early warning to a personnel object associated with the tracking object in the virtual workshop according to the danger value of the tracking object.
[0155] Furthermore, it also includes:
[0156] A first acquisition unit is used to acquire multiple images in a time series, and the generation time of each image is different;
[0157] A coincidence processing unit, used for performing coincidence processing on a plurality of images to obtain a state analysis image;
[0158] A separation unit, used for separating a portion of the state analysis image having the object, which is recorded as a state analysis sub-image;
[0159] A detection curve generating unit, used for establishing a detection area on the edge of the state analysis sub-image and generating a detection curve based on the detection area;
[0160] A statistical unit is used to count the number of peaks or troughs appearing on the detection curve to obtain a reference number;
[0161] A state determination unit is used to determine that the object included in the image is in a moving state when the reference number is the same as the image number.
[0162] Furthermore, the maximum width of the peak or the trough is calculated, and when the maximum width of the peak or the trough is less than or equal to the set width value, the time interval between two images in adjacent states in the time series is increased.
[0163] Furthermore, it also includes:
[0164] A second acquisition unit, used for acquiring an edge curve of an object in a moving state in an image;
[0165] A segmentation unit, used for segmenting the edge curve to obtain a clear edge curve segment and a fuzzy edge curve segment;
[0166] The processing unit is used to increase the clarity of the blurred edge curve segment by using a super-resolution processing method or to replace the blurred edge curve segment with a clear edge curve segment in an edge curve in another image in a time series.
[0167] Furthermore, it also includes:
[0168] A selection unit, used for selecting a corresponding clear edge curve segment according to the color difference on both sides of the fuzzy edge curve segment, recorded as a replacement curve segment;
[0169] A first establishing unit, used for establishing a first perpendicular line on the replacement curve segment;
[0170] A second establishing unit, configured to construct a first analysis curve based on pixel points in a first vertical direction;
[0171] A first decomposition unit is used to transfer the first analysis curve into the frequency domain for decomposition to obtain a low-frequency curve group, a medium-frequency curve group and a high-frequency curve group;
[0172] A third establishing unit, used for establishing a second perpendicular line on the fuzzy edge curve segment;
[0173] A fourth establishing unit, configured to construct a second analysis curve based on the pixel points in a second vertical direction;
[0174] A second decomposition unit, used for transferring the second curve into the frequency domain for decomposition to obtain a curve group;
[0175] A screening unit, used for screening the curves in the curve group, and retaining the curves whose frequencies and amplitudes are the same as or similar to those in the intermediate frequency curve group and the high frequency curve group;
[0176] A synthesis unit is used to replace the blurred edge curve segment with a synthesized edge curve segment synthesized by the preserved curve.
[0177] Furthermore, it also includes:
[0178] When the first analysis curve is transferred to the frequency domain for decomposition, the time domain feature is introduced at the same time, so that the curves in the low frequency curve group, the medium frequency curve group and the high frequency curve group have both the frequency domain feature and the time domain feature;
[0179] When the second curve is transferred to the frequency domain for decomposition, the time domain feature is introduced at the same time, so that the curves in the curve group have both frequency domain features and time domain features.
[0180] Furthermore, it also includes:
[0181] A first determining unit, used to determine a set of corresponding clear edge curve segment calculation adjustment parameters on two images in a time series;
[0182] A second determining unit is used to determine a clear edge curve segment at a corresponding position in the edge curve of other images in the time series, and record it as a clear edge curve segment to be adjusted;
[0183] An adjustment unit, configured to adjust the clear edge curve segment to be adjusted using an adjustment parameter set to obtain an adjusted clear edge curve segment;
[0184] A replacement unit, used for replacing the blurred edge curve segment at the corresponding position with the adjusted clear edge curve segment;
[0185] When there is an unreplaced portion on the blurred edge curve segment, a super-resolution processing method is used to increase the clarity of the unreplaced portion on the blurred edge curve segment.
[0186] In one example, the unit in any of the above devices can be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.
[0187] For another example, when the units in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).
[0188] Various objects such as various messages / information / equipment / network elements / systems / devices / actions / operations / processes / concepts that may appear in this application are named. It can be understood that these specific names do not constitute a limitation on the relevant objects. The names assigned may change with factors such as scenarios, contexts or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from the functions and technical effects embodied / executed in the technical scheme.
[0189] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0190] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0191] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0192] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0193] It should also be understood that in various embodiments of the present application, the first, second, etc. are only used to indicate that multiple objects are different. For example, the first time window and the second time window are only used to indicate different time windows. They should not have any impact on the time window itself, and the first, second, etc. mentioned above should not impose any limitations on the embodiments of the present application.
[0194] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.
[0195] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a computer-readable storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0196] The present application also provides a safety management and control system based on GIS+BIM+U3D, the system comprising:
[0197] one or more memories for storing instructions; and
[0198] One or more processors are used to call and run the instructions from the memory to execute the method as described above.
[0199] The present application also provides a computer program product, which includes instructions. When the instructions are executed, the security management and control system (terminal device and network device) based on GIS+BIM+U3D performs the operations of the security management and control system (terminal device and network device) based on GIS+BIM+U3D corresponding to the above-mentioned method.
[0200] The present application also provides a chip system, which includes a processor for implementing the functions involved in the above content, such as generating, receiving, sending, or processing the data and / or information involved in the above method.
[0201] The chip system may be composed of chips, or may include chips and other discrete devices.
[0202] The processor mentioned in any of the above places can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for executing programs for controlling the above-mentioned feedback information transmission method.
[0203] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wire or wireless means to support the chip system to implement various functions in the above embodiments. Alternatively, the processor and the memory can also be coupled on the same device.
[0204] Optionally, the computer instructions are stored in a memory.
[0205] Optionally, the memory is a storage unit within the chip, such as a register, a cache, etc. The memory can also be a storage unit within the terminal located outside the chip, such as a ROM or other types of static storage devices that can store static information and instructions, RAM, etc.
[0206] It can be understood that the memory in the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.
[0207] The nonvolatile memory may be a ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory.
[0208] The volatile memory may be a RAM, which is used as an external cache. There are many different types of RAM, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct memory bus RAM.
[0209] The embodiments of this specific implementation method are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Therefore, all equivalent changes made based on the structure, shape, and principle of the present application should be included in the protection scope of the present application.
Claims
1. A safety management and control method based on geographic information system + building information model + virtual visualization, characterized in that: include: Identify moving objects in the image and associate the objects with object models in a model library; When the association is successful, the object in the moving state is marked as a tracking object and marked in the virtual workshop generated based on geographic information system + building information model + virtual visualization; Continuously track the tracking object; determining characteristic indicators according to an object model associated with the tracked object, the characteristic indicators including velocity, acceleration, and vibration amplitude; Acquire characteristic indicators of the tracking object and determine the danger value of the tracking object according to the characteristic indicators of the tracking object; According to the danger value of the tracking object, an early warning is issued to the personnel object associated with the tracking object in the virtual workshop; Identifying objects in motion in an image includes: Acquire multiple images in time series, each of which is generated at a different time; Overlap multiple images to obtain a state analysis image; Separate the part of the state analysis image with the object, and record it as a state analysis sub-image; Establishing a detection region on the edge of the state analysis sub-image and generating a detection curve based on the detection region; Count the number of peaks or troughs appearing on the detection curve to obtain a reference number; When the reference number is the same as the image number, determining that the object included in the image is in a moving state; Calculate the maximum width of the peak or trough. When the maximum width of the peak or trough is less than or equal to the set width value, increase the time interval between two images that are adjacent in the time series.
2. The safety management and control method based on geographic information system + building information model + virtual visualization according to claim 1 is characterized in that: When obtaining the characteristic indications of the tracking object, you need to first clarify the outline of the tracking object. Clarifying the outline of the tracking object includes: Obtain edge curves of objects in motion in an image; Segment the edge curve to obtain clear edge curve segments and fuzzy edge curve segments; The clarity of the blurred edge curve segment is increased by using a super-resolution processing method or the blurred edge curve segment is replaced by a clear edge curve segment in an edge curve of another image in a time series.
3. The safety management and control method based on geographic information system + building information model + virtual visualization according to claim 2 is characterized in that: The super-resolution processing methods used to increase the clarity of blurred edge curve segments include: According to the color difference on both sides of the fuzzy edge curve segment, a corresponding clear edge curve segment is selected and recorded as a replacement curve segment; Establishing a first perpendicular line on the replacement curve segment; Constructing a first analysis curve based on the pixel points in the first vertical direction; The first analysis curve is transferred into the frequency domain for decomposition to obtain a low-frequency curve group, a medium-frequency curve group and a high-frequency curve group; Establishing a second perpendicular line on the fuzzy edge curve segment; Constructing a second analysis curve based on the pixel points in the second vertical direction; The second analysis curve is transferred into the frequency domain for decomposition to obtain a curve group; Filter the curves in the curve group and retain the curves whose frequencies and amplitudes are the same or similar to those in the medium frequency curve group and the high frequency curve group; Replaces the blurred edge curve segments with the composite edge curve segments of the preserved curve composite.
4. The safety management and control method based on geographic information system + building information model + virtual visualization according to claim 3 is characterized in that: When the first analysis curve is transferred to the frequency domain for decomposition, the time domain feature is introduced at the same time, so that the curves in the low frequency curve group, the medium frequency curve group and the high frequency curve group have both the frequency domain feature and the time domain feature; When the second curve is transferred to the frequency domain for decomposition, the time domain feature is introduced at the same time, so that the curves in the curve group have both frequency domain features and time domain features.
5. The safety management and control method based on geographic information system + building information model + virtual visualization according to claim 2 is characterized in that: Replacing blurred edge curve segments with sharp edge curve segments from edge curves in other images in the time series includes: Determine the corresponding clear edge curve segments on two images in the time series and calculate the adjustment parameter set; Determine the clear edge curve segment at the corresponding position in the edge curve of other images in the time series, and record it as the clear edge curve segment to be adjusted; Using the adjustment parameter set to adjust the clear edge curve segment to be adjusted, to obtain an adjusted clear edge curve segment; Use the adjusted clear edge curve segment to replace the fuzzy edge curve segment at the corresponding position; When there is an unreplaced portion on the blurred edge curve segment, a super-resolution processing method is used to increase the clarity of the unreplaced portion on the blurred edge curve segment.
6. A safety control device based on geographic information system + building information model + virtual visualization, characterized in that: include: A recognition and association unit, used for recognizing objects in a moving state in the image and associating the objects with object models in a model library; A marking unit is used to mark the moving object as a tracking object and mark it in the virtual workshop generated based on geographic information system + building information model + virtual visualization when the association is successful; A continuous tracking unit, used for continuously tracking a tracking object; a characteristic indicator determination unit, configured to determine characteristic indicators according to an object model associated with the tracked object, the characteristic indicators including velocity, acceleration and vibration amplitude; A characteristic indication acquisition unit, used for acquiring characteristic indications of the tracked object and determining a danger value of the tracked object according to the characteristic indications of the tracked object; An early warning unit, used to issue an early warning to a person object associated with the tracked object in the virtual workshop according to the danger value of the tracked object; A first acquisition unit is used to acquire multiple images in a time series, and the generation time of each image is different; A coincidence processing unit, used for performing coincidence processing on a plurality of images to obtain a state analysis image; A separation unit, used for separating a portion of the state analysis image having an object, which is recorded as a state analysis sub-image; A detection curve generating unit, used for establishing a detection area on the edge of the state analysis sub-image and generating a detection curve based on the detection area; A statistical unit is used to count the number of peaks or troughs appearing on the detection curve to obtain a reference number; a state determination unit, configured to determine that the object included in the image is in a moving state when the reference number is the same as the image number; Calculate the maximum width of the peak or trough. When the maximum width of the peak or trough is less than or equal to the set width value, increase the time interval between two images that are adjacent in the time series.
7. A security management and control system based on geographic information system + building information model + virtual visualization, characterized in that: The system comprises: one or more memories for storing instructions; and One or more processors, configured to call and execute the instructions from the memory to perform the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: The computer readable storage medium comprises: The program, when the program is executed by a processor, the method according to any one of claims 1 to 5 is executed.