Group tower operation monitoring method and monitoring system based on digital twinning

Abstract

The invention relates to a digital twinning-based group tower operation monitoring method and monitoring system. The method comprises the following steps: determining a motion stage of a monitored object according to a moving speed of an associated object; according to the current position, determining a detection end needing cooperative collection in a digital twinning-based tower group model; acquiring local three-dimensional point cloud data of the monitored object and local three-dimensional point cloud data of the detection end participating in collaborative acquisition; modeling a monitored object by using the obtained local three-dimensional point cloud data; continuously acquiring a space motion range of the space model of the monitored object; and the interference condition of the space motion range is judged in the digital twinning-based group tower model. According to the digital twinning-based group tower operation monitoring method and monitoring system disclosed by the invention, the types and the space motion ranges of the hoisting objects are determined through combination of the video data and the three-dimensional point cloud data, so that the interference conditions among the hoisting objects are determined, and the safe operation of the group tower is ensured.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a tower group operation monitoring method and monitoring system based on digital twins. Background Art

[0002] With the increase in the proportion of prefabricated buildings and frame buildings, the number of tower cranes used on construction sites has also begun to rise, accompanied by intensive and cross-operations, which has led to a significant increase in the probability of safety accidents.

[0003] To solve this problem, the current solution is to determine the location of the tower crane in the early stage of deployment and reserve a safe distance in the height direction. This method can only ensure basic safety, but the working range of the tower crane still overlaps and needs to be monitored. In addition, various sensors (speed sensors, acceleration sensors, gyroscopes, etc.) are used to collect data, calculate the working status of the tower crane, and simulate it in the virtual space to avoid safety accidents.

[0004] It should also be noted that the shape of the hoisted objects of the tower crane is uncertain, and its posture in the air cannot be fixed. There is a certain probability of mid-air collision if the operation is improper. Ways to solve this problem include separate operations and adding special safety supervisors, but without exception, they will lead to reduced work efficiency. In addition, there is a certain probability of misjudgment when safety supervisors observe and judge the moving objects in the air. Summary of the invention

[0005] The present application provides a tower group operation monitoring method and monitoring system based on digital twins, which determines the type and spatial motion range of the hoisted objects by combining video data with three-dimensional point cloud data, and then determines the interference between the hoisted objects, so as to ensure the safe operation of the tower group.

[0006] The above-mentioned purpose of the present application is achieved through the following technical solutions:

[0007] In a first aspect, the present application provides a method for monitoring the operation of a group of towers based on digital twins, comprising:

[0008] Obtain the moving speed of the associated object and the moving speed of the monitored object within the coverage area;

[0009] Determine the motion phase of the monitored object according to the moving speed of the associated object, and the motion phase includes a starting phase, a stabilizing phase, and a braking phase;

[0010] Determine the detection end that needs to cooperate with collaborative collection in the digital twin-based tower model according to the current location;

[0011] Acquire local 3D point cloud data of the monitored object and local 3D point cloud data of the detection end participating in the collaborative collection;

[0012] Modeling the monitored object using the obtained local three-dimensional point cloud data to obtain a spatial model of the monitored object;

[0013] Continuously obtain the spatial motion range of the spatial model of the monitored object;

[0014] Determine the interference of spatial motion range in a tower cluster model based on digital twins.

[0015] In a possible implementation manner of the first aspect, the method further includes:

[0016] When acquiring local three-dimensional point cloud data of the monitored object, image data of the monitored object is acquired simultaneously;

[0017] Put the image data and the local 3D point cloud data into the same coordinate system and adjust their positions so that the area corresponding to the local 3D point cloud data is located inside the image data;

[0018] Use local 3D point cloud data to fill in the depth data in the image data;

[0019] Use data overlay to fill in the depth data of the area where the depth data is missing in the image data;

[0020] A plurality of local space models belonging to the monitored object are obtained.

[0021] In a possible implementation manner of the first aspect, obtaining the spatial model of the monitored object includes:

[0022] Search for similar models in the model library based on the local space model;

[0023] Calculate the similarity between the close model and the local space model;

[0024] When the similarity meets the requirements, the reference model is used to fill in the vacant areas between adjacent local spatial models to obtain the spatial model of the monitored object;

[0025] Among them, a plurality of local space models are used to construct a boundary reference graph, and the boundary reference graph is used to search for similar models in a model library.

[0026] In a possible implementation manner of the first aspect, calculating the similarity between the close model and the local space model includes:

[0027] Placing similar models inside the area surrounded by multiple local space models;

[0028] The adjacent models are driven to rotate and adjust their proportions so that the volume of the area between the adjacent models and the adjacent local space models is minimized.

[0029] In a possible implementation manner of the first aspect, using a data overlay method to fill in the depth data of an area in the image data where the depth data is missing includes:

[0030] Obtain image data and local 3D point cloud data corresponding to two time points in the time series, the number of image data is two, and the number of local 3D point cloud data is two;

[0031] Determine the rotation axis based on the spatial posture comparison of the two image data;

[0032] Determining a correspondence between the two image data according to the rotation axis;

[0033] The local 3D point cloud data is superimposed on the subsequent image data in the time series.

[0034] In a possible implementation manner of the first aspect, continuously acquiring the spatial motion range of the spatial model of the monitored object includes:

[0035] Selecting a plurality of location points on a spatial model of a monitored object and constructing a reference network using the location points;

[0036] Continuously acquire reference networks in time series;

[0037] Compare the deformation of two adjacent reference networks in the time series and obtain the rotation amount according to the deformation;

[0038] The rotation quantity is used to drive the spatial model to rotate in the tower group model based on the digital twin to obtain the spatial motion range of the spatial model.

[0039] In a possible implementation manner of the first aspect, the method further includes:

[0040] On a plane perpendicular to the moving direction of the monitored object, when the projection area of ​​the monitored object increases, updating the projection of the monitored object, wherein the updating includes assigning the increased portion of the projection area to the projection;

[0041] Generate a virtual running trajectory of the monitored object according to the current movement parameters of the monitored object;

[0042] Generate the spatial motion range of the spatial model using the projection of the monitored object and the virtual running trajectory of the monitored object;

[0043] When the projection of the monitored object is updated, the spatial motion range of the spatial model is updated synchronously.

[0044] In a second aspect, the present application provides a group tower operation monitoring device based on digital twins, comprising:

[0045] A first data acquisition unit, used to acquire a moving speed of an associated object and a moving speed of a monitored object within a coverage area;

[0046] A state determination unit, used to determine the motion phase of the monitored object according to the moving speed of the associated object, the motion phase including the starting phase, the stabilization phase and the braking phase;

[0047] The collaborative unit is used to determine the detection end that needs to cooperate with collaborative collection in the digital twin-based tower group model according to the current position;

[0048] A second data acquisition unit is used to acquire local three-dimensional point cloud data of the monitored object and local three-dimensional point cloud data of the detection end participating in the collaborative acquisition;

[0049] A modeling unit, used to model the monitored object using the obtained local three-dimensional point cloud data to obtain a spatial model of the monitored object;

[0050] A third data acquisition unit, used for continuously acquiring the spatial motion range of the spatial model of the monitored object;

[0051] The interference check unit is used to determine the interference of the spatial motion range in the tower cluster model based on digital twins.

[0052] In a third aspect, the present application provides a group tower operation monitoring system based on digital twins, the system comprising:

[0053] one or more memories for storing instructions; and

[0054] One or more processors, used to call and run the instructions from the memory to execute the method as described in the first aspect and any possible implementation of the first aspect.

[0055] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium comprises:

[0056] Program, when the program is executed by a processor, the method described in the first aspect and any possible implementation of the first aspect is executed.

[0057] In a fifth aspect, the present application provides a computer program product, comprising program instructions. When the program instructions are executed by a computing device, the method described in the first aspect and any possible implementation of the first aspect is executed.

[0058] In a sixth aspect, the present application provides a chip system, which includes a processor for implementing the functions involved in the above aspects, for example, generating, receiving, sending, or processing the data and / or information involved in the above methods.

[0059] The chip system may be composed of chips, or may include chips and other discrete devices.

[0060] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wired or wireless means, or the processor and the memory can also be coupled on the same device. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a schematic flowchart of the steps of a group tower operation monitoring method based on digital twins provided in this application.

[0062] Figure 2 It is a deployment diagram of the hardware system involved in this application provided by this application.

[0063] Figure 3 This is a schematic diagram of the principle of binocular vision distance calculation provided by this application.

[0064] Figure 4 This is a comparative schematic diagram provided by the present application when the rotation axis is different.

[0065] Figure 5 This is a schematic diagram of a process for densifying sparse point cloud data provided by the present application.

[0066] Figure 6 It is a schematic diagram of the principle of a repair space model provided by this application.

[0067] Figure 7 This is a schematic diagram of the principle of calculating the rotation amount provided by this application.

[0068] Figure 8 This is another schematic diagram of the principle of calculating the rotation amount provided by this application. DETAILED DESCRIPTION

[0069] The technical solution in this application is further described in detail below in conjunction with the accompanying drawings.

[0070] This application discloses a method for monitoring the operation of a group of towers based on digital twins. Figure 1 In some examples, the tower group operation monitoring method based on digital twin disclosed in this application includes the following steps:

[0071] S101, obtaining the moving speed of the associated object and the moving speed of the monitored object within the coverage area;

[0072] S102, determining the motion stage of the monitored object according to the moving speed of the associated object, where the motion stage includes a starting stage, a stabilizing stage, and a braking stage;

[0073] S103, determining the detection end that needs to cooperate with the collaborative collection in the digital twin-based tower group model according to the current position;

[0074] S104, obtaining local three-dimensional point cloud data of the monitored object and local three-dimensional point cloud data of the detection terminals participating in the collaborative collection;

[0075] S105, using the obtained local three-dimensional point cloud data to model the monitored object to obtain a spatial model of the monitored object;

[0076] S106, continuously acquiring the spatial motion range of the spatial model of the monitored object;

[0077] S107, determining the interference situation of the spatial motion range in the tower group model based on the digital twin.

[0078] First, the hardware system involved in this application is introduced. The hardware system involved in this application consists of two parts, namely the monitoring terminal installed on the tower crane and the analysis server. Please refer to Figure 2 ,The dotted line in the figure represents the current coverage of the monitoring end, which is ,deployed with image sensors and lidars. The monitoring end collects information through ,image sensors and lidars, and then sends it to the analysis server for ,analysis.

[0079] For the monitoring end, the information collected by the image sensor is used to continuously track the hook part of the tower crane, and the tracking range includes the circumferential direction and the vertical direction. At the same time, the monitoring end is also equipped with a circular track, which can rotate in the circumferential direction without blind spots.

[0080] In step S101, it is necessary to first obtain the moving speed of the associated object and the moving speed of the monitored object within the coverage area. The associated object here refers to the tower crane, and the monitored object within the coverage area refers to the cargo lifted by the tower crane.

[0081] In step S102, the motion stage of the monitored object is determined according to the moving speed of the associated object. There are three motion stages, namely, the motion stage includes the starting stage, the stabilization stage and the braking stage. The purpose of determining the motion stage of the monitored object is to judge the stability of the monitored object.

[0082] It should be understood that when the motion state of the monitored object changes, the state of the monitored object needs to be changed (switch between the motion state and the static state). In this process, the state of the monitored object is prone to instability. This part of the content is further introduced in conjunction with the subsequent content.

[0083] In step S103, the detection end that needs to cooperate with collaborative collection will be determined in the digital twin-based tower group model according to the current position. The detection end that cooperates with collaborative collection is used to achieve full coverage collection of the monitored object. Because the collection range of a monitoring end is limited, it is impossible to achieve full coverage of the monitored object, and there is a natural data missing area.

[0084] Generally speaking, three detection terminals need to be mobilized to collect full coverage of a monitored object.

[0085] In step S104, local three-dimensional point cloud data of the monitored object and local three-dimensional point cloud data of the detection end participating in collaborative collection are obtained. These local three-dimensional point cloud data are used to model the monitored object to obtain a spatial model of the monitored object, which is the content of step S105.

[0086] After obtaining the spatial model of the monitored object, the spatial motion range of the monitored object can be obtained in combination with the spatial model of the monitored object. At this time, if the spatial motion range of the spatial model of the monitored object is continuously obtained, the interference between the towers can be determined, which is the content of step S106 and step S107.

[0087] It should be noted that in this application, the interference of the spatial motion range will be judged in the digital twin-based tower group model. The digital twin tower group model refers to modeling the tower group in three-dimensional space and dynamically simulating the working process of the tower group.

[0088] The present application simultaneously adds the hoisted objects (monitored objects) of the tower groups during the simulation process, and also tracks the size and movement of the hoisted objects (monitored objects) to determine the interference between the hoisted objects (monitored objects) and the surrounding environment in real time. This method can effectively avoid the occurrence of mid-air collision accidents.

[0089] Here, combined with the movement stages (starting stage, stabilization stage and braking stage), during the starting stage and the braking stage, the spatial movement range of the spatial model of the monitored object needs to be expanded. Generally, the expansion coefficient is 1.1-1.2. The expansion can be interpreted as expanding the size of the spatial model of the monitored object. The purpose of the expansion is to reserve a sufficient safety distance.

[0090] In addition, accurate judgment of the movement stage can also enable the selection of appropriate solutions during early warning. For example:

[0091] When both monitoring objects are in a stable stage, the distance between them can be increased;

[0092] When only one of the two monitoring objects is in the stable stage and the other is in the starting stage or braking stage, the monitoring object in the stable stage can be processed by slowing down or translating;

[0093] When two monitoring objects are both in the starting phase or the braking phase, one of them can be paused to handle it.

[0094] In some examples, the following is added:

[0095] S201, when acquiring local three-dimensional point cloud data of the monitored object, synchronously acquire image data of the monitored object;

[0096] S202, placing the image data and the local three-dimensional point cloud data into the same coordinate system and adjusting the positions so that the area corresponding to the local three-dimensional point cloud data is located inside the image data;

[0097] S203, using the local three-dimensional point cloud data to fill the depth data in the image data;

[0098] S204, using a data overlay method to fill in the depth data of the area where the depth data is missing in the image data;

[0099] S205, obtaining a plurality of local space models belonging to the monitored object.

[0100] The content in step S201 to step S205 is to expand the coverage of the depth data based on the combination of image data and local three-dimensional point cloud data.

[0101] It should be understood that the distance can be calculated to a certain extent using image data, for example, using a binocular vision algorithm. Figure 3 As shown, there is a difference in the imaging of the two cameras, and the distance S can be calculated based on the difference, but a high-precision image needs to be provided, and a complex algorithm needs to be matched. In particular, the accuracy of the algorithm decreases with the increase of distance. There is also the problem that only an approximate reference distance can be obtained for large objects.

[0102] The advantage of three-dimensional point cloud data is that it can obtain point-to-point distance, but it requires the laser radar to provide a sufficient number of beams. When the number of beams provided by the laser radar is limited, the distance information obtained will be limited.

[0103] In order to solve the above problems, the present application uses a combination of image data and local three-dimensional point cloud data to obtain more depth data. The specific method is to use data overlay to fill in the depth data of the area where the image data lacks depth data.

[0104] The method of using data overlay to fill in the depth data in the area where the image data lacks depth data is as follows:

[0105] Obtain image data and local 3D point cloud data corresponding to two time points in the time series, the number of image data is two, and the number of local 3D point cloud data is two;

[0106] Determine the rotation axis based on the spatial posture comparison of the two image data;

[0107] Determining a correspondence between the two image data according to the rotation axis;

[0108] The local 3D point cloud data is superimposed on the subsequent image data in the time series.

[0109] Specifically, it is first necessary to obtain image data and local three-dimensional point cloud data corresponding to two time points in the time series. The two time points have a clear order of precedence, and each time point includes an image data and a local three-dimensional point cloud data.

[0110] Then, the rotation axis is determined based on the spatial posture comparison of the two image data. The specific method is to first establish a rotation axis to be determined based on the angle of the lifting belt (wire rope) or in the vertical direction, and then move the rotation axis to be determined in a direction perpendicular to the rotation axis to be determined according to a fixed step distance and rotate one of the image data.

[0111] like Figure 4 As shown, when the position of the rotation axis is different, the obtained image data is also different. The difference is reflected in the rotation position. However, when the rotation angle is the same, the swing distance (S1, S2) of the upper end is different.

[0112] The purpose of the above method is to enable the two image data to overlap. When the two image data overlap, the rotation axis to be determined at this time is used as the rotation axis, and then the correspondence between the two image data is determined according to the rotation axis, and then the two local three-dimensional point cloud data are placed on the same image data.

[0113] In this way, multiple local 3D point cloud data can be superimposed to achieve the purpose of densifying sparse point cloud data, such as Figure 5 shown.

[0114] In some examples, the spatial model of the monitored object is obtained using the following method:

[0115] S301, searching for a similar model in a model library according to the local space model;

[0116] S302, calculating the similarity between the close model and the local space model;

[0117] S303, when the similarity meets the requirement, the reference model is used to fill in the vacant areas between the adjacent local spatial models to obtain the spatial model of the monitored object;

[0118] Among them, a plurality of local space models are used to construct a boundary reference graph, and the boundary reference graph is used to search for similar models in a model library.

[0119] Because in the actual process, the technical solution provided by this application still cannot obtain all the surface information of the monitored object because there is a natural visual blind spot. In order to solve this problem, in the content of step S301 to step S303, firstly, a similar model is searched in the model library, and then the similarity between the similar model and the local space model is calculated.

[0120] When the similarity meets the requirements, the reference model is used to complete the vacant areas between adjacent local spatial models to obtain the spatial model of the monitored object. That is, the spatial model of the monitored object consists of two parts: actual acquisition and model completion, such as Figure 6 shown.

[0121] The specific process of finding similar models in the model library is:

[0122] The model in the model library is enlarged or reduced and placed inside the local space model to minimize the gap between the two or even make them fit together. Models that meet this condition are all similar models.

[0123] Of course, it may be impossible to find a similar model at this time. There are two reasons for this. One is that the number of models in the model library is limited, and the other is that illegal operations occurred during the lifting process. Both of the above methods will issue reminders, and the staff will confirm the actual situation and choose to supplement the model or correct the lifting behavior.

[0124] The way to calculate the similarity between the close model and the local space model is:

[0125] Placing similar models inside the area surrounded by multiple local space models;

[0126] The adjacent models are driven to rotate and adjust their proportions so that the volume of the area between the adjacent models and the adjacent local space models is minimized.

[0127] At this time, one or more similar models may be obtained, and the model with the largest volume is selected as the similar model.

[0128] In some possible implementations, multiple local space models are used to construct a boundary reference graphic and the boundary reference graphic is used to find similar models in the model library. The purpose of using the boundary reference graphic is to reduce the amount of calculation, but this method needs to be limited to simple structures. If a similar model cannot be found, it is necessary to use the method described in the above content.

[0129] In some examples, the method of continuously obtaining the spatial motion range of the spatial model of the monitored object is:

[0130] S401, selecting a plurality of location points on a spatial model of a monitoring object and using the location points to construct a reference network;

[0131] S402, continuously acquiring a reference network in a time series;

[0132] S403, comparing the deformation conditions of two adjacent reference networks in the time series and obtaining the rotation amount according to the deformation conditions;

[0133] S404, using the rotation quantity to drive the spatial model to rotate in the tower group model based on the digital twin, to obtain the spatial motion range of the spatial model.

[0134] In steps S401 to S404, multiple position points are selected on the spatial model of the monitored object and a reference network is constructed using the position points. At this time, the multiple position points have spatial characteristics, so the rotation amount can be conveniently obtained by comparing the deformation of two adjacent reference networks in the time series. The rotation amount is then used to drive the spatial model to rotate in the tower group model based on the digital twin to obtain the spatial motion range of the spatial model.

[0135] The rotation amount is obtained by moving the two reference networks to the same position, and then determining that there are multiple sets of overlapping position points or multiple sets of corresponding position points are close to each other, and then generating the rotation amount based on the overlapping position points.

[0136] See also Figure 7 , when there are multiple sets of overlapping position points, use the overlapping position points to generate a straight line, and then select multiple sets of corresponding position points to calculate the rotation amount ( Figure 7 D) in the above equation and then take the average value;

[0137] See also Figure 8 , when there are multiple sets of corresponding position points with close distances, use the minimum distance method to generate a straight line (the minimum distances from the position points to the straight line are the same or close), and then select multiple sets of corresponding position points to calculate the rotation amount ( Figure 8 D) and then take the average value.

[0138] Furthermore, the following contents are added:

[0139] On a plane perpendicular to the moving direction of the monitored object, when the projection area of ​​the monitored object increases, updating the projection of the monitored object, wherein the updating includes assigning the increased portion of the projection area to the projection;

[0140] Generate a virtual running trajectory of the monitored object according to the current movement parameters of the monitored object;

[0141] Generate the spatial motion range of the spatial model using the projection of the monitored object and the virtual running trajectory of the monitored object;

[0142] When the projection of the monitored object is updated, the spatial motion range of the spatial model is updated synchronously.

[0143] The purpose of adding content is to monitor the dynamic projection area of ​​the object and realize the dynamic update of the virtual running trajectory of the object, so as to obtain the accurate spatial motion range of the spatial model.

[0144] The present application also provides a group tower operation monitoring device based on digital twin, comprising:

[0145] A first data acquisition unit, used to acquire a moving speed of an associated object and a moving speed of a monitored object within a coverage area;

[0146] A state determination unit, used to determine the motion phase of the monitored object according to the moving speed of the associated object, the motion phase including the starting phase, the stabilization phase and the braking phase;

[0147] The collaborative unit is used to determine the detection end that needs to cooperate with collaborative collection in the digital twin-based tower group model according to the current position;

[0148] A second data acquisition unit is used to acquire local three-dimensional point cloud data of the monitored object and local three-dimensional point cloud data of the detection end participating in the collaborative acquisition;

[0149] A modeling unit, used to model the monitored object using the obtained local three-dimensional point cloud data to obtain a spatial model of the monitored object;

[0150] A third data acquisition unit, used for continuously acquiring the spatial motion range of the spatial model of the monitored object;

[0151] The interference check unit is used to determine the interference of the spatial motion range in the tower cluster model based on digital twins.

[0152] Furthermore, it also includes:

[0153] A second data acquisition unit is used to simultaneously acquire image data of the monitored object when acquiring local three-dimensional point cloud data of the monitored object;

[0154] A first data processing unit is used to put the image data and the local three-dimensional point cloud data into the same coordinate system and adjust the positions so that the area corresponding to the local three-dimensional point cloud data is located inside the image data;

[0155] A second data processing unit, used to fill the depth data in the image data using the local three-dimensional point cloud data;

[0156] A third data processing unit, configured to fill in the depth data of the area where the depth data is missing in the image data by using a data superposition method;

[0157] The first result unit is used to obtain multiple local space models belonging to the monitored object.

[0158] Furthermore, it also includes:

[0159] A model search unit is used to search for similar models in the model library according to the local space model;

[0160] A model calculation unit, used to calculate the similarity between the close model and the local space model;

[0161] A model patching unit is used to fill in the missing areas between adjacent local spatial models using a reference model when the similarity meets the requirements, so as to obtain a spatial model of the monitored object;

[0162] Among them, a plurality of local space models are used to construct a boundary reference graph, and the boundary reference graph is used to search for similar models in a model library.

[0163] Furthermore, it also includes:

[0164] A first processing unit is used to place similar models inside the area surrounded by multiple local space models;

[0165] The second processing unit is used to drive the similar model to rotate and adjust the scale so that the volume of the area between the similar model and the adjacent local space model is minimized.

[0166] Furthermore, it also includes:

[0167] A third data acquisition unit is used to acquire image data and local three-dimensional point cloud data corresponding to two time points in the time series, the number of image data is two, and the number of local three-dimensional point cloud data is two;

[0168] A third processing unit, configured to determine a rotation axis according to a spatial posture comparison of the two image data;

[0169] a fourth processing unit, configured to determine a correspondence between two image data according to a rotation axis;

[0170] The fifth processing unit is used to superimpose the local three-dimensional point cloud data onto the next image data in the time series.

[0171] Furthermore, it also includes:

[0172] A reference network construction unit, used for selecting a plurality of position points on a spatial model of a monitoring object and using the position points to construct a reference network;

[0173] A reference network acquisition unit, used for continuously acquiring a reference network in a time series;

[0174] A reference network comparison unit is used to compare the deformation of two adjacent reference networks in the time series and obtain the rotation amount according to the deformation;

[0175] The second result unit is used to use the rotation quantity to drive the spatial model to rotate in the tower group model based on the digital twin, so as to obtain the spatial motion range of the spatial model.

[0176] Furthermore, it also includes:

[0177] A projection updating unit, used for updating the projection of the monitored object on a plane perpendicular to the moving direction of the monitored object when the projection area of ​​the monitored object increases, wherein the updating includes assigning the increased portion of the projection area to the projection;

[0178] A trajectory generating unit, used for generating a virtual running trajectory of the monitored object according to the current movement parameters of the monitored object;

[0179] A range simulation unit, for generating a spatial motion range of a spatial model using a projection of a monitored object and a virtual running trajectory of the monitored object;

[0180] When the projection of the monitored object is updated, the spatial motion range of the spatial model is updated synchronously.

[0181] In one example, the unit in any of the above devices can be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0182] For another example, when the units in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0183] Various objects such as various messages / information / equipment / network elements / systems / devices / actions / operations / processes / concepts that may appear in this application are named. It can be understood that these specific names do not constitute a limitation on the relevant objects. The names assigned may change with factors such as scenarios, contexts or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from the functions and technical effects embodied / executed in the technical scheme.

[0184] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0185] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0186] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0188] It should also be understood that in various embodiments of the present application, the first, second, etc. are only used to indicate that multiple objects are different. For example, the first time window and the second time window are only used to indicate different time windows. They should not have any impact on the time window itself, and the first, second, etc. mentioned above should not impose any limitations on the embodiments of the present application.

[0189] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0190] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a computer-readable storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0191] The present application also provides a group tower operation monitoring system based on digital twins, the system comprising:

[0192] one or more memories for storing instructions; and

[0193] One or more processors are used to call and run the instructions from the memory to execute the method as described above.

[0194] The present application also provides a computer program product, which includes instructions. When the instructions are executed, the tower group operation monitoring system (terminal equipment and network equipment) performs the operations of the tower group operation monitoring system (terminal equipment and network equipment) corresponding to the above method.

[0195] The present application also provides a chip system, which includes a processor for implementing the functions involved in the above content, such as generating, receiving, sending, or processing the data and / or information involved in the above method.

[0196] The chip system may be composed of chips, or may include chips and other discrete devices.

[0197] The processor mentioned in any of the above places can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for executing programs for controlling the above-mentioned feedback information transmission method.

[0198] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wire or wireless means to support the chip system to implement various functions in the above embodiments. Alternatively, the processor and the memory can also be coupled on the same device.

[0199] Optionally, the computer instructions are stored in a memory.

[0200] Optionally, the memory is a storage unit within the chip, such as a register, a cache, etc. The memory can also be a storage unit within the terminal located outside the chip, such as a ROM or other types of static storage devices that can store static information and instructions, RAM, etc.

[0201] It can be understood that the memory in the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0202] The nonvolatile memory may be a ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory.

[0203] The volatile memory may be a RAM, which is used as an external cache. There are many different types of RAM, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory.

[0204] The embodiments of this specific implementation method are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Therefore, all equivalent changes made based on the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A tower group operation monitoring method based on digital twins, characterized in that: include: Obtain the moving speed of the associated object and the moving speed of the monitored object within the coverage area; Determine the motion phase of the monitored object according to the moving speed of the associated object, and the motion phase includes a starting phase, a stabilizing phase, and a braking phase; Determine the detection end that needs to cooperate with collaborative collection in the digital twin-based tower model according to the current location; Acquire local 3D point cloud data of the monitored object and local 3D point cloud data of the detection end participating in the collaborative collection; Modeling the monitored object using the obtained local three-dimensional point cloud data to obtain a spatial model of the monitored object; Continuously obtain the spatial motion range of the spatial model of the monitored object; Determine the interference of spatial motion range in a tower cluster model based on digital twins.

2. The tower group operation monitoring method based on digital twin according to claim 1 is characterized in that: Also includes: When acquiring local three-dimensional point cloud data of the monitored object, image data of the monitored object is acquired simultaneously; Put the image data and the local 3D point cloud data into the same coordinate system and adjust their positions so that the area corresponding to the local 3D point cloud data is located inside the image data; Use local 3D point cloud data to fill in the depth data in the image data; Use data overlay to fill in the depth data of the area where the depth data is missing in the image data; A plurality of local space models belonging to the monitored object are obtained.

3. The tower group operation monitoring method based on digital twin according to claim 2 is characterized in that: The spatial model of the monitored object includes: Search for similar models in the model library based on the local space model; Calculate the similarity between the close model and the local space model; When the similarity meets the requirements, the reference model is used to fill in the vacant areas between adjacent local spatial models to obtain the spatial model of the monitored object; Among them, a plurality of local space models are used to construct a boundary reference graph, and the boundary reference graph is used to search for similar models in a model library.

4. The method for monitoring tower group operation based on digital twin according to claim 3 is characterized in that: Calculating the similarity between the close model and the local space model includes: Placing similar models inside the area surrounded by multiple local space models; The adjacent models are driven to rotate and adjust their proportions so that the volume of the area between the adjacent models and the adjacent local space models is minimized.

5. The method for monitoring tower group operation based on digital twin according to claim 2 is characterized in that: Using data overlay to fill in the depth data of the area where the depth data is missing in the image data includes: Obtain image data and local 3D point cloud data corresponding to two time points in the time series, the number of image data is two, and the number of local 3D point cloud data is two; Determine the rotation axis based on the spatial posture comparison of the two image data; Determining a correspondence between the two image data according to the rotation axis; The local 3D point cloud data is superimposed on the subsequent image data in the time series.

6. The method for monitoring tower group operation based on digital twin according to claim 1, characterized in that: The spatial motion range of the spatial model of the monitored object is continuously obtained including: Selecting a plurality of location points on a spatial model of a monitored object and constructing a reference network using the location points; Continuously obtain reference networks in time series; Compare the deformation of two adjacent reference networks in the time series and obtain the rotation amount according to the deformation; The rotation quantity is used to drive the spatial model to rotate in the tower group model based on the digital twin to obtain the spatial motion range of the spatial model.

7. The method for monitoring tower group operation based on digital twin according to claim 6 is characterized in that: Also includes: On a plane perpendicular to the moving direction of the monitored object, when the projection area of ​​the monitored object increases, updating the projection of the monitored object, wherein the updating includes assigning the increased portion of the projection area to the projection; Generate a virtual running trajectory of the monitored object according to the current movement parameters of the monitored object; Generate the spatial motion range of the spatial model using the projection of the monitored object and the virtual running trajectory of the monitored object; When the projection of the monitored object is updated, the spatial motion range of the spatial model is updated synchronously.

8. A tower group operation monitoring device based on digital twins, characterized in that: include: A first data acquisition unit, used to acquire a moving speed of an associated object and a moving speed of a monitored object within a coverage area; A state determination unit, used to determine the motion phase of the monitored object according to the moving speed of the associated object, the motion phase including a starting phase, a stabilization phase and a braking phase; The collaborative unit is used to determine the detection end that needs to cooperate with collaborative collection in the digital twin-based tower group model according to the current position; A second data acquisition unit is used to acquire local three-dimensional point cloud data of the monitored object and local three-dimensional point cloud data of the detection end participating in the collaborative acquisition; A modeling unit, used to model the monitored object using the obtained local three-dimensional point cloud data to obtain a spatial model of the monitored object; A third data acquisition unit, used for continuously acquiring the spatial motion range of the spatial model of the monitored object; The interference check unit is used to determine the interference of the spatial motion range in the tower cluster model based on digital twins.

9. A tower group operation monitoring system based on digital twins, characterized in that: The system comprises: one or more memories for storing instructions; and One or more processors, configured to call and execute the instructions from the memory to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer readable storage medium comprises: The program, when the program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images