Object security determination method, device, computer equipment and storage medium

Abstract

The present application relates to the field of positioning technology, and in particular to a method, device, computer equipment and storage medium for determining object safety. The method comprises: obtaining real-time point cloud data obtained by the acquisition device carried by the target object to acquire the target operation scene; performing similar matching between the real-time point cloud data and the global point cloud data of the target operation scene to obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data; determining the transformation information between the real-time point cloud data and the similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset and an orientation offset; determining the position and posture information of the acquisition device based on the transformation information; and determining whether the target object is located in a safe area in the target operation scene based on the position and posture information. The present application can improve positioning accuracy.

Description

Technical Field

[0001] The present application relates to the field of positioning technology, and in particular to a method, device, computer equipment and storage medium for determining object security. Background Art

[0002] With the large-scale construction of infrastructure projects, the construction of infrastructure digital twin systems has gradually increased. In the construction of infrastructure digital twin systems, high-precision location positioning management is a crucial link; through digital twin systems and high-precision positioning technology, managers can monitor the positions of all personnel and vehicles in the site in real time, so as to allocate resources more effectively. In addition, high-precision positioning can promptly detect whether personnel or vehicles have entered a dangerous area, and issue warnings or take emergency measures in a timely manner.

[0003] At present, traditional positioning technologies include Bluetooth positioning, Global Positioning System (GPS) and Ultra Wide Band (UWB) high-precision positioning. The range and area of ​​Bluetooth positioning are relatively narrow, and more Bluetooth devices need to be invested, and the positioning is not accurate; GPS positioning is based on high-altitude WeChat signals, which cannot be used indoors and in infrastructure caves; UWB positioning has active signal positioning devices, and a large number of positioning signal sources need to be installed in the positioning area, and the investment in software and hardware is very large.

[0004] Therefore, there is still a lack of high-accuracy positioning technology suitable for infrastructure digital twin systems, which urgently needs to be improved. Summary of the invention

[0005] Based on this, it is necessary to provide an object safety determination method, device, computer equipment and storage medium that can improve positioning accuracy in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for determining object security, the method comprising:

[0007] Acquire real-time point cloud data obtained by the acquisition device carried by the target object to collect the target operation scene;

[0008] Perform similarity matching between the real-time point cloud data and the global point cloud data of the target operation scene, and obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data;

[0009] Determine transformation information between the real-time point cloud data and the similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset, and an orientation offset;

[0010] Determine the position and posture information of the acquisition device according to the transformation information;

[0011] Based on the position information, determine whether the target object is located in a safe area in the target operation scene.

[0012] In one embodiment, the real-time point cloud data is similarly matched with the global point cloud data of the target operation scene to obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data, including:

[0013] Obtain global point cloud data of the target operation scene;

[0014] Generate a first feature point histogram according to the real-time point cloud data, and generate a second feature point histogram according to the global point cloud data;

[0015] Comparing the first feature point histogram with the second feature point histogram to obtain a target area in the second feature point histogram that overlaps with the first feature point histogram;

[0016] The point cloud data corresponding to the target area is determined as similar point cloud data in the global point cloud data that matches the real-time point cloud data.

[0017] In one embodiment, determining transformation information between real-time point cloud data and similar point cloud data includes:

[0018] Parsing the real-time point cloud data to obtain at least one first object;

[0019] Parsing the similar point cloud data to obtain at least one second object;

[0020] Determine at least one similar object pair according to the similarity relationship between each first object and each second object; wherein each similar object pair includes a first object and a second object;

[0021] determining transformation information between a first object and a second object in each pair of similar objects;

[0022] According to the transformation information corresponding to each similar object pair, the transformation information between the real-time point cloud data and the similar point cloud data is determined.

[0023] In one embodiment, determining the transformation information between the real-time point cloud data and the similar point cloud data according to the transformation information corresponding to each similar object pair includes:

[0024] The mean of the transformation information corresponding to each pair of similar objects is determined as the transformation information between the real-time point cloud data and the similar point cloud data.

[0025] In one embodiment, determining the position and posture information of the acquisition device according to the transformation information includes:

[0026] The position and posture information of the acquisition device is determined according to the transformation information and the position and posture information of the acquisition device corresponding to the similar point cloud data.

[0027] In one embodiment, determining whether the target object is located in a safe area in a target operation scene according to the posture information includes:

[0028] Obtain the location information of the electronic fence within the target operation scene;

[0029] Compare the position information in the posture information with the position information of the electronic fence to determine whether the target object is located in a safe area in the target operation scene.

[0030] In one embodiment, the method further comprises:

[0031] If the acquisition device is outside the safety area in the target operation scene, the image device corresponding to the acquisition device is started;

[0032] Controlling the image acquisition device to obtain image information at the location of the acquisition device;

[0033] If the image information contains obstacle information, the prompt information is output.

[0034] In a second aspect, the present application further provides an object safety determination device, the device comprising:

[0035] An acquisition module is used to acquire real-time point cloud data obtained by a collection device carried by a target object to collect a target operation scene;

[0036] A matching module is used to perform similarity matching between the real-time point cloud data and the global point cloud data of the target operation scene, and obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data;

[0037] A transformation module, used to determine transformation information between real-time point cloud data and similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset, and an orientation offset;

[0038] A posture determination module, used to determine the posture information of the acquisition device according to the transformation information;

[0039] The safety positioning module is used to determine whether the target object is located in a safe area in the target operation scene based on the posture information.

[0040] In a third aspect, the present application further provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0041] Acquire real-time point cloud data obtained by the acquisition device carried by the target object to collect the target operation scene;

[0042] Perform similarity matching between the real-time point cloud data and the global point cloud data of the target operation scene, and obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data;

[0043] Determine transformation information between the real-time point cloud data and the similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset, and an orientation offset;

[0044] Determine the position and posture information of the acquisition device according to the transformation information;

[0045] Based on the position information, determine whether the target object is located in a safe area in the target operation scene.

[0046] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0047] Acquire real-time point cloud data obtained by the acquisition device carried by the target object to collect the target operation scene;

[0048] Perform similarity matching between the real-time point cloud data and the global point cloud data of the target operation scene, and obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data;

[0049] Determine transformation information between the real-time point cloud data and the similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset, and an orientation offset;

[0050] Determine the position and posture information of the acquisition device according to the transformation information;

[0051] Based on the position information, determine whether the target object is located in a safe area in the target operation scene.

[0052] The above-mentioned object safety determination method, device, computer equipment and storage medium, this application compares real-time point cloud data and global point cloud data. This method can achieve high-precision positioning. The point cloud data itself contains rich three-dimensional spatial information, so the comparison result can accurately reflect the position and posture of the acquisition device (i.e., the target object) in the global coordinate system; by relying only on the comparison of point cloud data, this method simplifies the positioning process. Compared with traditional positioning methods that require the combination of multiple sensor data and complex algorithms, this method only needs to process point cloud data, thereby reducing the complexity of positioning; further, since the point cloud data comparison method does not rely on specific hardware or environmental features, it can be applied to a variety of different scenarios, including indoors, outdoors, complex environments, etc.; in addition, this application does not require the installation of additional hardware positioning devices, such as GPS receivers, beacons or other sensors, and is more suitable for infrastructure digital twin systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of a flow chart of a method for determining object security in one embodiment;

[0054] Figure 2 A schematic diagram of a process for determining similar point cloud data in one embodiment;

[0055] Figure 3 A schematic diagram of a process for determining transformation information between real-time point cloud data and similar point cloud data in one embodiment;

[0056] Figure 4 A schematic diagram of a process for determining whether a collection device is located in a safe area in one embodiment;

[0057] Figure 5 is a structural block diagram of an object safety determination device in one embodiment;

[0058] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] The object safety determination method provided in the embodiment of the present application is applied to a controller. In terms of hardware equipment, the controller, a micro radar point cloud acquisition device, a fisheye micro surveillance camera, a micro alarm speaker and a high-speed network communication device are integrated into hardware by designing a circuit board and a process shell.

[0061] Among them, the controller is the core of the whole system. The controller is responsible for processing and analyzing the data captured by the micro radar point cloud acquisition device and the fisheye micro surveillance camera. It runs the object safety determination method, determines in real time whether the target object is in the safe area through the algorithm, and issues a warning through the micro alarm speaker when necessary. The micro radar point cloud acquisition device is responsible for collecting point cloud data of the target operation scene in real time. The point cloud data provides three-dimensional position and shape information about the objects in the scene, which is the key input for subsequent safety analysis. The fisheye camera provides a wide field of view and can capture more environmental details. It can serve as a supplement to the point cloud data to provide visual confirmation and additional safety monitoring functions. When the controller determines that the target object is in an unsafe area, the micro alarm speaker will issue a sound warning. This instant feedback mechanism helps to quickly attract people's attention and take necessary safety measures. The high-speed network communication device ensures that the controller can transmit high-speed and stable data with other systems or devices. In application scenarios that require remote monitoring or data sharing, the high-speed network communication device is crucial. The circuit board is the basis for integrating all hardware components. It ensures the electrical connection and communication between the various components. The process shell protects the internal electronic components from the influence of the external environment.

[0062] In one embodiment, Figure 1 As shown, a method for determining object security is provided, which specifically includes the following steps:

[0063] S101, acquiring real-time point cloud data obtained by a collection device carried by a target object to collect a target operation scene.

[0064] Among them, the acquisition device in this embodiment is the above-mentioned micro radar point cloud acquisition device.

[0065] It can be understood that real-time point cloud data refers to the point cloud data obtained by the acquisition device in the target operation scene from the surrounding environment; the acquisition device will continuously scan the surrounding environment and record the scanned three-dimensional space information in the form of point cloud data. These data contain information such as the three-dimensional coordinates and reflection intensity of each point in the scene, and can accurately describe the three-dimensional structure and object distribution of the target operation scene.

[0066] S102, performing similarity matching between the real-time point cloud data and the global point cloud data of the target operation scene, and obtaining similar point cloud data in the global point cloud data that matches the real-time point cloud data.

[0067] Among them, the global point cloud data of the target operation scene is collected by the point cloud acquisition equipment. The point cloud acquisition equipment will scan the entire environment of the construction site and capture the three-dimensional coordinates and reflection intensity information of each point. This process ensures that the three-dimensional structure of the entire construction site is fully recorded. Through the point cloud data obtained by comprehensive scanning, the overall three-dimensional model of the construction site can be constructed. This model not only includes the terrain, buildings and other important structures of the construction site, but also reflects the spatial relationship between them.

[0068] In a construction site, some areas may change due to construction activities or other reasons. In order to ensure the accuracy of the global point cloud data, additional point cloud collection is required for these locally changed areas to update and supplement the original data. Through the above steps, a global positioning basic data set that includes the entire construction site and the locally changed areas can be obtained. This data set provides the basis for subsequent point cloud matching, pose solution and safety judgment.

[0069] Furthermore, in order to more effectively perform point cloud matching and recognition, it is also necessary to extract feature points from the global point cloud data. These feature points are usually points with significant geometric features in three-dimensional space, such as corner points, edge points, etc. By using the relationship between feature points, the continuous surface structure feature points of the three-dimensional point cloud can be further extracted, thereby improving the accuracy and efficiency of point cloud matching.

[0070] It can be understood that the similarity matching process is to find the part of the global point cloud data that is most similar to the real-time point cloud data through a specific algorithm. This usually involves point cloud feature extraction, point cloud registration and other technologies. The algorithm will analyze the geometric features, spatial distribution and other information of the real-time point cloud data and the global point cloud data to determine the similarity between the two. After the match is successful, similar point cloud data that matches the real-time point cloud data in the global point cloud data can be obtained. This part of the data provides a reference for the subsequent determination of transformation information and the solution of the position and posture of the acquisition device.

[0071] S103, determining transformation information between the real-time point cloud data and the similar point cloud data.

[0072] The transformation information includes at least one of a scaling ratio, a position offset, and a direction offset.

[0073] It is understandable that due to different acquisition devices or changes in acquisition distances, the size of the real-time point cloud data may differ from the corresponding part in the global point cloud data. Therefore, a scaling factor needs to be calculated to ensure the consistency of the two in size. There may be a positional offset between the real-time point cloud data and similar point cloud data. This offset can be corrected by calculating the translation vector between the two, that is, moving the real-time point cloud data a certain distance in a certain direction to align with the global point cloud data. In addition to position and size, the orientation of the real-time point cloud data may also be inconsistent with the global point cloud data. Therefore, a rotation matrix or quaternion needs to be calculated to represent this change in orientation so that the real-time point cloud data can be rotated to the correct direction.

[0074] Specifically, a point cloud matching algorithm (such as the Iterative Closest Point algorithm ICP) can be used to align real-time point cloud data with global point cloud data. The algorithm will try to find the best match between the two sets of point clouds, and continuously optimize the transformation parameters through an iterative process to minimize the distance error between corresponding points. During the matching process, the areas where the two sets of point cloud data overlap or are close will be determined, and these areas are the fusion areas.

[0075] It can be understood that the determination of the fusion area helps the subsequent point cloud data fusion and scene reconstruction. Through the matching algorithm, the precise position of the real-time point cloud data in the global point cloud data coordinate system can be calculated. This positional relationship reflects the spatial distribution of the real-time point cloud data relative to the global point cloud data. During the matching process, the number of corresponding points in the two sets of point clouds is also matched. This helps to evaluate the integrity and quality of the point cloud data, as well as detect possible data loss or outliers. Due to different acquisition conditions or equipment, there may be scale differences between the real-time point cloud data and the global point cloud data. By calculating the scaling relationship between the two sets of point clouds, their consistency in scale can be ensured. This usually involves calculating a scaling factor, which is used to adjust the size of the real-time point cloud data to match the global point cloud data.

[0076] S104, determining the position and posture information of the acquisition device according to the transformation information.

[0077] Specifically, first, the real-time point cloud data is scaled accordingly according to the previously calculated scaling relationship to ensure that it is consistent in scale with the global point cloud data. Secondly, the orientation offset (rotation matrix or quaternion) is used to adjust the orientation of the real-time point cloud data so that it is aligned with the orientation of the global point cloud data. The real-time point cloud data that has been scaled and oriented is placed in the coordinate system of the global point cloud data for point cloud fusion. This step usually involves matching and superimposing the real-time point cloud data with the corresponding areas in the global point cloud data. Thirdly, based on the fused point cloud data and the placement position, a specific algorithm is used to automatically solve and locate the precise position of the acquisition device in the three-dimensional scene. This process may involve complex spatial geometry calculations and optimization algorithms. Finally, the position and posture data of the acquisition device are solved, including the plane position positioning value (x, y, z coordinates), orientation value (which can be expressed in three-dimensional coordinate vectors or angles) and elevation value (relative height). These data together describe the precise position and posture of the acquisition device in the three-dimensional scene.

[0078] It can be understood that position refers to the spatial positioning mark of a point or object in a three-dimensional scene, which is generally expressed by the {x, y, z} values ​​that constitute the coordinates, that is, the expression of the x, y, z values ​​and directions of the point or object to the origin {0, 0, 0}; elevation refers to the expression of the relative height of a point or position relative to a reference, which is generally the difference in the Y axis of the point or object. Orientation refers to the rotational posture expression of an object in a coordinate system, which can be expressed by a three-dimensional coordinate {x, y, z} vector or by an angle.

[0079] S105: Determine whether the target object is located in a safe area in the target operation scene according to the position and posture information.

[0080] In this embodiment, it relies on the engineering digital twin visualization safety application, which is built based on a fine three-dimensional digital model of the project. In this model, key safety elements such as safety electronic fences, safety areas, and safety warnings are preset. These elements form safety boundaries and warning areas in three-dimensional space. Next, the controller compares the high-precision positioning data provided by the acquisition device with these preset safety elements. By using the relationship judgment logic such as inclusion, proximity, and distance in three-dimensional space, the system can evaluate the relative position between the target object and the safety area in real time. If the system detects that the target object is about to enter or has entered an unsafe area, or is too close to a dangerous area in the target operation scene, the system will trigger a safety warning mechanism. This warning may include sound, light, or other forms of signals to promptly remind on-site personnel to take necessary safety measures. .

[0081] In the above-mentioned object safety determination method, this application compares real-time point cloud data with global point cloud data. This method can achieve high-precision positioning. The point cloud data itself contains rich three-dimensional spatial information, so the comparison result can accurately reflect the position and posture of the acquisition device (i.e., the target object) in the global coordinate system; by relying only on the comparison of point cloud data, this method simplifies the positioning process. Compared with traditional positioning methods that require the combination of multiple sensor data and complex algorithms, this method only needs to process point cloud data, thereby reducing the complexity of positioning; further, since the point cloud data comparison method does not rely on specific hardware or environmental features, it can be applied to a variety of different scenarios, including indoors, outdoors, and complex environments; in addition, this application does not require the installation of additional hardware positioning devices, such as GPS receivers, beacons or other sensors, and is more suitable for infrastructure digital twin systems.

[0082] like Figure 2 As shown, this embodiment provides an optional method for similarly matching the real-time point cloud data with the global point cloud data of the target operation scene to obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data, that is, it provides a method for refining S102. The specific implementation process may include:

[0083] S201, obtaining global point cloud data of a target operation scene.

[0084] Specifically, use high-precision equipment such as 3D laser scanners (3D LiDAR) or structured light scanners to collect point cloud data of the target work scene. According to the size and complexity of the work scene, plan a suitable scanning path and angle to ensure that all key features of the scene can be captured. For large or complex scenes, multiple scans may be required and the results must be stitched together. Perform scanning operations according to the planned path and angle. Ensure that the equipment is stable during the scanning process to reduce data jitter or distortion. Preprocess the raw point cloud data obtained by the scan, including removing noise, filling missing data, etc. Ensure the integrity and accuracy of the data to provide a reliable foundation for subsequent point cloud matching and analysis.

[0085] S202, generating a first feature point histogram according to the real-time point cloud data, and generating a second feature point histogram according to the global point cloud data.

[0086] It is understandable that the point cloud histogram represents a feature of three-dimensional points, similar to the SIFT scale-invariant feature transform (SIFT), Speeded UpRobust Features (SURF), ORB (Oriented Fast and Rotated BRIEF) features in two-dimensional images, all of which carry certain specific information. Like the two-dimensional image configuration in image recognition, the point cloud histogram can be used for registration between three-dimensional point clouds.

[0087] S203: Compare the first feature point histogram and the second feature point histogram to obtain a target area in the second feature point histogram that overlaps with the first feature point histogram.

[0088] Optionally, ensure that the first feature point histogram (representing the real-time point cloud data) and the second feature point histogram (representing the global point cloud data) have been generated according to the same feature extraction method and parameters. Normalize the two histograms so that their sum is 1, which facilitates subsequent comparison and matching. Use a similarity metric method, such as histogram intersection, Bhattacharyya distance, or correlation coefficient, to quantify the similarity between the two histograms. For each feature point bin, calculate the similarity value of the corresponding bin in the two histograms. Based on the similarity value, identify those feature point bins that are significantly represented in both histograms, which constitute the overlapping area. A threshold can be set, and only when the similarity exceeds this threshold, the two bins are considered to overlap. For the initially determined overlapping area, a more sophisticated matching algorithm (such as the feature-based iterative closest point algorithm ICP or its variants) can be further used to optimize the matching results.

[0089] S204: determining the point cloud data corresponding to the target area as similar point cloud data in the global point cloud data that matches the real-time point cloud data.

[0090] Specifically, according to the overlapping area determined in step S203, the corresponding feature point interval is found in the global point cloud data. These feature point intervals define a specific three-dimensional spatial range in the global point cloud data, namely the target area. The point cloud data located in the target area is filtered out from the global point cloud data. This can be achieved by comparing the coordinates of each point with the boundary of the target area. For the filtered point cloud data, a more sophisticated matching algorithm (such as the closest point iteration algorithm ICP) can be used to further optimize the matching with the real-time point cloud data. The accuracy of the matching results is verified by calculating indicators such as matching error and overlap. The filtered and matched point cloud data is output as similar point cloud data that matches the real-time point cloud data. These data can be used for subsequent security monitoring, navigation, modeling and other applications.

[0091] In this embodiment, similar point cloud data matching the real-time point cloud data in the global point cloud data can be quickly identified through intuitive comparison using the feature point histogram.

[0092] like Figure 3 As shown, this embodiment provides an optional method for determining the transformation information between the real-time point cloud data and the similar point cloud data, that is, provides a method for refining S103. The specific implementation process may include:

[0093] S301, parsing real-time point cloud data to obtain at least one first object.

[0094] The first objects contained in the real-time point cloud data are Targ{T1, T2, T3.....T M}, where M is the number of first objects.

[0095] S302: parse the similar point cloud data to obtain at least one second object.

[0096] The second objects included in the global point cloud data are ALL{P1, P2, P3.....P N}, where N is the number of second objects, N ≥ M; each second object contained in the similar point cloud data is a subset of ALL, namely CELL{P1, P2, P3.....P M}.

[0097] S303: Determine at least one similar object pair according to the similarity relationship between each first object and each second object.

[0098] Exemplarily, the first object T1 in Targ and the second object P1 in CELL constitute a similar object pair, the first object T2 in Targ and the second object P2 in CELL constitute a similar object pair, the first object T3 in Targ and the second object P3 in CELL constitute a similar object pair, and so on, constituting a total of M similar object pairs.

[0099] Each similar object pair includes a first object and a second object.

[0100] S304: Determine transformation information between the first object and the second object in each similar object pair.

[0101] S305 , determining transformation information between the real-time point cloud data and the similar point cloud data according to the transformation information corresponding to each similar object pair.

[0102] Optionally, the mean of the transformation information corresponding to each pair of similar objects is determined as the transformation information between the real-time point cloud data and the similar point cloud data.

[0103] Specifically, the transformation information between the real-time point cloud data and the similar point cloud data is determined as shown in the following formula (1):

[0104] (1)

[0105] Where M is the number of similar object pairs; T1 is the i-th first object contained in the real-time point cloud data; P i is the i-th second object contained in the similar point cloud data; r represents the rotation value of Targ mapped to ALL overlaps (including the attitude synchronization rotation change value), and t represents the scaling value of Targ mapped to ALL overlaps (including scaling, elevation, and position movement values).

[0106] First, the values ​​of r and t are calculated according to the above formula (1). Secondly, the Targ point cloud is completely embedded into ALL according to the values ​​of r and t to calculate the pose information of the acquisition device. The pose information includes the position, elevation and orientation values ​​of the real-time point cloud data.

[0107] Specifically, the posture information of the acquisition device is determined according to the transformation information and the posture information of the acquisition device corresponding to the similar point cloud data.

[0108] It can be understood that, first, similar object pairs are extracted from the real-time point cloud data (Targ) and the global point cloud data (ALL). These object pairs may be found by feature matching, nearest point search or other registration techniques. Each similar object pair includes a point cloud object from Targ (the first object T1) and a point cloud object from ALL (the second object Pi). The number of similar object pairs M represents the number of successfully matched point pairs.

[0109] Secondly, these similar object pairs are used to estimate the transformation parameters from Targ to ALL, including the rotation value r and the scaling (or translation) value t. This is usually achieved by minimizing the distance error between the object pairs, such as using the least squares method or other optimization methods. The rotation value r describes how Targ needs to be rotated to align with ALL, while the scaling (or translation) value t describes how Targ needs to be moved (including changes in position and elevation) to align with ALL. Once the values ​​of r and t are calculated, these transformation parameters can be applied to the entire Targ point cloud so that it is fully embedded in the ALL point cloud. This means that according to the calculated rotation and scaling (or translation) parameters, the position and orientation of each point in Targ are adjusted to align with the corresponding point in ALL. After the Targ point cloud is aligned with the ALL point cloud, the pose information of the acquisition device can be calculated based on the transformed Targ point cloud. The pose information includes the position, elevation, and orientation (or attitude) of the acquisition device in three-dimensional space. If the pose information of the acquisition device corresponding to the similar point cloud data is known (for example, obtained by other sensors such as IMU or GPS), then this information can be combined to further improve the accuracy of the pose estimation.

[0110] In this embodiment, by constructing similar object pairs, the function of accurately calculating the transformation information between real-time point cloud data and similar point cloud data can be achieved.

[0111] like Figure 4 As shown, this embodiment provides an optional method for determining whether the target object is located in a safe area in the target operation scene according to the posture information, that is, it provides a method for refining S105. The specific implementation process may include:

[0112] S401, obtaining the location information of the electronic fence in the target operation scene.

[0113] Optionally, determine the physical boundaries of the electronic fence based on the actual needs and safety regulations of the operating scenario. This can be determined through mapping, field measurements, or using existing geographic information system (GIS) data. For outdoor scenarios, GNSS (Global Navigation Satellite System) positioning can be used, such as GPS (Global Positioning System) signals to determine the boundary points of the electronic fence. For indoor or signal-limited areas, other positioning technologies such as Wi-Fi positioning, Bluetooth beacons, RFID tags, etc. can be considered. Set up obvious signs at the boundaries of the electronic fence, such as physical signs, virtual boundaries, or clear markings on maps. These signs should be easy to identify and understand so that people and equipment can quickly identify whether to enter or leave the fenced area.

[0114] S402, comparing the position information in the posture information with the position information of the electronic fence to determine whether the target object is located in a safe area in the target operation scene.

[0115] Specifically, the controller provides the position information of the acquisition device to the digital twin system application, and the digital twin system restores the position of the acquisition device in the three-dimensional scene based on the data. The digital twin scene references the position of the acquisition device and the electronic fence, equipment safety area, etc. in the digital twin scene to perform three-dimensional space calculations such as inclusion, contact, proximity, and distance between the positions, and based on the attributes of the electronic fence, equipment safety area, etc., the digital twin determines whether it is necessary to initiate an alarm event trigger.

[0116] Furthermore, if the acquisition device is located outside the safety area in the target operation scene, the image device corresponding to the acquisition device is started; the image acquisition device is controlled to obtain image information at the location of the acquisition device; if the image information contains obstacle information, prompt information is output.

[0117] In this embodiment, through the digital twin system, an alarm message can be generated when a person / vehicle reaches an unsafe area, and the image acquisition device can be used to determine whether there are obstacles around the current location, so as to achieve the effect of safety warning.

[0118] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0119] Based on the same inventive concept, the embodiment of the present application also provides an object safety determination device 1 for implementing the object safety determination method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more object safety determination device embodiments provided below can refer to the limitations of the object safety determination method above, and will not be repeated here.

[0120] In one embodiment, Figure 5 As shown, a device 1 for determining object safety is provided, comprising:

[0121] An acquisition module 11 is used to acquire real-time point cloud data obtained by a collection device carried by a target object to collect a target operation scene;

[0122] A matching module 12 is used to perform similarity matching between the real-time point cloud data and the global point cloud data of the target operation scene, and obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data;

[0123] A transformation module 13, used to determine transformation information between the real-time point cloud data and the similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset, and a direction offset;

[0124] A posture determination module 14, used to determine the posture information of the acquisition device according to the transformation information;

[0125] The safety positioning module 15 is used to determine whether the target object is located in a safe area in the target operation scene according to the position information.

[0126] In one embodiment, the matching module 12 is further configured to:

[0127] Obtain global point cloud data of the target operation scene;

[0128] Generate a first feature point histogram according to the real-time point cloud data, and generate a second feature point histogram according to the global point cloud data;

[0129] Comparing the first feature point histogram with the second feature point histogram to obtain a target area in the second feature point histogram that overlaps with the first feature point histogram;

[0130] The point cloud data corresponding to the target area is determined as similar point cloud data in the global point cloud data that matches the real-time point cloud data.

[0131] In one embodiment, the transformation module 13 includes:

[0132] A first parsing submodule, used to parse the real-time point cloud data to obtain at least one first object;

[0133] A second parsing submodule, used to parse the similar point cloud data to obtain at least one second object;

[0134] A comparison submodule, configured to determine at least one similar object pair according to the similarity relationship between each first object and each second object; wherein each similar object pair includes a first object and a second object;

[0135] A similarity determination submodule, used to determine the transformation information between the first object and the second object in each similar object pair;

[0136] The comprehensive determination submodule is used to determine the transformation information between the real-time point cloud data and the similar point cloud data according to the corresponding transformation information of each similar object pair.

[0137] In one embodiment, the comprehensive determination submodule is further used to: determine the mean of the transformation information corresponding to each pair of similar objects as the transformation information between the real-time point cloud data and the similar point cloud data.

[0138] In one of the embodiments, the posture determination module 14 is further used to determine the posture information of the acquisition device according to the transformation information and the posture information of the acquisition device corresponding to the similar point cloud data.

[0139] In one embodiment, the safety positioning module 15 is further used to: obtain location information of the electronic fence in the target operation scene;

[0140] Compare the position information in the posture information with the position information of the electronic fence to determine whether the target object is located in a safe area in the target operation scene.

[0141] In one of the embodiments, the device further includes an obstacle recognition module, the obstacle recognition module being used to: if the acquisition device is located outside the safety area in the target operation scene, start an image device corresponding to the acquisition device;

[0142] Controlling the image acquisition device to obtain image information at the location of the acquisition device;

[0143] If the image information contains obstacle information, the prompt information is output.

[0144] Each module in the above-mentioned object safety determination device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0145] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data of the object security determination method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an object security determination method is implemented.

[0146] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0147] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0148] Acquire real-time point cloud data obtained by the acquisition device carried by the target object to collect the target operation scene;

[0149] Perform similarity matching between the real-time point cloud data and the global point cloud data of the target operation scene, and obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data;

[0150] Determine transformation information between the real-time point cloud data and the similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset, and an orientation offset;

[0151] Determine the position and posture information of the acquisition device according to the transformation information;

[0152] Based on the position information, determine whether the target object is located in a safe area in the target operation scene.

[0153] In one embodiment, when a processor executes a computer program to perform similar matching between real-time point cloud data and global point cloud data of a target work scene, and obtains logic for similar point cloud data in the global point cloud data that matches the real-time point cloud data, the following steps are specifically implemented: obtaining the global point cloud data of the target work scene; generating a first feature point histogram based on the real-time point cloud data, and generating a second feature point histogram based on the global point cloud data; comparing the first feature point histogram and the second feature point histogram to obtain a target area in the second feature point histogram that overlaps with the first feature point histogram; and determining the point cloud data corresponding to the target area as similar point cloud data in the global point cloud data that matches the real-time point cloud data.

[0154] In one embodiment, when the processor executes a computer program to determine the logic of transformation information between real-time point cloud data and similar point cloud data, the following steps are specifically implemented: parsing the real-time point cloud data to obtain at least one first object; parsing the similar point cloud data to obtain at least one second object; determining at least one similar object pair based on the similarity relationship between each first object and each second object; wherein each similar object pair includes a first object and a second object; determining the transformation information between the first object and the second object in each similar object pair; and determining the transformation information between the real-time point cloud data and the similar point cloud data based on the transformation information corresponding to each similar object pair.

[0155] In one embodiment, when the processor executes a computer program to determine the logic of transformation information between real-time point cloud data and similar point cloud data based on the transformation information corresponding to each pair of similar objects, the following steps are specifically implemented: the mean of the transformation information corresponding to each pair of similar objects is determined as the transformation information between the real-time point cloud data and the similar point cloud data.

[0156] In one embodiment, when the processor executes the logic of the computer program to determine the posture information of the acquisition device based on the transformation information, the following steps are specifically implemented: the posture information of the acquisition device is determined based on the transformation information and the posture information of the acquisition device corresponding to the similar point cloud data.

[0157] In one embodiment, when the processor executes the logic of a computer program to determine whether the target object is located in a safe area in a target work scene based on the posture information, the following steps are specifically implemented: obtaining the position information of an electronic fence in the target work scene; comparing the position information in the posture information with the position information of the electronic fence to determine whether the target object is located in a safe area in the target work scene.

[0158] In one embodiment, when the processor executes the computer program, the following steps are also implemented: if the acquisition device is located outside the safety area in the target operation scene, the image device corresponding to the acquisition device is started; the image acquisition device is controlled to obtain image information at the location of the acquisition device; if the image information contains obstacle information, prompt information is output.

[0159] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0160] Acquire real-time point cloud data obtained by the acquisition device carried by the target object to collect the target operation scene;

[0161] Perform similarity matching between the real-time point cloud data and the global point cloud data of the target operation scene, and obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data;

[0162] Determine transformation information between the real-time point cloud data and the similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset, and an orientation offset;

[0163] Determine the position and posture information of the acquisition device according to the transformation information;

[0164] Based on the position information, determine whether the target object is located in a safe area in the target operation scene.

[0165] In one embodiment, when a computer program similarly matches real-time point cloud data with global point cloud data of a target work scene and obtains similar point cloud data in the global point cloud data that matches the real-time point cloud data, the following steps are specifically implemented when the logic is executed by a processor: obtaining the global point cloud data of the target work scene; generating a first feature point histogram based on the real-time point cloud data, and generating a second feature point histogram based on the global point cloud data; comparing the first feature point histogram and the second feature point histogram to obtain a target area in the second feature point histogram that overlaps with the first feature point histogram; and determining the point cloud data corresponding to the target area as similar point cloud data in the global point cloud data that matches the real-time point cloud data.

[0166] In one embodiment, when the logic of a computer program for determining transformation information between real-time point cloud data and similar point cloud data is executed by a processor, the following steps are specifically implemented: parsing the real-time point cloud data to obtain at least one first object; parsing the similar point cloud data to obtain at least one second object; determining at least one similar object pair based on the similarity relationship between each first object and each second object; wherein each similar object pair includes a first object and a second object; determining the transformation information between the first object and the second object in each similar object pair; and determining the transformation information between the real-time point cloud data and the similar point cloud data based on the transformation information corresponding to each similar object pair.

[0167] In one embodiment, when the logic of determining the transformation information between the real-time point cloud data and the similar point cloud data based on the transformation information corresponding to each similar object pair is executed by the processor, the following steps are specifically implemented: the mean of the transformation information corresponding to each similar object pair is determined as the transformation information between the real-time point cloud data and the similar point cloud data.

[0168] In one embodiment, when the logic of the computer program for determining the posture information of the acquisition device based on the transformation information is executed by the processor, the following steps are specifically implemented: determining the posture information of the acquisition device based on the transformation information and the posture information of the acquisition device corresponding to the similar point cloud data.

[0169] In one embodiment, when the logic of a computer program that determines whether a target object is located in a safe area in a target work scene based on posture information is executed by a processor, the following steps are specifically implemented: obtaining the position information of an electronic fence in the target work scene; comparing the position information in the posture information with the position information of the electronic fence to determine whether the target object is located in a safe area in the target work scene.

[0170] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: if the acquisition device is located outside the safety area in the target work scene, the image device corresponding to the acquisition device is started; the image acquisition device is controlled to obtain image information at the location of the acquisition device; if the image information contains obstacle information, prompt information is output.

[0171] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0172] Acquire real-time point cloud data obtained by the acquisition device carried by the target object to collect the target operation scene;

[0173] Perform similarity matching between the real-time point cloud data and the global point cloud data of the target operation scene, and obtain similar point cloud data in the global point cloud data that matches the real-time point cloud data;

[0174] Determine transformation information between the real-time point cloud data and the similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset, and an orientation offset;

[0175] Determine the position and posture information of the acquisition device according to the transformation information;

[0176] Based on the position information, determine whether the target object is located in a safe area in the target operation scene.

[0177] In one embodiment, when a computer program similarly matches real-time point cloud data with global point cloud data of a target work scene and obtains similar point cloud data in the global point cloud data that matches the real-time point cloud data, the following steps are specifically implemented when the logic is executed by a processor: obtaining the global point cloud data of the target work scene; generating a first feature point histogram based on the real-time point cloud data, and generating a second feature point histogram based on the global point cloud data; comparing the first feature point histogram and the second feature point histogram to obtain a target area in the second feature point histogram that overlaps with the first feature point histogram; and determining the point cloud data corresponding to the target area as similar point cloud data in the global point cloud data that matches the real-time point cloud data.

[0178] In one embodiment, when the logic of a computer program for determining transformation information between real-time point cloud data and similar point cloud data is executed by a processor, the following steps are specifically implemented: parsing the real-time point cloud data to obtain at least one first object; parsing the similar point cloud data to obtain at least one second object; determining at least one similar object pair based on the similarity relationship between each first object and each second object; wherein each similar object pair includes a first object and a second object; determining the transformation information between the first object and the second object in each similar object pair; and determining the transformation information between the real-time point cloud data and the similar point cloud data based on the transformation information corresponding to each similar object pair.

[0179] In one embodiment, when the logic of determining the transformation information between the real-time point cloud data and the similar point cloud data based on the transformation information corresponding to each similar object pair is executed by the processor, the following steps are specifically implemented: the mean of the transformation information corresponding to each similar object pair is determined as the transformation information between the real-time point cloud data and the similar point cloud data.

[0180] In one embodiment, when the logic of the computer program for determining the posture information of the acquisition device based on the transformation information is executed by the processor, the following steps are specifically implemented: determining the posture information of the acquisition device based on the transformation information and the posture information of the acquisition device corresponding to the similar point cloud data.

[0181] In one embodiment, when the logic of a computer program that determines whether a target object is located in a safe area in a target work scene based on posture information is executed by a processor, the following steps are specifically implemented: obtaining the position information of an electronic fence in the target work scene; comparing the position information in the posture information with the position information of the electronic fence to determine whether the target object is located in a safe area in the target work scene.

[0182] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: if the acquisition device is located outside the safety area in the target work scene, the image device corresponding to the acquisition device is started; the image acquisition device is controlled to obtain image information at the location of the acquisition device; if the image information contains obstacle information, prompt information is output.

[0183] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0184] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0185] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for determining object security, characterized in that: The method comprises: Acquire real-time point cloud data obtained by the acquisition device carried by the target object to collect the target operation scene; Acquire global point cloud data of the target operation scene; Generating a first feature point histogram according to the real-time point cloud data, and generating a second feature point histogram according to the global point cloud data; Normalizing the first feature point histogram and the second feature point histogram so that the sum of the first feature point histogram and the second feature point histogram is 1; Comparing the first feature point histogram after the normalization process with the second feature point histogram, obtaining a target area in the second feature point histogram that overlaps with the first feature point histogram; Determine the point cloud data corresponding to the target area as similar point cloud data in the global point cloud data that matches the real-time point cloud data; Determine transformation information between the real-time point cloud data and the similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset, and a direction offset; According to the transformation information, aligning the real-time point cloud data with the similar point cloud data; Matching the aligned real-time point cloud data with the similar point cloud data to obtain a fusion area; wherein the fusion area is an overlapping or similar area between the aligned real-time point cloud data and the similar point cloud data; Matching the number of point clouds of the real-time point cloud data in the fusion area with the number of point clouds of similar point cloud data; If the match is successful, the real-time point cloud data that has been matched is scaled accordingly according to the scaling ratio relationship; Using a rotation matrix or a quaternion, adjusting the point cloud orientation of the scaled real-time point cloud data so that the point cloud orientation of the adjusted real-time point cloud data is consistent with the point cloud orientation of the global point cloud data; Placing the adjusted real-time point cloud data in the coordinate system of the global point cloud data, and performing point cloud fusion on the adjusted real-time point cloud data and the similar point cloud data; Determine the position information of the acquisition device according to the placement position of the fused real-time point cloud data; According to the position information, it is determined whether the target object is located in a safe area in the target operation scene.

2. The method according to claim 1, characterized in that The step of comparing the first feature point histogram after the normalization process with the second feature point histogram to obtain a target area in the second feature point histogram that overlaps with the first feature point histogram includes: For each feature point interval, calculating a similarity value of the feature point interval in the first feature point histogram and the second feature point histogram after two normalization processes; If the similarity value exceeds a similarity threshold, the feature point interval is added to a target area in the second feature point histogram that overlaps with the first feature point histogram.

3. The method according to claim 1, characterized in that The determining of transformation information between the real-time point cloud data and the similar point cloud data includes: Parsing the real-time point cloud data to obtain at least one first object; Parsing the similar point cloud data to obtain at least one second object; Determine at least one similar object pair according to the similarity relationship between each first object and each second object; wherein each similar object pair includes a first object and a second object; determining transformation information between a first object and a second object in each pair of similar objects; According to the transformation information corresponding to each similar object pair, the transformation information between the real-time point cloud data and the similar point cloud data is determined.

4. The method according to claim 3, characterized in that The step of determining the transformation information between the real-time point cloud data and the similar point cloud data according to the transformation information corresponding to each similar object pair includes: The mean value of the transformation information corresponding to each pair of similar objects is determined as the transformation information between the real-time point cloud data and the similar point cloud data.

5. The method according to claim 1, characterized in that Determining the position information of the acquisition device according to the placement position of the fused real-time point cloud data includes: The posture information of the acquisition device is determined according to the placement position of the fused real-time point cloud data and the posture information of the acquisition device corresponding to the similar point cloud data.

6. The method according to claim 1, characterized in that The step of determining whether the target object is located in a safe area in the target operation scene according to the position information includes: Obtaining location information of an electronic fence within the target operation scene; The position information in the posture information is compared with the position information of the electronic fence to determine whether the target object is located in a safe area in the target operation scene.

7. The method according to claim 6, characterized in that The method further comprises: If the target object is outside the safety area in the target operation scene, starting the image device corresponding to the acquisition device; Controlling the imaging device to collect image information at the location of the collecting device; If the image information contains obstacle information, prompt information is output.

8. An object safety determination device, characterized in that: The device comprises: An acquisition module is used to acquire real-time point cloud data obtained by a collection device carried by a target object to collect a target operation scene; A matching module, used for acquiring global point cloud data of the target operation scene; Generating a first feature point histogram according to the real-time point cloud data, and generating a second feature point histogram according to the global point cloud data; Normalizing the first feature point histogram and the second feature point histogram so that the sum of the first feature point histogram and the second feature point histogram is 1; Comparing the first feature point histogram after the normalization process with the second feature point histogram, obtaining a target area in the second feature point histogram that overlaps with the first feature point histogram; Determine the point cloud data corresponding to the target area as similar point cloud data in the global point cloud data that matches the real-time point cloud data; A transformation module, used to determine transformation information between the real-time point cloud data and the similar point cloud data; wherein the transformation information includes at least one of a scaling relationship, a position offset, and a direction offset; A posture determination module, used for aligning the real-time point cloud data with the similar point cloud data according to the transformation information; Matching the aligned real-time point cloud data with the similar point cloud data to obtain a fusion area; wherein the fusion area is an overlapping or similar area between the aligned real-time point cloud data and the similar point cloud data; Matching the number of point clouds of the real-time point cloud data in the fusion area with the number of point clouds of similar point cloud data; If the match is successful, the real-time point cloud data that has been matched is scaled accordingly according to the scaling ratio relationship; Using a rotation matrix or a quaternion, adjusting the point cloud orientation of the scaled real-time point cloud data so that the point cloud orientation of the adjusted real-time point cloud data is consistent with the point cloud orientation of the global point cloud data; Placing the adjusted real-time point cloud data in the coordinate system of the global point cloud data, and performing point cloud fusion on the adjusted real-time point cloud data and the similar point cloud data; Determine the position information of the acquisition device according to the placement position of the fused real-time point cloud data; A safety positioning module is used to determine whether the target object is located in a safe area in the target operation scene according to the posture information.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images