Metacosmic positioning abnormity real-time alarm method and system

Abstract

The invention provides a meta-universe positioning abnormity real-time alarm method, and relates to the technical field of data supervision, and the method comprises the steps: obtaining a positioning request initiated by a client of a user in a target space, the positioning request carrying context information capable of positioning the position of the user; performing pose calculation on the request picture set in the context information and the equipment information and a positioning map corresponding to the target space to obtain the number of points in the pose; judging whether positioning is abnormal or not according to the number of the points in the pose, when the positioning is abnormal, acquiring historical user VPS coordinates successfully positioned last time, and determining the current motion range of the user based on the historical user VPS coordinates and the equipment information; recognizing and generating a historical object picture set of the target space point cloud model in the current motion range by adopting an object recognition model; and performing anomaly analysis according to the historical object picture set and the current object picture set in the request picture set, and if there is an anomaly, generating a positioning alarm that there is an object change in the target space.

Description

Technical Field

[0001] This application relates to the field of data supervision technology, and in particular to a method and system for real-time alarm of location anomalies in the metaverse. Background Technology

[0002] With technological advancements, it has become possible to create virtual models of the metaverse within real-world spaces (such as virtual antiques or virtual tourist attractions). As the user moves, different virtual models will appear, and the stability and accuracy of these models depend on visual positioning within the real-world space. Visual positioning failures frequently occur, and these failures can be caused by various factors, including performance issues with the user's devices, insufficient system resources, and / or changes in the real-world space, such as the movement, addition, or removal of objects within the room, or renovations.

[0003] In existing technologies, changes in the real-world space cannot be promptly alerted. When visual positioning failures occur, users must report to customer service, and R&D personnel must travel to the site to investigate the cause. Once the cause is determined to be a change in the real-world space, data on the changed space must be re-collected, followed by repair of the spatial point cloud model. Due to the slow response time of R&D intervention in positioning and repair, and the long repair time, customers cannot experience the accurately positioned virtual model of the metaverse in a timely manner, affecting the overall user experience. Furthermore, the lack of real-time capability prevents time from being allocated for subsequent video re-collection and repair of the spatial point cloud model. Summary of the Invention

[0004] This application provides a method and system for real-time alarm of positioning anomalies in the metaverse, which can promptly detect changes in the target space, improve convenience and real-time performance, and enhance service responsiveness.

[0005] The first aspect of this application provides a real-time alarm method for location anomalies in the metaverse. The method is applied to a server and includes... The location request initiated by the client of the user in the target space is obtained. The location request carries contextual information that can locate the user's location. The target space refers to the real space that presents a virtual model of the metaverse. The request image set and device information in the context information are compared with the localization map corresponding to the target space to calculate the pose and obtain the number of points in the pose. The system determines whether the positioning is abnormal based on the number of points in the pose. When the positioning is abnormal, it obtains the historical user VPS coordinates of the last successful positioning and determines the user's current movement range based on the historical user VPS coordinates and device information. An object recognition model is used to identify and generate a set of historical object images within the current motion range of the target spatial point cloud model. The object recognition model is constructed by training a fully convolutional neural network based on the object point cloud data. Anomaly analysis is performed based on the historical object image set and the current object image set in the requested image set. If anomalies are found, a location alarm is generated indicating that there are changes in the object within the target space.

[0006] In some embodiments, determining whether a positioning anomaly is caused by the number of pose in-points includes: If the number of points within the pose is greater than or equal to the number threshold, the positioning request is considered successful and the positioning is normal. The context information of this request is saved and the current user VPS coordinates obtained from the pose calculation are returned to the client.

[0007] If the number of points within the pose is less than the number threshold, the localization request is deemed to have failed. When a location request fails, check if the issue is due to changes in non-target objects and if historical user VPS coordinates can be obtained. If the problem is related to changes in objects outside the target space or if the object cannot be obtained, the client will be returned a location request failure message. If the issue is not a change in non-target space objects and can be obtained, then it is judged as a location anomaly.

[0008] In some embodiments, the problem of changes in non-target space objects is a problem of insufficient performance of the client device or insufficient system resources.

[0009] In some embodiments, the pose calculation is performed by comparing the requested image set and device information in the context information with the positioning map corresponding to the target space, including: Extract the first two-dimensional feature points of the requested image set and match them with the second two-dimensional features in the positioning map corresponding to the target space to match the target two-dimensional feature points in the second two-dimensional feature points; By binding the target 2D feature points with 3D map points, the matching pairs and correspondences between the first 2D feature points and 3D map points are determined, and pose calculation is performed based on the matching pairs, correspondences, and device information.

[0010] In some embodiments, device information includes the client user's fastest movement speed, slowest movement speed, and most frequently occurring movement speed in the target space. The current movement range of the user is determined based on historical user VPS coordinates and device information, including: Construct a Simpson distribution based on the fastest speed, slowest speed, and most frequently occurring speed. In each round of Monte Carlo simulation, multiple random motion velocities are sampled from the Simpson distribution; Divide the interval between the fastest and slowest speeds into several equal groups; The frequency of random motion speeds in each group is counted and the probability is calculated. The upper limit of the group is determined by the cumulative probability exceeding 99% as the target motion speed. Based on multiple target motion speeds obtained from multiple rounds of Monte Carlo simulation, if the statistical convergence calculated based on multiple target motion speeds is less than 1%, the target motion speed of the current round is confirmed as the user's current speed. Get the time interval between the last successful location request and the location request initiated by the client; The motion radius is determined based on the time interval and the user's current speed; Using the historical user VPS coordinates as the origin, the user's current movement range is determined based on the movement radius.

[0011] In some embodiments, the construction of the object recognition model includes: Acquire the object point cloud data of each historical object in the target space, and label the object point cloud data with the corresponding object category labels to generate training data; The PointNet or 3D U-Net network is trained based on the training data. The predicted object category is calculated during the forward propagation of the training process. The loss value is calculated based on the predicted object category and the object category label. The weights and biases of the PointNet or 3D U-Net network are optimized during the backpropagation based on the loss value. Under the preset training stopping conditions, the object recognition model is constructed.

[0012] In some embodiments, an object recognition model is used to identify and generate a set of historical object images of the target spatial point cloud model within the current motion range, including: Load the preset target space point cloud model, and cut out the local point cloud data corresponding to the current motion range from the target space point cloud model; The local point cloud data is input into the object recognition model, the point cloud data of historical objects is output, and a set of historical object images is drawn based on the point cloud data of historical objects.

[0013] In some embodiments, anomaly analysis is performed based on the historical object image set and the current object image set in the requested image set. If an anomaly is found, a location alarm indicating a change in the target space is generated, including: The system performs a similarity analysis between the historical object image set and the current object image set in the requested image set. If the similarity is lower than a preset value, a location alarm is generated indicating that a new object has entered the target space; or Extract the first positional relationship between the target object and its neighboring objects in the historical object image set, and the second positional relationship between the approximate objects that are similar to the target object and its neighboring objects in the current object image set; If the first positional relationship and the second positional relationship are determined to be different, a positioning alarm is generated indicating that a target object is moving within the target space.

[0014] In some embodiments, after generating a location alarm indicating a change in an object within the target space, the method further includes: Create a real-time spatial video acquisition task and send it to the client via a long link; Receive spatial video sent by the client, model the spatial video, and generate a block texture model and a sparse point cloud model for updating the target spatial point cloud model; Add spatial video, block texture model and sparse point cloud model to alarm logs.

[0015] The second aspect of this application provides a real-time alarm system for location anomalies in a metaverse, the alarm system including a server and a client; The server obtains location requests initiated by clients of users located in the target space. These requests carry contextual information that can pinpoint the user's location. The target space refers to the real-world space that contains a virtual model of the metaverse. The requested image set and device information from the context information are compared with the localization map corresponding to the target space to calculate the pose and obtain the number of points within the pose; and The system determines whether a positioning anomaly is detected based on the number of pose in-points. If an anomaly is detected, the system retrieves the historical user VPS coordinates from the last successful positioning attempt. Based on these historical VPS coordinates and device information, the system determines the user's current movement range. An object recognition model is used to identify and generate a set of historical object images within the current motion range of the target spatial point cloud model. The object recognition model is constructed by training a fully convolutional neural network based on the object point cloud data. Anomaly analysis is performed based on the historical object image set and the current object image set in the requested image set. If anomalies are found, a location alarm is generated indicating that there are changes in the object within the target space.

[0016] Understandably, the metaverse positioning anomaly real-time alarm method and system provided in this application calculates the pose of the requested image set and device information in the context information against the positioning map corresponding to the target space, obtains the number of points within the pose, and judges the positioning anomaly based on the number of points within the pose, which can preliminarily determine that there is a change in the target space. Next, based on the historical user VPS coordinates and device information, the user's current movement range is determined, and an object recognition model is used to identify and generate a set of historical object images of the target space point cloud model within the current movement range. Finally, anomaly analysis is performed based on the historical object image set and the current object image set in the requested image set. If an anomaly is found, a positioning alarm is generated indicating that there is a change in objects within the target space. By comparing the image set of the user's current movement range, the range that needs to be compared is narrowed, greatly improving the computing performance. At the same time, as the user moves, different areas of the target space can be located, and it can be found that objects in different areas have changed. Through real-time calculation, changes in the target space can be detected in a timely manner, and the R&D personnel can be notified to update the target space point cloud model in a timely manner, eliminating the need for R&D personnel to travel to the site to investigate the cause one by one, improving convenience and real-time performance. This improves the responsiveness of the service and optimizes the user experience. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] Figure 1 A schematic diagram of the structure of a real-time alarm system for positioning anomalies in the metaverse provided in this application embodiment; Figure 2 A flowchart illustrating a real-time alarm method for positioning anomalies in the metaverse provided in this application embodiment; Figure 3 Another flowchart illustrating the real-time alarm method for positioning anomalies in the metaverse provided in this application embodiment.

[0019] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0021] The terms “first”, “second”, etc. used in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0022] Please see Figure 1 , Figure 1 The metaverse positioning anomaly real-time alarm system provided in this application includes a server 10 and a client 20. The client 20 cooperates with the server 10 to execute the metaverse positioning anomaly real-time alarm method provided in this application.

[0023] The technical solution of this application and how the technical solution of this application solves the technical problem are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0024] Please see Figure 2 , Figure 2 A flowchart illustrating the real-time alarm method for location anomalies in the metaverse provided in this application is shown. This alarm method is applied to the server and may include the following steps: Step S210: Obtain the location request initiated by the client of the user in the target space. The location request carries context information that can locate the user's location.

[0025] Specifically, the target space refers to the real space that presents a virtual model of the metaverse. The client device can be an augmented reality (AR) device. The user carries an AR device and initiates a location request to the server through the device. The device captures the requested image set and collects device information to form context information, which is then sent to the server.

[0026] Step S220: Perform pose calculation on the requested image set and device information in the context information and the corresponding localization map in the target space to obtain the number of points in the pose.

[0027] In one implementation, pose calculation includes the following steps: Step S221: Extract the first two-dimensional feature points of the requested image set and match them with the second two-dimensional features in the positioning map corresponding to the target space to match the target two-dimensional feature points in the second two-dimensional feature points.

[0028] Specifically, the server pre-stores a location map corresponding to the target space. This location map consists of second-dimensional feature points and three-dimensional map points strongly bound to these second-dimensional feature points. By matching the first-dimensional feature points with the second-dimensional feature points, candidate image sets similar to the requested image set can be identified from the location map. Further comparison of the second-dimensional feature points with the first-dimensional feature points in the candidate image sets allows for the selection of the target image set. The second-dimensional feature points in the target image set are the target two-dimensional feature points.

[0029] Step S222: Determine the matching pairs and correspondences between the first two-dimensional feature points and the three-dimensional map points by binding the target two-dimensional feature points and the three-dimensional map points, and perform pose calculation based on the matching pairs, correspondences and device information.

[0030] Specifically, a matching pair refers to a combination of two-dimensional feature points and three-dimensional map points that have a corresponding relationship. Device information includes information used by the client device for pose calculation, such as camera intrinsic parameter matrices and distortion coefficients.

[0031] The server, based on the correspondence and device information, uses algorithms such as EPnP or UPnP to solve for the device's six-degree-of-freedom pose, i.e., the device pose. Next, using the device pose, 3D map points are projected onto the 2D plane of the requested image set to obtain 2D projected points. The reprojection error between the 2D projected points and the first 2D feature points is calculated. Matching pairs corresponding to the first 2D feature points with reprojection errors less than a preset threshold are identified as in-pose points, thus obtaining the number of in-pose points. Simultaneously, the device pose can be converted into the device's VPS coordinates, which is equivalent to converting it into the current user's VPS coordinates, to obtain the current user's VPS coordinates.

[0032] Step S230: Determine whether the positioning is abnormal based on the number of points in the pose. If the positioning is abnormal, obtain the historical user VPS coordinates of the last successful positioning, and determine the user's current movement range based on the historical user VPS coordinates and device information.

[0033] In one implementation, the device information also includes the client user's fastest movement speed, slowest movement speed, and most frequently occurring movement speed in the target space. The determination of the user's current movement range includes the following steps: Step a: Construct a Simpson distribution based on the fastest, slowest, and most frequently occurring movement speeds.

[0034] Specifically, the Simpson distribution is constructed using the most frequent speed *c* as the mean and the standard deviation *σ* calculated from the fastest speed *b* and the slowest speed *a*, forming a probability density function graph. The X-axis of the Simpson distribution represents speed, and the Y-axis represents the probability density. The mathematical expression of the Simpson distribution is as follows:

[0035] in, Represents probability density, Indicates the speed of movement.

[0036] Step b: In each round of Monte Carlo simulation, multiple random motion velocities are sampled from the Simpson distribution.

[0037] Specifically, in each round of Monte Carlo simulation, M random samples (M can be 1000) are randomly sampled from the normal distribution N(μ,σ²) to obtain M random motion velocities.

[0038] Step c: Divide the interval from the fastest to the slowest speed into several equal groups. Specifically, divide the interval [a, b] into several equal groups according to speed from slowest to fastest. Step d: Count the number of times the random movement speed appears in each group and calculate the probability. Determine the upper limit of the group as the target movement speed by the cumulative probability exceeding 99%.

[0039] Specifically, the frequency of random motion speeds in each group is counted, and the frequency is divided by the number of samples to obtain the probability corresponding to each group. The probabilities corresponding to each group are continuously accumulated to obtain an updated cumulative probability. When the cumulative probability exceeds 99%, the upper limit of the group being accumulated is taken as the target motion speed (from a visualization perspective, a cumulative distribution function graph is generated at this point, with the group data as the X-axis and the cumulative probability obtained by continuously accumulating probabilities as the Y-axis, so that users can easily determine the upper limit of the group being accumulated when the cumulative probability exceeds 99%).

[0040] Step e: Based on the multiple target motion speeds obtained from the multi-round Monte Carlo simulation, if the statistical convergence calculated based on the multiple target motion speeds is less than 1%, the target motion speed of the current round is confirmed as the user's current speed.

[0041] Specifically, after multiple rounds of Monte Carlo simulation, multiple target motion velocities are obtained. Convergence is judged based on the statistical characteristics of multiple sets of target motion velocities (such as average value, coefficient of variation, etc.). When the convergence is less than 1%, the user's current speed is estimated. There are various estimation methods. For example, when the changes in multiple target motion velocities tend to reach a stable state (convergence less than 1%), the stable target motion speed is determined as the user's current target speed.

[0042] Step f: Obtain the time interval between the last successful location request and the location request initiated by the client.

[0043] Step g: Determine the motion radius based on the time interval and the user's current speed.

[0044] Step h: Using the historical user VPS coordinates as the origin, determine the user's current movement range based on the movement radius.

[0045] Step S240: Use an object recognition model to identify and generate a set of historical object images of the target space point cloud model within the current motion range.

[0046] The object recognition model is constructed by training a fully convolutional neural network based on object point cloud data.

[0047] Understandably, if the number of points within the pose is less than the threshold, it indicates that the number of matching pairs is too small. The currently captured requested image indicates that the current target space is different from the previous target space, suggesting possible object changes, requiring further image analysis.

[0048] Step S250: Perform anomaly analysis based on the historical object image set and the current object image set in the requested image set. If anomalies are found, generate a location alarm indicating that there are changes in the target space.

[0049] Specifically, after a location alarm is generated, it can be stored in the alarm log.

[0050] In one implementation of anomaly analysis, a similarity analysis is performed on the historical object image set and the current object image set in the requested image set. When the similarity is lower than a preset value, a location alarm is generated indicating that a new object has entered the target space.

[0051] One approach is to use a histogram tool to compare the similarity of the RGB three channels of each image in the historical object image set with that of each image in the current object image set.

[0052] In another implementation of anomaly analysis, a first positional relationship between the target object and its neighboring objects is extracted from the historical object image set, and a second positional relationship between approximate objects and their neighboring objects in the current object image set that are similar to the target object. Then, if the first and second positional relationships are determined to be different, a location alarm is generated indicating that the target object has moved within the target space.

[0053] The first and second positional relationships both include distance and orientation. The position extraction method involves traversing the object image set, identifying the target object, and cropping out an area that reflects the distance and orientation between the target object and its neighbors to form a first screenshot. Similar objects and their neighbors are also processed to form second screenshots. The similarity between the first and second screenshots is compared; if the similarity is below a preset threshold, the first and second positional relationships are determined to be different.

[0054] Understandably, this application uses cloud computing power on the server side to compare images and determine whether objects have changed, requiring no computing power from the client side, thus saving user bandwidth and computing / storage resources. Through image similarity comparison tasks, tasks can be executed in parallel, and computing power can be horizontally and hierarchically expanded as needed.

[0055] Understandably, in the above technical solution, the requested image set and device information in the context information are compared with the positioning map corresponding to the target space to calculate the pose, obtain the number of points within the pose, and determine the target space as a change when the number of points within the pose indicates a positioning anomaly. Next, based on the historical user VPS coordinates and device information, the user's current movement range is determined, and an object recognition model is used to identify and generate a set of historical object images within the current movement range of the target space point cloud model. Finally, anomaly analysis is performed based on the historical object image set and the current object image set in the requested image set. If an anomaly is found, a positioning alarm indicating a change in objects within the target space is generated. By comparing the image set of the user's current movement range, the range to be compared is narrowed, greatly improving computational performance. Simultaneously, as the user moves, different regions of the target space can be located, and changes in objects in different regions can be identified. Through real-time calculation, changes in the target space can be detected promptly, notifying developers to update the target space point cloud model in a timely manner, eliminating the need for developers to travel to the site to investigate the cause, thus improving convenience and real-time performance. This improves service responsiveness and optimizes the user experience.

[0056] In some embodiments, step S230, which determines whether a positioning anomaly is caused by the number of pose in-points, includes the following steps: Step S231: If the number of points in the pose is greater than or equal to the number threshold, the positioning request is determined to be successful and the positioning is normal. The context information of this positioning request is saved and returned to the client as the current user VPS coordinates obtained by the pose calculation.

[0057] Step S232: If the number of points in the pose is less than the number threshold, the positioning request is determined to have failed.

[0058] When the location request fails in step S233, check whether it is a problem of changes in non-target space objects and whether historical user VPS coordinates can be obtained.

[0059] Step S234: If the problem is a change in the non-target space object or it cannot be obtained, the location request is returned to the client as a failure.

[0060] Step S235: If it is not a problem of changes in non-target space objects and can be obtained, then it is judged as a positioning anomaly.

[0061] Understandably, there are many reasons why a location request might fail. Besides changes in the target space, these include issues such as device performance, insufficient system resources, and the inability to obtain historical user VPS coordinates or address changes in the target space due to the user's first use of the client device. This embodiment avoids warnings for location request failures caused by device performance, insufficient system resources, or the inability to obtain historical user VPS coordinates; it only warns of changes in the target space. This prevents misjudgments of changes in the target space caused by other issues such as device performance.

[0062] In some embodiments, after generating a location alarm indicating an object change within the target space in step S250, the alarm method further includes: S310: Creates real-time spatial video acquisition tasks and sends them to the client via a long link.

[0063] S320: Receives spatial video sent by the client, models the spatial video, and generates a block texture model and a sparse point cloud model for updating the target spatial point cloud model.

[0064] S330: Add spatial video, block texture model and sparse point cloud model to alarm logs.

[0065] Specifically, alarm records are pushed to developers' mobile phones or accounts in real time using technologies such as SMS and email. Developers log in to the operations platform to view the alarm record. By comparing the video collected by the user terminal in the alarm record with the video collected during previous modeling, they determine whether the target space has changed. If it has changed, a process to repair the target space point cloud model needs to be executed. Developers enter the spatial AR editor to load the running target space point cloud model and the block texture model used for repair. They drag, cut, and fit the model. If the block texture model is sufficient to repair the target space point cloud model, the block texture model and sparse point cloud model are replaced. If it is not sufficient for repair, offline methods are used to re-collect and remodel the model for repair.

[0066] Understandably, the block texture model and sparse point cloud model generated by the R&D personnel in this example can be used to update the point cloud model of the target space in real time without having to collect data in the target space, thus further improving real-time performance and convenience.

[0067] In some embodiments, the construction of an object recognition model includes the following steps: Step S110: Obtain the object point cloud data of each historical object in the target space, and label the object point cloud data with the corresponding object category labels to generate training data.

[0068] Step S120: Train a PointNet network or a 3D U-Net network based on the training data. Calculate the predicted object category during the forward propagation of the training process. Calculate the loss value based on the predicted object category and the object category label. Optimize the weights and biases of the PointNet network or the 3D U-Net network during backpropagation based on the loss value. Construct the object recognition model when the preset training stopping conditions are met.

[0069] In some embodiments, step S240: using an object recognition model to identify and generate a set of historical object images of the target spatial point cloud model within the current motion range, includes the following steps: Step S241: Load the preset target space point cloud model, and cut out the local point cloud data corresponding to the current motion range from the target space point cloud model.

[0070] Step S242: Input the local point cloud data into the object recognition model, output the point cloud data of historical objects, and draw a set of historical object images based on the point cloud data of historical objects.

[0071] Please see Figure 3 This application further illustrates the principles of some of the above embodiments through an application scenario. The real-time alarm method for positioning anomalies in the metaverse includes the following steps: Step S1: Obtain the location request initiated by the client of the user in the target space. The location request carries contextual information that can locate the user's location.

[0072] Step S2: Perform pose calculation on the requested image set and device information in the context information and the corresponding positioning map in the target space to obtain the number of points in the pose and the current user VPS coordinates.

[0073] Step S3: Determine whether the number of points in the pose is greater than or equal to the number threshold. If yes, proceed to step S4; otherwise, proceed to step S5.

[0074] Step S4: Determine if the location request is successful, save the context information of this request, and return the current user VPS coordinates obtained from the pose calculation to the client.

[0075] Step S5: Determine that the location request failed.

[0076] Step S6: Detect whether it is a problem of changes in non-target space objects. If yes, proceed to step S8; otherwise, proceed to step S9.

[0077] Step S7: Check if the coordinates of historical user VPS can be obtained. If yes, proceed to step S8; otherwise, proceed to step S9.

[0078] Step S8: The client is returned a location request failure message.

[0079] Step S9: Determine that the location is abnormal.

[0080] Step S10: Obtain the coordinates of the historical user VPS from the last successful location, and determine the user's current movement range based on the historical user VPS coordinates and device information.

[0081] Step S11: Use an object recognition model to identify and generate a set of historical object images of the target space point cloud model within the current motion range.

[0082] Step S12: Perform anomaly analysis based on the historical object image set and the current object image set in the requested image set. If an anomaly is found, proceed to step S13. If no anomaly is found, proceed to step S14.

[0083] Step S13: Generate a location alarm indicating changes in objects within the target space.

[0084] Step S14: Alarm ends.

[0085] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A real-time alarm method for location anomalies in a metaverse, the method being applied to a server, characterized in that, The methods include, The location request initiated by the client of the user in the target space is obtained. The location request carries contextual information that can locate the user's location. The target space refers to the real space that presents a virtual model of the metaverse. The request image set and device information in the context information are compared with the localization map corresponding to the target space to calculate the pose and obtain the number of points in the pose. The system determines whether a positioning error is due to the number of points within the pose. If a positioning error is due, the system obtains the historical user VPS coordinates of the last successful positioning and determines the user's current range of motion based on the historical user VPS coordinates and device information. An object recognition model is used to identify and generate a set of historical object images of the target spatial point cloud model within the current motion range. The object recognition model is constructed by training a fully convolutional neural network based on the object point cloud data. Anomaly analysis is performed based on the historical object image set and the current object image set in the requested image set. If anomalies are found, a location alarm is generated indicating that there are changes in the object within the target space.

2. The method according to claim 1, characterized in that, The determination of whether a positioning anomaly is caused by the number of points within the pose includes: If the number of points within the pose is greater than or equal to the number threshold, the positioning request is determined to be successful and the positioning is normal. The context information of this positioning request is saved and returned to the client with the current user VPS coordinates obtained from the pose calculation. If the number of points within the pose is less than the number threshold, the localization request is deemed to have failed. When a location request fails, check if the issue is due to changes in non-target objects and if historical user VPS coordinates can be obtained. If the problem is related to changes in objects outside the target space or if the object cannot be obtained, the client will be returned a location request failure message. If the issue is not a change in non-target space objects and can be obtained, then it is judged as a location anomaly.

3. The method according to claim 2, characterized in that, The issue of changes in non-target spatial objects is a problem of insufficient performance of the client device or insufficient system resources.

4. The method according to claim 1, characterized in that, The step of calculating the pose of the requested image set and device information in the context information and the corresponding positioning map in the target space includes: Extract the first two-dimensional feature points of the requested image set and match them with the second two-dimensional features in the positioning map corresponding to the target space to match the target two-dimensional feature points in the second two-dimensional feature points; By binding the target 2D feature points with 3D map points, the matching pairs and correspondences between the first 2D feature points and 3D map points are determined, and pose calculation is performed based on the matching pairs, correspondences, and device information.

5. The method according to claim 1, characterized in that, The device information includes the client user's fastest movement speed, slowest movement speed, and most frequently occurring movement speed in the target space. Determining the user's current movement range based on historical user VPS coordinates and device information includes: Construct a Simpson distribution based on the fastest speed, slowest speed, and most frequently occurring speed. In each round of Monte Carlo simulation, multiple random motion velocities are sampled from the Simpson distribution; Divide the interval between the fastest and slowest speeds into several equal groups; The frequency of random motion speeds in each group is counted and the probability is calculated. The upper limit of the group is determined by the cumulative probability exceeding 99% as the target motion speed. Based on multiple target motion speeds obtained from multiple rounds of Monte Carlo simulation, if the statistical convergence calculated based on multiple target motion speeds is less than 1%, the target motion speed of the current round is confirmed as the user's current speed. Get the time interval between the last successful location request and the location request initiated by the client; The motion radius is determined based on the time interval and the user's current speed; Using the historical user VPS coordinates as the origin, the user's current movement range is determined based on the movement radius.

6. The method according to claim 1, characterized in that, The construction of the object recognition model includes: Acquire the object point cloud data of each historical object in the target space, and label the object point cloud data with the corresponding object category labels to generate training data; The PointNet network or 3D U-Net network is trained based on the training data. The predicted object category is calculated during the forward propagation of the training process. The loss value is calculated based on the predicted object category and the object category label. The weights and biases of the PointNet network or 3D U-Net network are optimized during the backpropagation based on the loss value. The object recognition model is constructed when the preset training stopping conditions are met.

7. The method according to claim 1, characterized in that, The process of using an object recognition model to identify and generate a set of historical object images of the target spatial point cloud model within the current motion range includes: Load the preset target space point cloud model, and cut out the local point cloud data corresponding to the current motion range from the target space point cloud model; The local point cloud data is input into the object recognition model, the point cloud data of historical objects is output, and a set of historical object images is drawn based on the point cloud data of historical objects.

8. The method according to claim 1, characterized in that, The process involves anomaly analysis based on the historical object image set and the current object image set in the requested image set. If anomalies are detected, a location alarm indicating changes in the target space is generated, including: The system performs a similarity analysis between the historical object image set and the current object image set in the requested image set. If the similarity is lower than a preset value, a location alarm is generated indicating that a new object has entered the target space; or Extract the first positional relationship between the target object and its neighboring objects in the historical object image set, and the second positional relationship between the approximate objects that are similar to the target object and its neighboring objects in the current object image set; If the first positional relationship and the second positional relationship are determined to be different, a positioning alarm is generated indicating that a target object is moving within the target space.

9. The method according to claim 1 or 8, after generating the location alarm indicating a change in the target space, the method further includes: Create a real-time spatial video acquisition task and send it to the client via a long link; Receive spatial video sent by the client, model the spatial video, and generate a block texture model and a sparse point cloud model for updating the target spatial point cloud model; Add spatial video, block texture model and sparse point cloud model to alarm logs.

10. A real-time alarm system for location anomalies in a metaverse, characterized in that, The alarm system includes a server and a client; The server obtains a location request initiated by the client of a user in the target space. The location request carries contextual information that can locate the user's location. The target space refers to the real space that presents a virtual model of the metaverse. as well as The request image set and device information in the context information are compared with the localization map corresponding to the target space to calculate the pose and obtain the number of points in the pose. as well as The system determines whether a positioning error is due to the number of points within the pose. If a positioning error is due, the system obtains the historical user VPS coordinates of the last successful positioning and determines the user's current range of motion based on the historical user VPS coordinates and device information. as well as An object recognition model is used to identify and generate a set of historical object images of the target spatial point cloud model within the current motion range. The object recognition model is constructed by training a fully convolutional neural network based on the object point cloud data. as well as Anomaly analysis is performed based on the historical object image set and the current object image set in the requested image set. If anomalies are found, a location alarm is generated indicating that there are changes in the object within the target space.

Images