Method for creating three-dimensional dynamic scene, computer equipment, and storage medium

Abstract

The present application discloses a method for creating a three-dimensional dynamic scene, comprising: determining three-dimensional mapping parameters of at least one image acquisition device by analyzing historical dynamic images; mapping two-dimensional attribute information of a first object in a real-time dynamic image into three-dimensional attribute information based on the three-dimensional mapping parameters, wherein the historical dynamic image or the real-time dynamic image includes a sequence of images and / or video images; adding the three-dimensional attribute information of the first object to a three-dimensional dynamic template; and rendering the three-dimensional dynamic template in real time to obtain a real-time three-dimensional dynamic scene. Compared to three-dimensional static scenes obtained by traditional three-dimensional reconstruction methods, dynamic scenes contain more comprehensive and rich information, which can serve as a basis for analyzing the behavior of dynamic objects in the scene.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and specifically to a method for creating a three-dimensional dynamic scene, a behavior analysis method based on a three-dimensional scene, a data processing method, and corresponding devices, a computer device, and a computer-readable storage medium. Background Art

[0002] Compared with two-dimensional images, three-dimensional images can more accurately reflect the information of the real world. Therefore, it is of great significance to convert the more commonly collected two-dimensional images into three-dimensional images.

[0003] At present, three-dimensional reconstruction methods are usually used to obtain three-dimensional images. Specifically, it refers to obtaining data images of scene objects through a camera, analyzing and processing the images, and then combining computer vision knowledge to obtain three-dimensional information of objects in the real environment.

[0004] The applicant has found in actual applications that simply obtaining a three-dimensional image is still not enough to meet the needs, and it is necessary to provide a solution for creating a three-dimensional dynamic scene. Summary of the Invention

[0005] In view of the above problems, the present application is proposed to provide a method for creating a three-dimensional dynamic scene, a computer device, and a computer-readable storage medium that overcomes the above problems or at least partially solves the above problems.

[0006] According to one aspect of the present application, a method for creating a three-dimensional dynamic scene is provided, comprising:

[0007] determining three-dimensional mapping parameters of at least one image acquisition device by analyzing historical dynamic images;

[0008] Mapping two-dimensional attribute information of a first object in a real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters, wherein the historical dynamic image or the real-time dynamic image includes a sequence image and / or a video image;

[0009] adding the three-dimensional attribute information of the first object to the three-dimensional dynamic template;

[0010] The three-dimensional dynamic template is rendered in real time to obtain a real-time three-dimensional dynamic scene.

[0011] Optionally, determining the three-dimensional mapping parameters of the video device by analyzing historical dynamic images includes:

[0012] Extract single images from historical dynamic images;

[0013] Determine multiple images corresponding to the same object by comparison;

[0014] The three-dimensional mapping parameters of the image acquisition device are determined based on a plurality of images corresponding to the same object.

[0015] Optionally, determining the multiple images corresponding to the same object by comparing includes:

[0016] extracting feature information of the image;

[0017] The feature information of the images is matched, and multiple images corresponding to the same object are determined based on the matching results.

[0018] Optionally, before mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters, the method further includes:

[0019] Multiple processing threads are created in the central processing unit and / or the graphics processing unit, and the multiple processing threads are used to concurrently execute the step of mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters.

[0020] Optionally, before adding the three-dimensional attribute information of the first object to the three-dimensional dynamic template, the method further includes:

[0021] The three-dimensional attribute information object corresponding to the second object in the three-dimensional dynamic scene is added to the three-dimensional dynamic template of the three-dimensional dynamic scene.

[0022] Optionally, the adding the three-dimensional attribute information object corresponding to the second object of the three-dimensional scene to the three-dimensional dynamic template of the three-dimensional dynamic scene includes:

[0023] Call the device information of IoT devices in the IoT system;

[0024] mapping the device information into three-dimensional attribute information according to the three-dimensional mapping parameters as the three-dimensional attribute information corresponding to the second object;

[0025] The three-dimensional attribute information of the Internet of Things device is added to the three-dimensional dynamic template of the three-dimensional dynamic scene.

[0026] Optionally, the device information of the IoT device in the IoT system is called including:

[0027] Identify the location information of the image acquisition device and determine the Internet of Things device corresponding to the location information.

[0028] Optionally, the method further includes:

[0029] By detecting the real-time dynamic image, it is determined that the property of the second object has changed, and the three-dimensional dynamic template is updated.

[0030] Optionally, the determining that an attribute change occurs to the second object by detecting the real-time dynamic image, and updating the three-dimensional dynamic template includes:

[0031] Acquiring two-dimensional attribute information of the second object in the real-time dynamic image;

[0032] mapping the two-dimensional attribute information of the second object into three-dimensional attribute information according to the three-dimensional mapping parameters;

[0033] The three-dimensional attribute information of the second object is used to update the three-dimensional dynamic template.

[0034] Optionally, the three-dimensional attribute information includes position information and posture information.

[0035] Optionally, the image acquisition device includes at least one of the following: a surveillance camera, and an unmanned aerial vehicle device.

[0036] Optionally, the method further includes:

[0037] Acquiring three-dimensional attribute information of at least one first object in the three-dimensional dynamic scene;

[0038] Behavior recognition of the first object is performed based on the three-dimensional attribute information.

[0039] Optionally, the performing behavior recognition of the first object based on the three-dimensional attribute information includes:

[0040] determining behavior information of the first object based on the three-dimensional attribute information;

[0041] An event determination is performed based on the behavior information of the at least one first object, and whether a target event occurs is determined according to the determination result.

[0042] Optionally, the performing behavior recognition of the first object based on the three-dimensional attribute information includes:

[0043] determining behavior information of a plurality of first objects based on the three-dimensional attribute information;

[0044] Based on the behavior information of the multiple first objects, behavior association is performed on the multiple first objects, and whether there is a target relationship is determined according to the association result.

[0045] Optionally, the performing behavior recognition of the first object based on the three-dimensional attribute information includes:

[0046] determining a behavior pattern of the first object based on the three-dimensional attribute information;

[0047] Behavior prediction is performed on the first object based on the behavior pattern.

[0048] Optionally, the method further includes:

[0049] A building information model is obtained from a business system, and a three-dimensional dynamic model is generated based on the building information model.

[0050] This application also provides a behavior analysis method based on a three-dimensional scene, including:

[0051] Rendering a 3D dynamic template based on real-time dynamic images to obtain a real-time 3D dynamic scene;

[0052] Acquiring three-dimensional attribute information of at least one first object in the three-dimensional dynamic scene;

[0053] determining a behavior pattern of the first object based on the three-dimensional attribute information;

[0054] Behavior prediction is performed on the first object based on the behavior pattern.

[0055] Optionally, rendering a three-dimensional dynamic template according to the real-time dynamic image to obtain a real-time three-dimensional dynamic scene includes:

[0056] Mapping two-dimensional attribute information of a first object in a real-time dynamic image into three-dimensional attribute information according to three-dimensional mapping parameters, wherein the historical dynamic image or the real-time dynamic image includes a sequence image and / or a video image;

[0057] adding the three-dimensional attribute information of the first object to the three-dimensional dynamic template;

[0058] The three-dimensional dynamic template is rendered in real time to obtain a real-time three-dimensional dynamic scene.

[0059] Optionally, the method further includes:

[0060] The three-dimensional mapping parameters of at least one image acquisition device are determined by analyzing historical dynamic images.

[0061] This application also provides a method for creating a three-dimensional dynamic scene, including:

[0062] Calling the Internet of Things system to obtain three-dimensional attribute information of a second object in the three-dimensional dynamic scene;

[0063] updating the three-dimensional dynamic template according to the three-dimensional attribute information of the second object;

[0064] Acquire three-dimensional attribute information of the first object and add it to the three-dimensional dynamic template;

[0065] The three-dimensional dynamic template is rendered in real time to obtain a real-time three-dimensional dynamic scene.

[0066] The present application also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements one or more of the above methods when executing the computer program.

[0067] The present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the program implements one or more of the above methods when executed by a processor.

[0068] This application also provides a data processing method, including:

[0069] Acquire at least two 2D image data, wherein the photographed objects presented by the at least two 2D image data have an intersection, and the photographed objects of the at least two 2D image data are not completely the same;

[0070] Mapping the at least two 2D image data to a 3D model to obtain 3D object data;

[0071] Event detection is performed based on the 3D object data.

[0072] The present application also provides a data processing method, wherein the event detection based on the 3D target data includes:

[0073] identifying a plurality of data objects and object attributes of the data objects from the 3D object data;

[0074] establishing associations between the plurality of data objects according to object attributes of the data objects;

[0075] Event determination is performed based on the correlation results.

[0076] Optionally, the performing event detection based on the 3D target data includes:

[0077] identifying a plurality of data objects and object attributes of the data objects from the 3D object data;

[0078] collecting statistics of group characteristics of the plurality of data objects based on the object data of the plurality of data objects;

[0079] Event determination is performed based on the group characteristics.

[0080] Optionally, it also includes:

[0081] Acquire a plurality of the 3D object data, wherein the plurality of the 3D object data are time-series-related;

[0082] Event prediction based on neural network technology.

[0083] Optionally, before performing event prediction based on the neural network technology, the method further includes:

[0084] Creating a neural network model based on a plurality of data objects included in the historical 3D target data and target events corresponding to the 3D target data;

[0085] The event prediction based on neural network technology includes:

[0086] Event prediction is performed based on the neural network model and the 3D target data.

[0087] This application also provides a data processing method, including:

[0088] Acquire at least two 2D image data, wherein the at least two 2D image data are from cameras located at different positions or angles and photographing the same spatial area;

[0089] Mapping the at least two 2D image data to a 3D model to obtain 3D object data;

[0090] Event detection is performed based on the 3D object data.

[0091] This application also provides a data processing method, including:

[0092] Acquiring at least two 2D image data, wherein the at least two 2D image data are not completely identical;

[0093] Mapping the at least two 2D image data to a 3D model to obtain 3D object data;

[0094] Event detection is performed based on the 3D object data.

[0095] According to the embodiments of the present application, dynamic images such as sequence images and video images are analyzed to determine the three-dimensional mapping parameters of the image acquisition device, so that the two-dimensional attribute information of the first object in the real-time dynamic image can be mapped into three-dimensional attribute information based on the three-dimensional mapping parameters, and the three-dimensional dynamic template with the added three-dimensional attribute information can be further rendered in real time to obtain a real-time three-dimensional dynamic scene. Compared with the three-dimensional static scene obtained by the traditional three-dimensional reconstruction method, the information contained in the dynamic scene is more comprehensive and rich, and can be used as a basis for analyzing the behavior of dynamic objects in the scene.

[0096] The three-dimensional dynamic template of the present application can be added with information from IoT devices in the IoT system, enriching the types of information provided in the three-dimensional dynamic template and making the restoration of the three-dimensional dynamic scene more realistic.

[0097] The present application also creates multiple processing threads in the central processing unit and / or graphics processing unit, thereby speeding up the processing of dynamic images and ensuring the consistency of the rendering effect of the three-dimensional dynamic scene.

[0098] The three-dimensional dynamic scene obtained based on rendering can be further used for scene cognitive analysis, and scene monitoring can be achieved by analyzing the behavior of objects in the scene.

[0099] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0101] Figure 1 A flowchart of an embodiment of a method for creating a three-dimensional dynamic scene according to the first embodiment of the present application is shown;

[0102] Figure 2 A flowchart of an embodiment of a method for creating a three-dimensional dynamic scene according to the second embodiment of the present application is shown;

[0103] Figure 3 A flowchart of an embodiment of a method for creating a three-dimensional dynamic scene according to the third embodiment of the present application is shown;

[0104] Figure 4 A flowchart of a behavior analysis method based on a three-dimensional scene according to the fourth embodiment of the present application is shown;

[0105] Figure 5 A flowchart of an embodiment of a method for creating a three-dimensional dynamic scene according to the fifth embodiment of the present application is shown;

[0106] Figure 6 A schematic diagram of a process for obtaining three-dimensional mapping parameters in an example of the present application is shown;

[0107] Figure 7 A schematic diagram of creating a three-dimensional dynamic scene in an example of the present application is shown;

[0108] Figure 8 A schematic diagram of a behavior analysis solution based on a three-dimensional scene in an example of the present application is shown;

[0109] Figure 9 A structural block diagram of an embodiment of a device for creating a three-dimensional dynamic scene according to a sixth embodiment of the present application is shown;

[0110] Figure 10 A structural block diagram of an embodiment of a behavior analysis device based on a three-dimensional scene according to the seventh embodiment of the present application is shown;

[0111] Figure 11 A structural block diagram of an embodiment of a device for creating a three-dimensional dynamic scene according to an eighth embodiment of the present application is shown;

[0112] Figure 12 A flowchart of a data processing method according to a ninth embodiment of the present application is shown;

[0113] Figure 13 A flowchart of a data processing method according to the tenth embodiment of the present application is shown;

[0114] Figure 14 A flowchart of a data processing method according to the eleventh embodiment of the present application is shown;

[0115] Figure 15 An exemplary system is shown that can be used to implement the various embodiments described in this disclosure. DETAILED DESCRIPTION

[0116] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0117] To help those skilled in the art better understand this application, the concepts involved in this application are explained below:

[0118] This application renders a three-dimensional dynamic scene based on dynamic images. Compared with a three-dimensional static scene, the three-dimensional dynamic scene provides richer information, such as the movement of dynamic objects, which can serve as the basis for object behavior analysis.

[0119] The scenes described in this application can be physical scenes, such as streets, indoor environments, etc., or virtual scenes.

[0120] The dynamic images described in this application include multiple images, specifically, a sequence of images, or a video image, or a combination of a sequence of images and a video image. Therefore, the historical dynamic images described in this application include one or more of a sequence of images and a video image, and the real-time dynamic images include one or more of a sequence of images and a video image.

[0121] Sequence images include multiple images in order, such as multiple images captured sequentially by devices such as drones, and multiple images captured sequentially by mobile terminals using continuous shooting. Video images include images captured by video acquisition devices (such as surveillance cameras, mobile terminals, etc.).

[0122] Dynamic images are obtained through image acquisition devices. Image acquisition devices can include devices for acquiring sequence images such as drones or video acquisition devices for acquiring videos. One scene can correspond to multiple image acquisition devices of the same or different types.

[0123] The image objects included in the dynamic image are divided into a first object and a second object. One optional division method of the present application is that the first object includes the foreground object and the second object includes the background object. The foreground object is the subject of the image expression or setting, generally an object located close to the lens and in motion, and the background object is generally the environment in which the foreground object is located. The specific content of the first object and the second object can also be set according to actual business needs, and this application does not impose any restrictions on this.

[0124] For example, in a three-dimensional dynamic scene of a street, static objects such as street buildings, public facilities, and greenery serve as background objects, while dynamic objects such as pedestrians and vehicles on the street serve as foreground objects.

[0125] The attribute information of the first object or the second object in a two-dimensional scene is called two-dimensional attribute information, which may include one or more appearance features such as position information, posture information, shape and texture. Three-dimensional attribute information refers to the attribute information of an image object in a three-dimensional scene, which has an additional third dimension of information compared to two-dimensional attribute information, namely depth information. Therefore, mapping two-dimensional attribute information to three-dimensional attribute information is to project the two-dimensional attribute information into the third dimension through three-dimensional mapping parameters. The three-dimensional mapping parameters determine the projection relationship of the image acquisition device from the three-dimensional scene to the two-dimensional image, and may include the device parameters of the image acquisition device, such as the motion parameters of the camera. If there are multiple image acquisition devices, the three-dimensional mapping parameters may also include the geometric relationship between the cameras.

[0126] The 3D dynamic template is a general template created based on a dynamic scene. The 3D attribute information of the first object is added to the 3D dynamic template as a texture, and a 3D dynamic scene can be rendered.

[0127] The 3D dynamic template can be supplemented with 3D attribute information of a second object. This 3D attribute information of the second object can be derived from information about IoT devices in the IoT system, enriching the types of information provided in the 3D dynamic template and making the restoration of the 3D dynamic scene more realistic. The IoT system connects multiple IoT devices based on the IoT and provides device information for multiple IoT devices. This device information is used as the second attribute information of the second object in the 3D dynamic scene in the present application, and is mapped to 3D attribute information based on 3D mapping parameters. The 3D attribute information of the second object is then added to the 3D dynamic template.

[0128] This application first analyzes existing historical dynamic images to obtain 3D mapping parameters for the image acquisition device, which are then used to perform 3D mapping of real-time dynamic images. The 3D attribute information obtained through mapping is added to a 3D dynamic template and rendered in real time. By using dynamic images to map 3D attribute information in real time, a real-time 3D dynamic scene can be generated, thus providing an innovative method for generating 3D dynamic scenes. The dynamic scene contains more comprehensive and rich information, which can be further used for behavioral analysis of dynamic objects.

[0129] In order to speed up the three-dimensional processing of dynamic images, this application also creates multiple processing threads in the central processing unit and / or graphics processor to speed up the processing of dynamic images and ensure the consistency of the rendering effect of the three-dimensional dynamic scene.

[0130] The present application can also monitor the first object in the three-dimensional dynamic scene. When the properties of the first object change, the three-dimensional dynamic template can be updated according to the changed properties, thereby achieving timely update of the three-dimensional dynamic template.

[0131] Reference Figure 1 , shows a flowchart of an embodiment of a method for creating a three-dimensional dynamic scene according to the first embodiment of the present application. The method may specifically include the following steps:

[0132] Step 101: Determine three-dimensional mapping parameters of at least one image acquisition device by analyzing historical dynamic images.

[0133] The dynamic images captured by the image acquisition device are two-dimensional images. This application analyzes multiple ordered two-dimensional images to obtain three-dimensional mapping parameters.

[0134] The three-dimensional mapping parameters represent the mapping relationship between the two-dimensional attribute information and the three-dimensional attribute information. The two-dimensional attribute information can be mapped into the three-dimensional attribute information according to the three-dimensional mapping parameters.

[0135] The scheme for determining three-dimensional mapping parameters in an embodiment of the present application may include, first, extracting a single image from historical dynamic images, further determining multiple images corresponding to the same object by comparing the images, and then determining the three-dimensional mapping parameters of the image acquisition device based on the multiple images corresponding to the same object.

[0136] When comparing multiple images corresponding to the same object, feature information of the images can be extracted and matched. Based on the matching results, the multiple images corresponding to the same object can be determined. Feature information can include scale-invariant feature transform (SIFT). Image feature information is extracted using the SIFT algorithm, and matching is performed. Images whose feature information meets the matching requirements are considered multiple images corresponding to the same object. The matching conditions are set in the SIFT algorithm and can be configured according to actual needs.

[0137] In this application, the present embodiment employs the SFM (Structure from Motion) algorithm to obtain 3D mapping parameters. The SFM algorithm uses camera movement to determine spatial and geometric relationships. Using the SFM method, the 3D mapping parameters are iteratively solved, and then used to obtain 3D attribute information, thereby reconstructing a sparse point cloud of the 3D scene.

[0138] In actual processing, after obtaining feature information, a scale reconstruction can be performed based on the actual position of the image acquisition device, converting the feature information data scale from the camera coordinate system to the world coordinate system. Furthermore, due to the limited number of data points corresponding to the feature information, only sparse point cloud data is obtained. To achieve a more realistic 3D scene, the sparse point cloud can be densified, and feature information matching can be performed on the densified point cloud again. The 3D mapping parameters of the image acquisition device can then be determined using the MVS (Multi-View Stereo) algorithm combined with the SFM algorithm.

[0139] Step 102 : Mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters.

[0140] In the embodiments of the present application, the two-dimensional attribute information may include position information and posture information, and may also include other information such as the color information and identity information of the first object, correspondingly representing the position and state of the first object in the two-dimensional image. By performing structured analysis on the image, the first object in the image can be identified and the two-dimensional attribute information of the first object can be further extracted.

[0141] Step 103: Add the three-dimensional attribute information of the first object to the three-dimensional dynamic template.

[0142] Step 104: Render the three-dimensional dynamic template in real time to obtain a real-time three-dimensional dynamic scene.

[0143] Reference Figure 2 , shows a flowchart of an embodiment of a method for creating a three-dimensional dynamic scene according to the third embodiment of the present application. The method may specifically include the following steps:

[0144] Step 201: Determine three-dimensional mapping parameters of at least one image acquisition device by analyzing historical dynamic images.

[0145] Step 202: Create multiple processing threads in the central processing unit and / or the graphics processing unit.

[0146] Step 203 : Mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters in the multiple processing threads.

[0147] This application involves real-time processing of large numbers of images. To ensure the rendering effect of three-dimensional dynamic scenes, multiple processing threads can be created to perform concurrent processing, thereby accelerating the processing of dynamic images. These multiple threads can be created simultaneously in the central processing unit (CPU) and graphics processing unit (GPU), and through heterogeneous processing, processing efficiency can be further improved.

[0148] Step 204: Add the three-dimensional attribute information object corresponding to the second object in the three-dimensional dynamic scene to the three-dimensional dynamic template of the three-dimensional dynamic scene.

[0149] In this embodiment, the three-dimensional attribute information of the second object can be added to the three-dimensional dynamic template, which enriches the types of information provided in the three-dimensional dynamic template and makes the restoration of the three-dimensional dynamic scene more realistic.

[0150] Specifically, the device information of the Internet of Things device in the Internet of Things system can be called, the device information is used as the two-dimensional attribute information, and the device information is mapped into three-dimensional attribute information according to the three-dimensional mapping parameters, as the three-dimensional attribute information corresponding to the second object, and the three-dimensional attribute information of the Internet of Things device is added to the three-dimensional dynamic template of the three-dimensional dynamic scene.

[0151] When obtaining device information of IoT devices from the IoT system, it is first necessary to confirm which IoT devices belong to the current scene. Therefore, the location information of the image acquisition device can be identified and the IoT devices with corresponding location information can be determined, that is, the IoT devices in the same area as the image acquisition device can be determined.

[0152] Step 205: Add the three-dimensional attribute information of the first object to the three-dimensional dynamic template.

[0153] It should be noted that the steps of adding the three-dimensional attribute information of the first object and the three-dimensional attribute information of the second object to the three-dimensional dynamic template can be set in an execution order according to actual needs, and this application does not impose any restrictions on this.

[0154] Step 206: Render the three-dimensional dynamic template in real time to obtain a real-time three-dimensional dynamic scene.

[0155] Step 207: determining that the property of the second object has changed by detecting the real-time dynamic image, and updating the three-dimensional dynamic template.

[0156] For the second object whose three-dimensional attribute information is added to the three-dimensional dynamic template, if a change in its attribute is detected, the three-dimensional dynamic template needs to be updated.

[0157] By analyzing the real-time dynamic image, the two-dimensional attribute information of the second object in the real-time dynamic image can be obtained, and the two-dimensional attribute information of the second object can be further mapped into three-dimensional attribute information according to the three-dimensional mapping parameters, and the three-dimensional attribute information in the three-dimensional dynamic template can be updated using the re-determined three-dimensional attribute information.

[0158] Reference Figure 3 , shows a flowchart of an embodiment of a method for creating a three-dimensional dynamic scene according to the third embodiment of the present application. The method may specifically include the following steps:

[0159] Step 301: Determine three-dimensional mapping parameters of at least one image acquisition device by analyzing historical dynamic images.

[0160] Step 302 : Mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters, wherein the historical dynamic image or the real-time dynamic image includes a sequence image and / or a video image.

[0161] Step 303: Add the three-dimensional attribute information of the first object to the three-dimensional dynamic template.

[0162] Step 304: Render the three-dimensional dynamic template in real time to obtain a real-time three-dimensional dynamic scene.

[0163] Step 305: Acquire three-dimensional attribute information of at least one first object in the three-dimensional dynamic scene.

[0164] Step 306: Perform behavior recognition of the first object based on the three-dimensional attribute information.

[0165] The above-mentioned embodiment 1 and embodiment 2 provide a method for creating a three-dimensional dynamic scene. This embodiment can identify the behavior of an object based on the rendered three-dimensional dynamic scene, that is, realize the cognition of the three-dimensional scene and realize scene monitoring.

[0166] A three-dimensional dynamic scene consists of multiple three-dimensional scenes with a time sequence. The first object has corresponding three-dimensional attribute information in each three-dimensional scene and will change in multiple three-dimensional scenes. Therefore, by analyzing multiple ordered three-dimensional scenes, the behavior information of the first object can be obtained, and then the behavior of the first object can be identified.

[0167] First, the first object is detected in the three-dimensional dynamic scene. Specifically, a deep learning algorithm can be used to achieve high-precision detection.

[0168] Behavior recognition can be performed on a single first object or on a collective of multiple first objects. For example, in a street scene, where the first object includes a person, the walking trajectory of a single first object on the street can be monitored to determine whether the individual has the behavioral characteristics of running fast. The walking trajectories of two people can also be identified to determine whether there is any abnormal tailing or accompanying behavior between the two people. The walking trajectories of multiple people can also be identified to determine whether a group of people has the behavioral characteristics of abnormal gathering.

[0169] In one example, behavior recognition is performed on a single first object. The behavior information of the first object can be determined based on the three-dimensional attribute information. An event judgment is further performed based on the behavior information of at least one first object, and whether the target event has occurred is determined based on the judgment result. The three-dimensional attribute information may include the position information and posture information of the first object, and the behavior information may include the position information, speed information, direction information, action information, etc. of the first object. The judgment of the behavior information can be combined with the needs of specific business scenarios. For example, if the security management department needs to monitor the safety of the first object, corresponding judgment rules can be set for behavior information such as speed and direction; if the traffic department needs to monitor the behavior speed and behavior range of the first object, corresponding judgment rules can be set for behavior speed and behavior range; new retail businesses (such as takeout and express delivery businesses) monitor the behavior trajectory and corresponding judgment rules can be set for their behavior trajectory.

[0170] Furthermore, after determining the target event, further event classification processing can be performed, for example, classification according to the urgency of the event, or classification according to the region where the event occurred, so as to speed up the processing efficiency of the event and optimize the processing results of the event.

[0171] In another example, behavior recognition can be performed on multiple behavioral objects. Behavior information of multiple first objects can be determined based on three-dimensional attribute information. Based on the behavior information of the multiple first objects, the multiple first objects can be behaviorally associated, and whether a target relationship exists can be determined based on the association results. For example, the behavior trajectories of multiple pedestrians can be monitored, and corresponding judgment rules can be set based on the behavior trajectories of the multiple pedestrians to identify the relationship between the multiple people, such as whether they are companions or strangers, and then determine whether an abnormal tailgating incident has occurred; or the relationship between the multiple people can be identified as a group or strangers, and then determine whether an abnormal gathering incident has occurred.

[0172] Among them, after identifying the relationship between multiple first objects, the relationship between the first objects can be described in the form of a graph, with the first objects as nodes and the relationship between the first objects as edges. The graph is combined with a neural network model for inference and prediction to obtain the relationship between the first objects.

[0173] After determining the target relationship, the target relationship can be further classified. For example, the target relationship can be classified according to the probability that the target relationship is a stranger, and stranger classifications with different levels of probability can be obtained. Therefore, corresponding processing measures can be adopted according to the classification of the relationship between the first objects to handle the following event.

[0174] In the above two examples, a target tracking algorithm can be used to collect three-dimensional attribute information of the first object in multiple ordered three-dimensional scenes, and the behavior information of the first object can be determined based on the three-dimensional attribute information, and structured data about the first object can be further established based on the three-dimensional attribute information and behavior information.

[0175] In another example, the behavior pattern of the first object can be determined based on the 3D attribute information, and then the behavior of the first object can be predicted based on the behavior pattern. A deep learning algorithm is used to learn from multiple ordered 3D scenes, extracting the 3D attribute information from each scene to form structured data. A graph neural network is then used to learn from this structured data and infer the evolutionary patterns within it, i.e., the behavior patterns. Based on these evolutionary patterns, the real-time 3D scene can be deduced to predict future behavior.

[0176] Among them, the three-dimensional dynamic model can be generated according to the building information model (Building Information Modeling) of the business system. In the security management system, transportation system, and new retail system, there are existing three-dimensional models, namely building information models. By adjusting the coordinate mapping rules of the building information model, the position mapping of the two-dimensional attribute information in the dynamic image and the building information model can be achieved. The adjustment of the coordinate mapping rules of the building information model, for example, marks multiple groups of two-dimensional pictures (from camera shots) and matches the point pairs in the three-dimensional model (the position of the same point in the two-dimensional picture and the three-dimensional model), and solves the matrix H through the transformation relationship between the three-dimensional world coordinate system and the two-dimensional pixel coordinate system. Specifically, the three-dimensional world coordinate = matrix H * pixel coordinate, where the matrix H is the internal and external parameters of the image acquisition device.

[0177] Reference Figure 4 , shows a flowchart of an embodiment of a behavior analysis method based on a three-dimensional scene according to the fourth embodiment of the present application. The method may specifically include the following steps:

[0178] Step 401 : Render a three-dimensional dynamic template according to the real-time dynamic image to obtain a real-time three-dimensional dynamic scene.

[0179] This step can be implemented using the solutions of the above embodiments 1-2, and will not be described again here.

[0180] Step 402: Acquire three-dimensional attribute information of at least one first object in the three-dimensional dynamic scene.

[0181] Step 403: Determine a behavior pattern of the first object based on the three-dimensional attribute information.

[0182] Step 404: predict the behavior of the first object based on the behavior pattern.

[0183] In an embodiment of the present application, preferably, the rendering of a three-dimensional dynamic template based on a real-time dynamic image to obtain a real-time three-dimensional dynamic scene includes: mapping the two-dimensional attribute information of a first object in the real-time dynamic image into three-dimensional attribute information according to three-dimensional mapping parameters, wherein the historical dynamic image or the real-time dynamic image includes a sequence image and / or a video image; adding the three-dimensional attribute information of the first object to the three-dimensional dynamic template; and rendering the three-dimensional dynamic template in real time to obtain a real-time three-dimensional dynamic scene.

[0184] In the embodiment of the present application, preferably, the method further includes:

[0185] The three-dimensional mapping parameters of at least one image acquisition device are determined by analyzing historical dynamic images.

[0186] The specific implementation of steps 402-404 and their sub-steps can refer to the solution of the above embodiment 3 and will not be repeated here.

[0187] According to the embodiments of the present application, dynamic images such as sequence images and video images are analyzed to determine the three-dimensional mapping parameters of the image acquisition device, so that the two-dimensional attribute information of the first object in the real-time dynamic image can be mapped into three-dimensional attribute information based on the three-dimensional mapping parameters, and the three-dimensional dynamic template with the added three-dimensional attribute information can be further rendered in real time to obtain a real-time three-dimensional dynamic scene. Compared with the three-dimensional static scene obtained by the traditional three-dimensional reconstruction method, the information contained in the dynamic scene is more comprehensive and rich, and can be used as a basis for analyzing the behavior of dynamic objects in the scene.

[0188] The three-dimensional dynamic scene obtained based on rendering can be further used for scene cognitive analysis, and scene monitoring can be achieved by analyzing the behavior of objects in the scene.

[0189] Reference Figure 5 , shows a flowchart of an embodiment of a method for creating a three-dimensional dynamic scene according to the fifth embodiment of the present application. The method may specifically include the following steps:

[0190] Step 501: Call the Internet of Things system to obtain three-dimensional attribute information of a second object in a three-dimensional dynamic scene.

[0191] Step 502: Update the 3D dynamic template according to the 3D attribute information of the second object.

[0192] Step 503: Acquire the three-dimensional attribute information of the first object and add it to the three-dimensional dynamic template.

[0193] Step 504: Render the three-dimensional dynamic template in real time to obtain a real-time three-dimensional dynamic scene.

[0194] According to an embodiment of the present application, a 3D dynamic template with the 3D attribute information of a second object in the IoT system is used. The 3D attribute information of the first object is further added to the 3D dynamic template. Real-time rendering of the 3D dynamic template yields a real-time 3D dynamic scene. Compared to 3D static scenes obtained using traditional 3D reconstruction methods, the dynamic scene contains more comprehensive and rich information, which can serve as a basis for analyzing the behavior of dynamic objects within the scene. The addition of information from IoT devices in the IoT system enriches the variety of information provided by the 3D dynamic template, making the restoration of the 3D dynamic scene more realistic.

[0195] In order to enable those skilled in the art to better understand the present application, a method for creating a three-dimensional dynamic scene of the present application is described below through specific examples.

[0196] Reference Figure 6, which shows a schematic diagram of a process for obtaining three-dimensional mapping parameters in an example of the present application, Figure 7 A schematic diagram of creating a three-dimensional dynamic scene in an example of the present application is shown, specifically including:

[0197] 1. Obtaining 3D mapping parameters

[0198] 1. Image acquisition steps

[0199] Acquire drone images and surveillance video images to obtain an image set.

[0200] 2. Feature extraction

[0201] Extract feature information of each image in an image collection.

[0202] 3. Parallel matching

[0203] Multi-threaded execution of feature information matching operations is established on the CPU and GPU to determine multiple images corresponding to the same object.

[0204] 4. SFM

[0205] For multiple images, a sparse point cloud of the three-dimensional scene is constructed based on the SFM algorithm and the real position information of the image acquisition device.

[0206] 5. Feature point area expansion

[0207] The sparse point cloud is densified to obtain a dense point cloud.

[0208] 6. Parallel matching

[0209] Multithreading is used to perform feature information matching operations on dense point clouds.

[0210] 7. MVS surface reconstruction

[0211] The matched dense point cloud is operated by the MVS algorithm to obtain the three-dimensional attribute information of the image acquisition device.

[0212] 2. Create 3D dynamic scenes

[0213] 1. Monitoring video stream

[0214] Real-time video images are acquired through a video acquisition device, and images are extracted one by one for the following processing.

[0215] 2. Image structured analysis

[0216] A first object in the dynamic image is identified, and two-dimensional attribute information of the first object is further obtained.

[0217] 3. Intelligent data analysis and integration

[0218] The two-dimensional attribute information of the second object is obtained from the Internet of Things system and mapped into three-dimensional attribute information.

[0219] 4. Rendering target 3D model

[0220] The three-dimensional attribute information of the second object is added to the 3D model.

[0221] 5. Project and update the 3D scene

[0222] Rendering the above 3D model will produce a 3D dynamic scene. Executing the loop processing of the next chapter image will further produce a real-time 3D dynamic scene.

[0223] refer to Figure 8 A schematic diagram of a behavior analysis solution based on a three-dimensional scene in an example of this application is shown, specifically including:

[0224] 1. Mapping

[0225] Multiple 2D images from different angles are mapped to a 3D scene.

[0226] 2. Understanding

[0227] For 3D scenes, event detection, multi-target association and crowd density detection are performed.

[0228] 3. Video Inference 3D to 4D

[0229] Acquire multiple time-series 3D scenes, use graphs combined with neural networks to infer future situation data.

[0230] 4. BIM information docking

[0231] Utilizing the existing BIM model in the system, combined with stereo vision technology (MVS), 3D reconstruction is performed to obtain 3D mapping parameters, and 3D dynamic scenes are generated using 3D mapping methods. Further analysis can be performed on passenger flow, anomaly detection, lost person search, and rescue plans.

[0232] Reference Figure 9 , shows a structural block diagram of an embodiment of a device for creating a three-dimensional dynamic scene according to the sixth embodiment of the present application, which may specifically include:

[0233] A parameter determination module 601 is configured to determine three-dimensional mapping parameters of at least one image acquisition device by analyzing historical dynamic images;

[0234] A mapping module 602 is configured to map the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters, wherein the historical dynamic image or the real-time dynamic image includes a sequence image and / or a video image;

[0235] A first attribute adding module 603 is configured to add the three-dimensional attribute information of the first object to the three-dimensional dynamic template;

[0236] The rendering module 604 is configured to render the 3D dynamic template in real time to obtain a real-time 3D dynamic scene.

[0237] In a preferred embodiment of the present application, the parameter determination module includes:

[0238] Single image extraction submodule, used to extract single images from historical dynamic images;

[0239] A multiple image determination submodule is used to determine multiple images corresponding to the same object through comparison;

[0240] The parameter calculation submodule is used to determine the three-dimensional mapping parameters of the image acquisition device based on multiple images corresponding to the same object.

[0241] In a preferred embodiment of the present application, the multiple image determination submodule is specifically used to extract feature information of the image; match the feature information of the image, and determine multiple images corresponding to the same object based on the matching results.

[0242] In a preferred embodiment of the present application, the device further comprises:

[0243] A thread creation module is used to create multiple processing threads in the central processing unit and / or graphics processing unit, and the multiple processing threads are used to concurrently execute the step of mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information based on the three-dimensional mapping parameters.

[0244] In a preferred embodiment of the present application, the device further comprises:

[0245] The second attribute adding module is used to add the three-dimensional attribute information object corresponding to the second object in the three-dimensional dynamic scene to the three-dimensional dynamic template of the three-dimensional dynamic scene.

[0246] In a preferred embodiment of the present application, the second attribute adding module includes:

[0247] The device information calling submodule is used to call the device information of the IoT devices in the IoT system;

[0248] a device information mapping submodule, configured to map the device information into three-dimensional attribute information according to the three-dimensional mapping parameters, as the three-dimensional attribute information corresponding to the second object;

[0249] The information adding submodule is used to add the three-dimensional attribute information of the Internet of Things device to the three-dimensional dynamic template of the three-dimensional dynamic scene.

[0250] In a preferred embodiment of the present application, the device information calling submodule is specifically used to identify the location information of the image acquisition device and determine the Internet of Things device corresponding to the location information.

[0251] In a preferred embodiment of the present application, the device further comprises:

[0252] The template updating module is used to determine that the property of the second object has changed by detecting the real-time dynamic image, and update the three-dimensional dynamic template.

[0253] In a preferred embodiment of the present application, the template updating module includes:

[0254] a two-dimensional attribute information acquisition submodule, configured to acquire the two-dimensional attribute information of the second object in the real-time dynamic image;

[0255] a two-dimensional information mapping submodule, configured to map the two-dimensional attribute information of the second object into three-dimensional attribute information according to the three-dimensional mapping parameters;

[0256] The template updating submodule is configured to update the three-dimensional dynamic template using the three-dimensional attribute information of the second object.

[0257] In a preferred embodiment of the present application, the three-dimensional attribute information includes position information and posture information.

[0258] In a preferred embodiment of the present application, the image acquisition device includes at least one of the following: a surveillance camera, and a drone device.

[0259] In a preferred embodiment of the present application, the device further comprises:

[0260] a three-dimensional attribute information acquisition module, configured to acquire three-dimensional attribute information of at least one first object in the three-dimensional dynamic scene;

[0261] A behavior recognition module is used to perform behavior recognition of the first object based on the three-dimensional attribute information.

[0262] In a preferred embodiment of the present application, the behavior recognition module includes:

[0263] a first behavior information determining submodule, configured to determine behavior information of the first object according to the three-dimensional attribute information;

[0264] The behavior information determination submodule is configured to perform event determination based on the behavior information of the at least one first object, and determine whether a target event occurs according to the determination result.

[0265] In a preferred embodiment of the present application, the behavior recognition module includes:

[0266] a second behavior information determining submodule, configured to determine behavior information of a plurality of first objects according to the three-dimensional attribute information;

[0267] The relationship determination submodule is configured to perform behavior association on the plurality of first objects based on the behavior information of the plurality of first objects, and determine whether there is a target relationship according to the association result.

[0268] In a preferred embodiment of the present application, the behavior recognition module includes:

[0269] a mode determination submodule, configured to determine a behavior mode of the first object based on the three-dimensional attribute information;

[0270] A behavior prediction submodule is used to predict the behavior of the first object based on the behavior pattern.

[0271] In a preferred embodiment of the present application, the device further comprises:

[0272] The model generation module is used to obtain a building information model from a business system and generate a three-dimensional dynamic model based on the building information model.

[0273] According to the embodiments of the present application, dynamic images such as sequence images and video images are analyzed to determine the three-dimensional mapping parameters of the image acquisition device, so that the two-dimensional attribute information of the first object in the real-time dynamic image can be mapped into three-dimensional attribute information based on the three-dimensional mapping parameters, and the three-dimensional dynamic template with the added three-dimensional attribute information can be further rendered in real time to obtain a real-time three-dimensional dynamic scene. Compared with the three-dimensional static scene obtained by the traditional three-dimensional reconstruction method, the information contained in the dynamic scene is more comprehensive and rich, and can be used as a basis for analyzing the behavior of dynamic objects in the scene.

[0274] The three-dimensional dynamic template of the present application can be added with information from IoT devices in the IoT system, enriching the types of information provided in the three-dimensional dynamic template and making the restoration of the three-dimensional dynamic scene more realistic.

[0275] The present application also creates multiple processing threads in the central processing unit and / or graphics processing unit, thereby speeding up the processing of dynamic images and ensuring the consistency of the rendering effect of the three-dimensional dynamic scene.

[0276] The three-dimensional dynamic scene obtained based on rendering can be further used for scene cognitive analysis, and scene monitoring can be achieved by analyzing the behavior of objects in the scene.

[0277] Reference Figure 10 , shows a structural block diagram of an embodiment of a behavior analysis device based on a three-dimensional scene according to the seventh embodiment of the present application, which may specifically include:

[0278] The scene rendering module 701 is used to render a 3D dynamic template according to the real-time dynamic image to obtain a real-time 3D dynamic scene;

[0279] An attribute information acquisition module 702 is configured to acquire three-dimensional attribute information of at least one first object in the three-dimensional dynamic scene;

[0280] A behavior pattern determination module 703 is configured to determine a behavior pattern of the first object based on the three-dimensional attribute information;

[0281] The behavior prediction module 704 is configured to perform behavior prediction on the first object based on the behavior pattern.

[0282] In a preferred embodiment of the present application, the scene rendering module includes:

[0283] an attribute information mapping submodule, configured to map two-dimensional attribute information of a first object in a real-time dynamic image into three-dimensional attribute information according to three-dimensional mapping parameters, wherein the historical dynamic image or the real-time dynamic image includes a sequence image and / or a video image;

[0284] an attribute information adding submodule, configured to add the three-dimensional attribute information of the first object to the three-dimensional dynamic template;

[0285] The rendering submodule is used to render the three-dimensional dynamic template in real time to obtain a real-time three-dimensional dynamic scene.

[0286] In a preferred embodiment of the present application, the device further comprises:

[0287] The parameter determination module is used to determine the three-dimensional mapping parameters of at least one image acquisition device by analyzing historical dynamic images.

[0288] According to the embodiments of the present application, dynamic images such as sequence images and video images are analyzed to determine the three-dimensional mapping parameters of the image acquisition device, so that the two-dimensional attribute information of the first object in the real-time dynamic image can be mapped into three-dimensional attribute information based on the three-dimensional mapping parameters, and the three-dimensional dynamic template with the added three-dimensional attribute information can be further rendered in real time to obtain a real-time three-dimensional dynamic scene. Compared with the three-dimensional static scene obtained by the traditional three-dimensional reconstruction method, the information contained in the dynamic scene is more comprehensive and rich, and can be used as a basis for analyzing the behavior of dynamic objects in the scene.

[0289] The three-dimensional dynamic scene obtained based on rendering can be further used for scene cognitive analysis, and scene monitoring can be achieved by analyzing the behavior of objects in the scene.

[0290] Reference Figure 11 , shows a structural block diagram of an embodiment of a device for creating a three-dimensional dynamic scene according to the eighth embodiment of the present application, which may specifically include:

[0291] The three-dimensional information acquisition module 801 is used to call the Internet of Things system to obtain three-dimensional attribute information of the second object in the three-dimensional dynamic scene;

[0292] A template updating module 802 is configured to update the 3D dynamic template according to the 3D attribute information of the second object;

[0293] An information adding module 803 is used to obtain three-dimensional attribute information of the first object and add it to the three-dimensional dynamic template;

[0294] The scene rendering module 804 is used to render the 3D dynamic template in real time to obtain a real-time 3D dynamic scene.

[0295] According to an embodiment of the present application, a 3D dynamic template with the 3D attribute information of a second object in the IoT system is used. The 3D attribute information of the first object is further added to the 3D dynamic template. Real-time rendering of the 3D dynamic template yields a real-time 3D dynamic scene. Compared to 3D static scenes obtained using traditional 3D reconstruction methods, the dynamic scene contains more comprehensive and rich information, which can serve as a basis for analyzing the behavior of dynamic objects within the scene. The addition of information from IoT devices in the IoT system enriches the variety of information provided by the 3D dynamic template, making the restoration of the 3D dynamic scene more realistic.

[0296] Reference Figure 12 , shows a flow chart of an embodiment of a data processing method according to Embodiment 9 of the present application, which method may specifically include the following steps:

[0297] Step 901: Acquire at least two 2D image data, wherein the photographed objects presented by the at least two 2D image data have an intersection, and the photographed objects of the at least two 2D image data are not completely the same.

[0298] Among them, 2D image data may include serial images collected by equipment such as drones, or video images collected by video equipment such as cameras, and this application does not impose any restrictions on this.

[0299] The at least two 2D image data items are different images captured from the same scene. Therefore, the corresponding captured objects overlap and are not identical. This may be due to different angles or changes in the subject matter of the scene due to different capture times. The images may have been captured using the same device or different devices.

[0300] Step 902: Map the at least two 2D image data to a 3D model to obtain 3D target data.

[0301] Step 903: Perform event detection based on the 3D target data.

[0302] In one example, multiple data objects in 3D data can be identified, and the association between the multiple data objects can be determined. Specifically, when performing event detection based on the 3D target data, multiple data objects and object attributes of the data objects, such as behavioral information of the data objects, are identified from the 3D target data; associations between the multiple data objects are established based on the object attributes of the data objects; and event determination is performed based on the association results. For example, the behavior trajectories of multiple pedestrians can be monitored, and corresponding determination rules can be set based on the behavior trajectories of the multiple pedestrians to identify the relationship between the multiple people, such as whether they are companions or strangers, and then determine whether an abnormal following incident has occurred; or identify whether the relationship between the multiple people is a collective or stranger relationship, and then determine whether an abnormal gathering incident has occurred.

[0303] In another example, multiple data objects in 3D data may be identified, and event detection may be performed based on object properties of the multiple data objects, such as behavioral characteristics.

[0304] Specifically, when performing event detection based on 3D target data, multiple data objects and object attributes of the data objects are identified from the 3D target data; group characteristics of the multiple data objects are counted based on the object data of the multiple data objects; and event determination is performed based on the group characteristics.

[0305] For example, multiple people can be identified in 3D target data, and the location area where the people are located can be identified. Based on the location area where the multiple people are located, the crowd density of the multiple people in the area can be counted, and then whether a congestion event occurs can be determined based on the crowd density.

[0306] In a specific implementation, multiple 3D target data may be acquired, wherein the multiple 3D target data are time-series-related; and event prediction may be further performed based on neural network technology.

[0307] Prior to this, a neural network model can be created based on multiple data objects included in the historical 3D target data and target events corresponding to the 3D target data; when performing event prediction, event prediction is performed based on the neural network model and the 3D target data.

[0308] It should be noted that the data object may be a person or a moving object, and the object data may be action characteristics, or characteristics such as age, height, etc. This application does not impose any restrictions on this.

[0309] Reference Figure 13 , shows a flow chart of an embodiment of a data processing method according to the tenth embodiment of the present application, which method may specifically include the following steps:

[0310] Step 1001: Acquire at least two 2D image data, wherein the at least two 2D image data are from cameras located at different positions or angles and photographing the same spatial area.

[0311] Step 1002: Map the at least two 2D image data to a 3D model to obtain 3D target data.

[0312] Step 1003: Perform event detection based on the 3D target data.

[0313] In this embodiment, the 2D image data may come from cameras targeting the same scene, and may be captured by different cameras located at different positions or angles, or may be the same camera captured from different angles after rotation to obtain 2D image data.

[0314] For specific detection details, please refer to the solutions in other embodiments and will not be repeated here.

[0315] Reference Figure 14 , shows a flow chart of an embodiment of a data processing method according to the eleventh embodiment of the present application, which method may specifically include the following steps:

[0316] Step 1101: Acquire at least two 2D image data, wherein the at least two 2D image data are not completely identical;

[0317] Step 1102 , mapping the at least two 2D image data to a 3D model to obtain 3D object data;

[0318] Step 1103: Perform event detection based on the 3D target data.

[0319] The at least two 2D image data mentioned in this embodiment may come from image acquisition devices for the same scene or different scenes, or from the same image acquisition device or different image acquisition devices, and may be taken by the same image acquisition device from the same angle or from different angles.

[0320] Different 2D images are not completely the same, that is, their captured contents are different, and some of the captured objects may be the same, some may be different, or they may include completely different captured objects.

[0321] It should be noted that for more details on event prediction and event detection in the above embodiments 9, 10 and 11, reference can be made to the above embodiments 3, 4 and various examples.

[0322] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0323] Embodiments of the present disclosure may be implemented as a system using any suitable hardware, firmware, software, or any combination thereof to perform the desired configuration. Figure 15 An exemplary system (or device) 1200 that can be used to implement various embodiments described in this disclosure is schematically illustrated.

[0324] For one embodiment, Figure 15 An exemplary system 1200 is shown having one or more processors 1202, a system control module (chipset) 1204 coupled to at least one of the processor(s) 1202, a system memory 1206 coupled to the system control module 1204, a non-volatile memory (NVM) / storage device 1208 coupled to the system control module 1204, one or more input / output devices 1210 coupled to the system control module 1204, and a network interface 1212 coupled to the system control module 1206.

[0325] Processor 1202 may include one or more single-core or multi-core processors, and processor 1202 may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.) In some embodiments, system 1200 can function as a browser as described in the embodiments of this application.

[0326] In some embodiments, the system 1200 may include one or more computer-readable media (e.g., system memory 1206 or NVM / storage device 1208) having instructions and one or more processors 1202 configured in conjunction with the one or more computer-readable media to execute the instructions to implement modules to perform the actions described in this disclosure.

[0327] For one embodiment, the system control module 1204 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1202 and / or any suitable device or component in communication with the system control module 1204 .

[0328] The system control module 1204 may include a memory controller module to provide an interface to the system memory 1206. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0329] System memory 1206 can be used, for example, to load and store data and / or instructions for system 1200. For one embodiment, system memory 1206 can include any suitable volatile memory, such as suitable DRAM. In some embodiments, system memory 1206 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0330] For one embodiment, the system control module 1204 may include one or more input / output controllers to provide interfaces to the NVM / storage device 1208 and the input / output device(s) 1210 .

[0331] For example, NVM / storage 1208 may be used to store data and / or instructions. NVM / storage 508 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0332] NVM / storage device 1208 may include storage resources that are physically part of the device on which system 1200 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 1208 may be accessed over a network via input / output device(s) 1210.

[0333] (One or more) input / output devices 1210 may provide an interface for system 1200 to communicate with any other appropriate devices. Input / output devices 1210 may include communication components, audio components, sensor components, etc. Network interface 1212 may provide an interface for system 1200 to communicate via one or more networks. System 1200 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard such as WiFi, 2G or 3G, or a combination thereof for wireless communication.

[0334] For one embodiment, at least one of the processor(s) 1202 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the system control module 1204. For one embodiment, at least one of the processor(s) 1202 may be packaged together with the logic of one or more controllers of the system control module 1204 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1202 may be integrated on the same die with the logic of one or more controllers of the system control module 1204. For one embodiment, at least one of the processor(s) 1202 may be integrated on the same die with the logic of one or more controllers of the system control module 1204 to form a system-on-chip (SoC).

[0335] In various embodiments, system 1200 may be, but is not limited to, a browser, a workstation, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, system 1200 may have more or fewer components and / or a different architecture. For example, in some embodiments, system 1200 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0336] If the display includes a touch panel, the display screen can be implemented as a touch screen display to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensors can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide action.

[0337] An embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a terminal device, the terminal device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0338] In one example, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a method as in an embodiment of the present application when executing the computer program.

[0339] In one example, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, one or more methods of the embodiments of the present application are implemented.

[0340] The embodiments of the present application disclose a method and apparatus for creating a three-dimensional dynamic scene. Example 1 includes a method for creating a three-dimensional dynamic scene, including:

[0341] determining three-dimensional mapping parameters of at least one image acquisition device by analyzing historical dynamic images;

[0342] Mapping two-dimensional attribute information of a first object in a real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters, wherein the historical dynamic image or the real-time dynamic image includes a sequence image and / or a video image;

[0343] adding the three-dimensional attribute information of the first object to the three-dimensional dynamic template;

[0344] The three-dimensional dynamic template is rendered in real time to obtain a real-time three-dimensional dynamic scene.

[0345] Example 2 may include the method of Example 1, wherein determining the three-dimensional mapping parameters of the video device by analyzing historical dynamic images includes:

[0346] Extract single images from historical dynamic images;

[0347] Determine multiple images corresponding to the same object by comparison;

[0348] The three-dimensional mapping parameters of the image acquisition device are determined based on a plurality of images corresponding to the same object.

[0349] Example 3 may include the method of Example 2, wherein determining, by comparing, multiple images corresponding to the same object comprises:

[0350] extracting feature information of the image;

[0351] The feature information of the images is matched, and multiple images corresponding to the same object are determined based on the matching results.

[0352] Example 4 may include the method of Example 1, wherein, before mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters, the method further comprises:

[0353] Multiple processing threads are created in the central processing unit and / or the graphics processing unit, and the multiple processing threads are used to concurrently execute the step of mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters.

[0354] Example 5 may include the method of Example 1, wherein, before adding the three-dimensional attribute information of the first object to the three-dimensional dynamic template, the method further includes:

[0355] The three-dimensional attribute information object corresponding to the second object in the three-dimensional dynamic scene is added to the three-dimensional dynamic template of the three-dimensional dynamic scene.

[0356] Example 6 may include the method of Example 5, wherein adding the three-dimensional attribute information object corresponding to the second object of the three-dimensional scene to the three-dimensional dynamic template of the three-dimensional dynamic scene includes:

[0357] Call the device information of IoT devices in the IoT system;

[0358] mapping the device information into three-dimensional attribute information according to the three-dimensional mapping parameters as the three-dimensional attribute information corresponding to the second object;

[0359] The three-dimensional attribute information of the Internet of Things device is added to the three-dimensional dynamic template of the three-dimensional dynamic scene.

[0360] Example 7 may include the method of Example 6, wherein calling device information of an IoT device in the IoT system includes:

[0361] Identify the location information of the image acquisition device and determine the Internet of Things device corresponding to the location information.

[0362] Example 8 may include the method of Example 1 or Example 5, wherein the method further includes:

[0363] By detecting the real-time dynamic image, it is determined that the property of the second object has changed, and the three-dimensional dynamic template is updated.

[0364] Example 9 may include the method of Example 8, wherein determining that an attribute change occurs in the second object by detecting the real-time dynamic image, and updating the three-dimensional dynamic template includes:

[0365] Acquiring two-dimensional attribute information of the second object in the real-time dynamic image;

[0366] mapping the two-dimensional attribute information of the second object into three-dimensional attribute information according to the three-dimensional mapping parameters;

[0367] The three-dimensional attribute information of the second object is used to update the three-dimensional dynamic template.

[0368] Example 10 may include the method of Example 1, wherein the three-dimensional attribute information includes position information and posture information.

[0369] Example 11 may include the method of Example 1, wherein the image acquisition device includes at least one of the following: a surveillance camera, a drone device.

[0370] Example 12 may include the method of Example 1, wherein the method further comprises:

[0371] Acquiring three-dimensional attribute information of at least one first object in the three-dimensional dynamic scene;

[0372] Behavior recognition of the first object is performed based on the three-dimensional attribute information.

[0373] Example 13 may include the method of Example 12, wherein the performing behavior recognition of the first object based on the three-dimensional attribute information includes:

[0374] determining behavior information of the first object based on the three-dimensional attribute information;

[0375] An event determination is performed based on the behavior information of the at least one first object, and whether a target event occurs is determined according to the determination result.

[0376] Example 14 may include the method of Example 12, wherein the identifying a behavior of the first object based on the three-dimensional attribute information includes:

[0377] determining behavior information of a plurality of first objects based on the three-dimensional attribute information;

[0378] Based on the behavior information of the multiple first objects, behavior association is performed on the multiple first objects, and whether there is a target relationship is determined according to the association result.

[0379] Example 15 may include the method of Example 12, wherein the identifying the behavior of the first object based on the three-dimensional attribute information includes:

[0380] determining a behavior pattern of the first object based on the three-dimensional attribute information;

[0381] Behavior prediction is performed on the first object based on the behavior pattern.

[0382] Example 16 may include the method of Example 1, wherein the method further comprises:

[0383] A building information model is obtained from a business system, and a three-dimensional dynamic model is generated based on the building information model.

[0384] Example 17 includes a behavior analysis method based on a three-dimensional scene, comprising:

[0385] Rendering a 3D dynamic template based on real-time dynamic images to obtain a real-time 3D dynamic scene;

[0386] Acquiring three-dimensional attribute information of at least one first object in the three-dimensional dynamic scene;

[0387] determining a behavior pattern of the first object based on the three-dimensional attribute information;

[0388] Behavior prediction is performed on the first object based on the behavior pattern.

[0389] Example 18 may include the method of Example 17, wherein rendering the three-dimensional dynamic template according to the real-time dynamic image to obtain the real-time three-dimensional dynamic scene includes:

[0390] Mapping two-dimensional attribute information of a first object in a real-time dynamic image into three-dimensional attribute information according to three-dimensional mapping parameters, wherein the historical dynamic image or the real-time dynamic image includes a sequence image and / or a video image;

[0391] adding the three-dimensional attribute information of the first object to the three-dimensional dynamic template;

[0392] The three-dimensional dynamic template is rendered in real time to obtain a real-time three-dimensional dynamic scene.

[0393] Example 19 may include the method of Example 18, wherein the method further comprises:

[0394] The three-dimensional mapping parameters of at least one image acquisition device are determined by analyzing historical dynamic images.

[0395] Example 20 includes a method for creating a three-dimensional dynamic scene, which includes:

[0396] Calling the Internet of Things system to obtain three-dimensional attribute information of a second object in the three-dimensional dynamic scene;

[0397] updating the three-dimensional dynamic template according to the three-dimensional attribute information of the second object;

[0398] Acquire three-dimensional attribute information of the first object and add it to the three-dimensional dynamic template;

[0399] The three-dimensional dynamic template is rendered in real time to obtain a real-time three-dimensional dynamic scene.

[0400] Example 21 includes a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of one or more of claims 1-20 when executing the computer program.

[0401] Example 22 includes a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method of one or more of claims 1-20.

[0402] Example 23 includes a data processing method, comprising:

[0403] Acquire at least two 2D image data, wherein the photographed objects presented by the at least two 2D image data have an intersection, and the photographed objects of the at least two 2D image data are not completely the same;

[0404] Mapping the at least two 2D image data to a 3D model to obtain 3D object data;

[0405] Event detection is performed based on the 3D object data.

[0406] Example 24 may include the method of Example 23, wherein detecting an event based on the 3D object data comprises:

[0407] identifying a plurality of data objects and object attributes of the data objects from the 3D object data;

[0408] establishing associations between the plurality of data objects according to object attributes of the data objects;

[0409] Event determination is performed based on the correlation results.

[0410] Example 25 may include the method of Example 23, wherein detecting an event based on the 3D object data comprises:

[0411] identifying a plurality of data objects and object attributes of the data objects from the 3D object data;

[0412] collecting statistics of group characteristics of the plurality of data objects based on the object data of the plurality of data objects;

[0413] Event determination is performed based on the group characteristics.

[0414] Example 26 may include the method of Example 23, further comprising:

[0415] Acquire a plurality of the 3D object data, wherein the plurality of the 3D object data are time-series-related;

[0416] Event prediction based on neural network technology.

[0417] Example 27 may include the method of Example 26, wherein, before performing event prediction based on the neural network technology, the method further comprises:

[0418] Creating a neural network model based on a plurality of data objects included in the historical 3D target data and target events corresponding to the 3D target data;

[0419] The event prediction based on neural network technology includes:

[0420] Event prediction is performed based on the neural network model and the 3D target data.

[0421] Example 28 includes a data processing method, comprising:

[0422] Acquire at least two 2D image data, wherein the at least two 2D image data are from cameras located at different positions or angles and photographing the same spatial area;

[0423] Mapping the at least two 2D image data to a 3D model to obtain 3D object data;

[0424] Event detection is performed based on the 3D object data.

[0425] Example 29 includes a data processing method, comprising:

[0426] Acquiring at least two 2D image data, wherein the at least two 2D image data are not completely identical;

[0427] Mapping the at least two 2D image data to a 3D model to obtain 3D object data;

[0428] Event detection is performed based on the 3D object data.

[0429] Although certain embodiments are provided for illustration and description purposes, various alternatives, and / or equivalent implementations, or calculations to achieve the same purpose as shown and described in the embodiments do not depart from the scope of this application. This application is intended to cover any modifications or variations of the embodiments discussed herein. Therefore, it is apparent that the embodiments described herein are limited only by the claims and their equivalents.

Claims

1. A method for creating a three-dimensional dynamic scene, characterized in that: include: Determining three-dimensional mapping parameters of at least one image acquisition device by analyzing historical dynamic images, including: extracting a single image from the historical dynamic images; determining multiple images corresponding to the same object by comparison; determining three-dimensional mapping parameters of the image acquisition device based on the multiple images corresponding to the same object; the three-dimensional mapping parameters include the projection relationship of the image acquisition device from the three-dimensional scene to the two-dimensional image; determining multiple images corresponding to the same object by comparison includes: extracting feature information of the image; matching the feature information of the image, and determining multiple images corresponding to the same object based on the matching results; after obtaining the feature information, the method further includes: reconstructing the scale according to the real position information of the image acquisition device, converting the data scale of the feature information from the camera coordinate system to the world coordinate system, and densifying the sparse point cloud corresponding to the feature information, and matching the feature information of the densified point cloud again; Mapping two-dimensional attribute information of a first object in a real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters, wherein the historical dynamic image or the real-time dynamic image includes a sequence of images and / or a video image, the two-dimensional attribute information represents attribute information in a two-dimensional scene, including at least one of position information, posture information, and appearance feature information; the three-dimensional attribute information represents attribute information of the dynamic image in a three-dimensional scene, including depth information and at least one of position information, posture information, and appearance feature information; and the first object includes a foreground object; adding the three-dimensional attribute information of the first object to the three-dimensional dynamic template; The three-dimensional dynamic template is rendered in real time to obtain a real-time three-dimensional dynamic scene, which includes: adding the three-dimensional attribute information of the first object as a texture to the three-dimensional dynamic template, and rendering to obtain the three-dimensional dynamic scene.

2. The method according to claim 1, characterized in that Before mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters, the method further includes: Multiple processing threads are created in the central processing unit and / or the graphics processing unit, and the multiple processing threads are used to concurrently execute the step of mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information according to the three-dimensional mapping parameters.

3. The method according to claim 1, characterized in that Before adding the three-dimensional attribute information of the first object to the three-dimensional dynamic template, the method further includes: The three-dimensional attribute information object corresponding to the second object in the three-dimensional dynamic scene is added to the three-dimensional dynamic template of the three-dimensional dynamic scene.

4. The method according to claim 3, characterized in that The adding of the three-dimensional attribute information object corresponding to the second object of the three-dimensional dynamic scene to the three-dimensional dynamic template of the three-dimensional dynamic scene includes: Call the device information of IoT devices in the IoT system; mapping the device information into three-dimensional attribute information according to the three-dimensional mapping parameters as the three-dimensional attribute information corresponding to the second object; The three-dimensional attribute information of the Internet of Things device is added to the three-dimensional dynamic template of the three-dimensional dynamic scene.

5. The method according to claim 4, characterized in that The device information of the IoT device in the IoT system includes: Identify the location information of the image acquisition device and determine the Internet of Things device corresponding to the location information.

6. The method according to claim 1 or 3, characterized in that The method further comprises: By detecting the real-time dynamic image, it is determined that the property of the second object has changed, and the three-dimensional dynamic template is updated.

7. The method according to claim 6, characterized in that Determining that the property of the second object has changed by detecting the real-time dynamic image, and updating the three-dimensional dynamic template includes: Acquiring two-dimensional attribute information of the second object in the real-time dynamic image; mapping the two-dimensional attribute information of the second object into three-dimensional attribute information according to the three-dimensional mapping parameters; The three-dimensional attribute information of the second object is used to update the three-dimensional dynamic template.

8. The method according to claim 1, characterized in that The three-dimensional attribute information includes position information and posture information.

9. The method according to claim 1, characterized in that The image acquisition device includes at least one of the following: a surveillance camera and an unmanned aerial vehicle device.

10. The method according to claim 1, characterized in that The method further comprises: Acquiring three-dimensional attribute information of at least one first object in the three-dimensional dynamic scene; Behavior recognition of the first object is performed based on the three-dimensional attribute information.

11. The method according to claim 10, characterized in that The performing behavior recognition of the first object based on the three-dimensional attribute information includes: determining behavior information of the first object based on the three-dimensional attribute information; An event determination is performed based on the behavior information of the at least one first object, and whether a target event occurs is determined according to the determination result.

12. The method according to claim 10, characterized in that The performing behavior recognition of the first object based on the three-dimensional attribute information includes: determining behavior information of a plurality of first objects based on the three-dimensional attribute information; Based on the behavior information of the multiple first objects, behavior association is performed on the multiple first objects, and whether there is a target relationship is determined according to the association result.

13. The method according to claim 10, characterized in that The performing behavior recognition of the first object based on the three-dimensional attribute information includes: determining a behavior pattern of the first object based on the three-dimensional attribute information; Behavior prediction is performed on the first object based on the behavior pattern.

14. The method according to claim 1, wherein The method further comprises: A building information model is obtained from a business system, and a three-dimensional dynamic model is generated based on the building information model.

15. A behavior analysis method based on three-dimensional scenes, characterized in that: include: Rendering a three-dimensional dynamic template according to a real-time dynamic image to obtain a real-time three-dimensional dynamic scene, which includes: mapping the two-dimensional attribute information of a first object in the real-time dynamic image into three-dimensional attribute information according to a three-dimensional mapping parameter, wherein the historical dynamic image or the real-time dynamic image includes a sequence image and / or a video image; adding the three-dimensional attribute information of the first object to the three-dimensional dynamic template; rendering the three-dimensional dynamic template in real time to obtain a real-time three-dimensional dynamic scene; the first object includes a foreground object; a method for determining the three-dimensional mapping parameters includes: extracting a single image from the historical dynamic image; determining multiple images corresponding to the same object by comparison; determining the three-dimensional mapping parameters of an image acquisition device according to the multiple images corresponding to the same object, which includes: matching feature information of the image, and determining multiple images corresponding to the same object according to the matching results; after obtaining the feature information, performing scale reconstruction according to the real position information of the image acquisition device, converting the data scale of the feature information from the camera coordinate system to the world coordinate system, and performing densification processing on the sparse point cloud corresponding to the feature information, and matching the feature information of the densified point cloud again; Acquire three-dimensional attribute information of at least one first object in the three-dimensional dynamic scene, where the three-dimensional attribute information represents attribute information of the dynamic image in the three-dimensional scene, including depth information and at least one of position information, posture information, and appearance feature information; determining a behavior pattern of the first object based on the three-dimensional attribute information; Based on the behavior pattern, behavior prediction is performed on the first object, which includes: using a deep learning algorithm to learn the three-dimensional dynamic scene, extracting the three-dimensional attribute information in each of the three-dimensional dynamic scenes to form structured data, using a graph neural network to learn the structured data, inferring the behavior pattern therein, and deducing the real-time three-dimensional dynamic scene according to the behavior pattern to perform the behavior prediction.

16. The method according to claim 15, characterized in that The method further comprises: The three-dimensional mapping parameters of at least one image acquisition device are determined by analyzing historical dynamic images.

17. A method for creating a three-dimensional dynamic scene, characterized in that: include: Invoking an Internet of Things system to obtain three-dimensional attribute information of a second object in a three-dimensional dynamic scene, comprising: invoking device information of an Internet of Things device in the Internet of Things system, using the device information as two-dimensional attribute information, and mapping the device information to the three-dimensional attribute information according to three-dimensional mapping parameters as the three-dimensional attribute information corresponding to the second object; the three-dimensional attribute information represents attribute information of the dynamic image in the three-dimensional scene, including depth information and at least one of position information, posture information, and appearance feature information; the second object includes a background object; Updating the 3D dynamic template according to the 3D attribute information of the second object, including: determining whether the 3D attribute information of the second object has changed, and if so, updating the 3D attribute information in the 3D dynamic template; Acquiring three-dimensional attribute information of the first object and adding it to the three-dimensional dynamic template includes: analyzing historical dynamic images to determine three-dimensional mapping parameters of at least one image acquisition device; and mapping the two-dimensional attribute information of the first object in the real-time dynamic image into three-dimensional attribute information based on the three-dimensional mapping parameters; The three-dimensional dynamic template is rendered in real time to obtain a real-time three-dimensional dynamic scene.

18. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to one or more of claims 1 to 17 is implemented.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to one or more of claims 1 to 17 is implemented.

20. A data processing method, characterized in that: include: Acquire at least two 2D image data, wherein the photographed objects presented by the at least two 2D image data have an intersection, and the photographed objects of the at least two 2D image data are not completely the same; Mapping the at least two 2D image data to a 3D model to obtain 3D target data; mapping the at least two 2D image data to the 3D model according to three-dimensional mapping parameters, wherein the method for determining the three-dimensional mapping parameters comprises: extracting feature information of the image; matching the feature information of the image, and determining multiple images corresponding to the same object based on the matching results; and determining three-dimensional mapping parameters of an image acquisition device based on the multiple images corresponding to the same object; wherein, after obtaining the feature information, scale reconstruction is performed based on the real position information of the image acquisition device, the data scale of the feature information is converted from a camera coordinate system to a world coordinate system, and a sparse point cloud corresponding to the feature information is densified, and the feature information of the densified point cloud is matched again; Based on the 3D target data, event detection is performed, which includes: identifying multiple data objects and object properties of the data objects from the 3D target data; establishing associations between the multiple data objects based on the object properties of the data objects; performing event judgment based on the association results, and determining whether a target event occurs based on the judgment results, wherein the event judgment includes: performing event judgment based on behavior information of at least one first object.

21. The data processing method according to claim 20, characterized in that: The performing event detection based on the 3D target data includes: identifying a plurality of data objects and object attributes of the data objects from the 3D object data; collecting statistics of group characteristics of the plurality of data objects based on the object data of the plurality of data objects; Event determination is performed based on the group characteristics.

22. The data processing method according to claim 20, characterized in that: Also includes: Acquire a plurality of the 3D object data, wherein the plurality of the 3D object data are time-series-related; Event prediction based on neural network technology.

23. The data processing method according to claim 22, characterized in that: Before performing event prediction based on the neural network technology, the method further includes: Creating a neural network model based on a plurality of data objects included in the historical 3D target data and target events corresponding to the 3D target data; The event prediction based on neural network technology includes: Event prediction is performed based on the neural network model and the 3D target data.

24. A data processing method, characterized in that: include: Acquire at least two 2D image data, wherein the at least two 2D image data are from cameras located at different positions or angles and photographing the same spatial area; Mapping the at least two 2D image data to a 3D model to obtain 3D target data; mapping the at least two 2D image data to the 3D model according to three-dimensional mapping parameters, wherein the method for determining the three-dimensional mapping parameters comprises: extracting feature information of the image; matching the feature information of the image, and determining multiple images corresponding to the same object based on the matching results; and determining three-dimensional mapping parameters of an image acquisition device based on the multiple images corresponding to the same object; wherein, after obtaining the feature information, scale reconstruction is performed based on the real position information of the image acquisition device, the data scale of the feature information is converted from a camera coordinate system to a world coordinate system, and a sparse point cloud corresponding to the feature information is densified, and the feature information of the densified point cloud is matched again; Based on the 3D target data, event detection is performed, which includes: identifying multiple data objects and object properties of the data objects from the 3D target data; establishing associations between the multiple data objects based on the object properties of the data objects; performing event judgment based on the association results, and determining whether a target event occurs based on the judgment results, wherein the event judgment includes: performing event judgment based on behavior information of at least one first object.

25. A data processing method, characterized in that: include: Acquiring at least two 2D image data, wherein the at least two 2D image data are not completely identical; Mapping the at least two 2D image data to a 3D model to obtain 3D target data; mapping the at least two 2D image data to the 3D model according to three-dimensional mapping parameters, wherein the method for determining the three-dimensional mapping parameters comprises: extracting feature information of the image; matching the feature information of the image, and determining multiple images corresponding to the same object based on the matching results; and determining three-dimensional mapping parameters of an image acquisition device based on the multiple images corresponding to the same object; wherein, after obtaining the feature information, scale reconstruction is performed based on the real position information of the image acquisition device, the data scale of the feature information is converted from a camera coordinate system to a world coordinate system, and a sparse point cloud corresponding to the feature information is densified, and the feature information of the densified point cloud is matched again; Based on the 3D target data, event detection is performed, which includes: identifying multiple data objects and object properties of the data objects from the 3D target data; establishing associations between the multiple data objects based on the object properties of the data objects; performing event judgment based on the association results, and determining whether a target event occurs based on the judgment results, wherein the event judgment includes: performing event judgment based on behavior information of at least one first object.

Images