Digital twin space three-dimensional mapping method and device based on visual identification

Abstract

The invention discloses a digital twin space three-dimensional mapping method and device based on visual identification. The method comprises the following steps: acquiring video data containing a to-be-detected target object in a real-time monitoring scene; inputting the preprocessed video data into a deep neural network-based YOLO target detection model to carry out target position prediction, matching and correction so as to generate target tracking data of the target object; generating a twin scene of the target object by using the modeling tool; and calling model tracking target position information pre-stored in a model library according to the target tracking data in the twinborn scene, so as to match a target in the real-time monitoring scene with a virtual target in the twinborn scene to obtain a target data matching result of the real-time monitoring scene and the twinborn scene. According to the method, a 3D modeling tool and a visual identification technology can be combined, digital twinborn simulation of a construction site is realized, and an augmented reality effect is provided.

Description

Technical Field

[0001] The present invention relates to the field of digital twin technology, and in particular to a method and device for three-dimensional mapping of digital twin space based on visual recognition. Background Art

[0002] In modern project site construction, real-time monitoring and safety management are becoming increasingly important. By using advanced computer vision technology and 3D modeling technology, efficient monitoring of the construction site and personnel positioning simulation can be achieved.

[0003] How to achieve high-precision target detection in complex construction site environments while maintaining real-time processing capabilities. Using pre-trained deep learning models (such as YOLO), how to maintain the robustness and continuity of target tracking when the target is occluded, the lighting changes, and the background is disturbed. How to match and update the location information of the target in real time in the 3D twin scene to ensure that the virtual scene is highly consistent with the real scene. How to fix delays in real-time monitoring images and display the target location in the next few seconds in advance to improve the system's response speed. How to effectively manage resources during system operation to avoid memory leaks and resource waste. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the present invention proposes a three-dimensional mapping method of digital twin space based on visual recognition. Through the technical process of video stream acquisition, preprocessing, target monitoring, target tracking and three-dimensional tracking, efficient construction site management and personnel positioning simulation can be achieved.

[0006] Another object of the present invention is to propose a three-dimensional mapping device of digital twin space based on visual recognition.

[0007] To achieve the above-mentioned purpose, the present invention proposes a three-dimensional mapping method of digital twin space based on visual recognition, comprising:

[0008] Acquire video data containing the target object to be detected in the real-time monitoring scene;

[0009] The preprocessed video data is input into the YOLO target detection model based on the deep neural network for target detection to obtain the target detection result of the current frame image;

[0010] Acquire initial target position information of the first frame image according to the target detection result of the first frame image in the current frame image, and predict, match and correct the target position of the current frame image according to the initial target position information to generate target tracking data of the target object;

[0011] A twin scene of the target object is generated using a modeling tool, and in the twin scene, the model library pre-stored model is called according to the target tracking data to track the target position information, so as to match the target in the real-time monitoring scene with the virtual target in the twin scene to obtain the target matching prediction result between the real-time monitoring scene and the twin scene.

[0012] The digital twin space three-dimensional mapping method based on visual recognition in the embodiment of the present invention may also have the following additional technical features:

[0013] In one embodiment of the present invention, after acquiring video data containing the target object to be detected in the real-time monitoring scene, the method further includes:

[0014] Extracting frame images from the video data at a preset frame rate;

[0015] Adjust the frame image to the input size data required by the YOLO target detection model;

[0016] The input size data is normalized to obtain a normalized image sequence.

[0017] In one embodiment of the present invention, inputting the preprocessed video data into a YOLO target detection model based on a deep neural network to perform target detection to obtain a target detection result of the current frame image includes:

[0018] Use OpenCV's DNN module to load the YOLO target detection model and weight file;

[0019] The normalized image sequence is input into the YOLO target detection model to perform target detection and obtain the detection results;

[0020] The detection results are parsed to obtain the target position information and category information of the current frame image.

[0021] In one embodiment of the present invention, obtaining initial target position information of the first frame image according to the target detection result of the first frame image in the current frame image, and predicting, matching and correcting the target position of the current frame image according to the initial target position information to generate target tracking data of the target object includes:

[0022] Use the YOLO target detection model to detect the object in the first frame image to obtain the corresponding initial position information;

[0023] Use the initial position information of the object in the previous frame image to predict the possible position information of the object in the current frame image;

[0024] By matching the predicted possible position information in the current frame image with the target detection result of the previous frame image, the position information of the object in the current frame image is updated;

[0025] The tracking status of the target object is determined by updating the position information of each frame of the image;

[0026] The target position information of each frame of image is recorded to generate the moving trajectory data of the target object.

[0027] In one embodiment of the present invention, the method further includes: when the program ends after the target information disappears, releasing camera resources and closing all models to clean up related cache data.

[0028] To achieve the above-mentioned purpose, the present invention proposes, on the other hand, a three-dimensional mapping device for digital twin space based on visual recognition, comprising:

[0029] A video data acquisition module is used to acquire video data containing the target object to be detected in the real-time monitoring scene;

[0030] The image target detection module is used to input the preprocessed video data into the YOLO target detection model based on the deep neural network to perform target detection to obtain the target detection result of the current frame image;

[0031] A target tracking generation module is used to obtain the initial target position information of the first frame image according to the target detection result of the first frame image in the current frame image, and predict, match and correct the target position of the current frame image according to the initial target position information to generate target tracking data of the target object;

[0032] The target matching prediction module is used for the target content matching module, which is used to generate a twin scene of the target object using a modeling tool, and call the model library pre-stored model to track the target position information according to the target tracking data in the twin scene, so as to match the target in the real-time monitoring scene with the virtual target in the twin scene to obtain the target matching prediction result between the real-time monitoring scene and the twin scene.

[0033] In the embodiment of the present invention, the digital twin space three-dimensional mapping method and device based on visual recognition is used to mark the identified personnel in the on-site images captured by the camera during the construction of the project site, and the moving coordinates of the personnel in the image are tracked. The target information is identified through target detection, the target coordinates are obtained through target tracking, and the personnel positioning simulation of the digital twin scene is realized through model matching and tracking targets.

[0034] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0036] Figure 1 is a flow chart of a method for three-dimensional mapping of a digital twin space based on visual recognition according to an embodiment of the present invention;

[0037] Figure 2 It is a structural diagram of a three-dimensional mapping device of a digital twin space based on visual recognition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] The following describes with reference to the accompanying drawings a method and device for three-dimensional mapping of a digital twin space based on visual recognition according to an embodiment of the present invention.

[0041] Figure 1 is a flow chart of a method for three-dimensional mapping of a digital twin space based on visual recognition according to an embodiment of the present invention. Figure 1 As shown, including:

[0042] S1, obtaining video data containing the target object to be detected in the real-time monitoring scene.

[0043] Understandably, real-time video surveillance is a vital technology in project site construction, as it can provide real-time feedback on on-site conditions and help managers identify and resolve problems in a timely manner.

[0044] In one embodiment of the present invention, real-time video is obtained and transmitted in real time using a streaming media protocol such as RTSP, RTMP, WebRTC, etc.

[0045] RTSP protocol application: RTSP (Real Time Streaming Protocol) is a network control protocol, commonly used to establish and control sessions on streaming media servers. In video surveillance systems, RTSP protocol can achieve real-time monitoring screen transmission and playback, support low-latency transmission and rich control functions, such as play, pause, stop, etc. In addition, RTSP also supports session management, using session ID to identify specific media streaming sessions.

[0046] RTMP protocol application: RTMP (Real-Time Messaging Protocol) is a proprietary protocol developed by Adobe, which supports Adobe Flash plug-ins and is often used in live broadcast scenarios. RTMP servers such as SRS and Adobe Media Server can receive RTMP streams from various sources and distribute them as live streams. Nginx supports RTMP through the RTMP module and can process video and audio streams, including live streaming support, streaming recording, HLS support, etc.

[0047] WebRTC protocol application: WebRTC (Web Real-Time Communications) is a technology that supports real-time communication (RTC) in web browsers. It allows audio and video calls and data sharing through web pages without relying on any plug-ins. WebRTC uses the RTCPeerConnection API to stream video between WebRTC clients, which is suitable for application scenarios that require low latency and high interactivity, such as video conferencing and real-time monitoring.

[0048] It is known that real-time video transmission technology can be implemented through a variety of protocols, including HTTP, RTMP, RTSP, etc. Adaptive streaming technology can dynamically adjust video quality according to network conditions to improve user experience. Media Source Extensions (MSE) allow browsers to play streaming media that is not HTTP protocol.

[0049] Its application scenarios include capturing real-time images of the construction site through cameras to ensure that managers can understand the on-site situation at any time. Monitor the safety status of the construction site and promptly discover safety hazards, such as not wearing a safety helmet and illegal operations. Real-time monitoring of construction progress helps managers better arrange and dispatch resources. Through the Internet, managers can access the video stream of the construction site from anywhere for remote management and decision-making. Allow multiple users to access the video stream at the same time to facilitate collaboration and communication between project team members. Record and store video streams for later playback and analysis. Use artificial intelligence technology to analyze behavioral patterns in the video, such as personnel flow and work efficiency.

[0050] In project site construction, efficient real-time video monitoring can be achieved by using streaming protocols (such as RTSP, RTMP, WebRTC) and camera equipment. This technology not only provides real-time feedback on the on-site situation, but also supports remote management and data analysis, greatly improving the safety and management efficiency of construction.

[0051] S2, input the preprocessed video data into the YOLO target detection model based on the deep neural network for target detection to obtain the target detection result of the current frame image.

[0052] As you can understand, you can use OpenCV's DNN module to load the YOLO model and its weights file. Input the processed frame into the model for object detection. Pass the input into the model and get the output result. The output usually contains the results of multiple layers, which are parsed to extract the location and category information of the target; including the bounding box coordinates of the target (such as (x, y, w, h)), category information (such as person, car), and confidence score.

[0053] Understandably, OpenCV provides a powerful DNN module that supports model loading and inference for multiple deep learning frameworks such as TensorFlow, Caffe, Darknet, etc. YOLO is a popular real-time object detection algorithm known for its fast and accurate detection capabilities. The YOLO model can detect multiple objects at the same time and output their location and category information. The weight file of the YOLO model contains the parameters learned by the model during the training process, which enables the model to perform accurate object detection on new images.

[0054] Specifically, before object detection and object tracking, the video stream needs to be preprocessed to ensure that the input data meets the requirements of the model. The following are the three main steps of preprocessing: frame extraction, size adjustment, and normalization.

[0055] The embodiments of the present invention extract frames from the video stream, usually at a fixed frame rate (e.g., 30 frames per second), to ensure that the processing speed matches the video playback speed and that enough frames can be captured to achieve continuous target detection and tracking.

[0056] Choose a suitable frame rate (such as 30FPS) to balance the processing time and the amount of information. Too high a frame rate may lead to excessive processing burden, while too low a frame rate may cause key information to be missed. If the frame rate of the video stream is higher than the required frame rate, you can choose to extract it every certain number of frames. For example, for a 30FPS video stream, if you want to process 30 frames per second, you do not need to skip frames; if you want to process 15 frames per second, you can extract every other frame.

[0057] The embodiment of the present invention adjusts the frame image to the input size required by the target detection model, thereby ensuring the consistency of the image with the model input requirements, and helping to improve the accuracy and consistency of detection.

[0058] Most object detection models (such as YOLO) have fixed input size requirements, such as 416x416 pixels or 608x608 pixels. Use appropriate interpolation methods (such as bilinear interpolation) to resize the image to minimize distortion.

[0059] The embodiment of the present invention scales the pixel value to between 0 and 1, or performs a normalization process, thereby improving the stability and convergence speed of the model and ensuring the consistency of comparison between different images.

[0060] Scale pixel values ​​from their original range (e.g. 0-255) to between 0 and 1. Subtract the mean and divide by the standard deviation to make the pixel value distribution closer to a standard normal distribution.

[0061] The above preprocessing steps ensure that each frame in the video stream is properly adjusted and processed for input into the object detection model. These steps include: extracting frames from the video stream at a fixed frame rate. Adjusting the frames to the input size required by the object detection model. Scaling the pixel values ​​to between 0 and 1, or normalizing them. These preprocessing steps provide high-quality input data for subsequent object detection and object tracking, thereby improving the performance and accuracy of the overall system.

[0062] S3, obtaining the initial target position information of the first frame image according to the target detection result of the first frame image in the current frame image, and predicting, matching and correcting the target position of the current frame image according to the initial target position information to generate target tracking data of the target object.

[0063] It is understandable that visual recognition technology first detects the target object in the image captured by the camera. Target recognition is achieved through deep learning algorithms such as convolutional neural networks (CNNs). These algorithms can learn the characteristics of objects and accurately detect objects in new images or videos. By identifying the determined object labels, the models in the digital twin model library are called for the simulation of digital twin scenes.

[0064] For example, the category of an object (such as a person, a car, an animal, etc.), shape, size, color, and other features can be identified.

[0065] Once the target object is detected, visual recognition technology will start tracking the object. The tracking process usually involves continuous observation and position update of the object to ensure that the object's movement trajectory can be accurately depicted. To achieve this, the algorithm uses the position information of the object in the previous frame to predict the possible position of the object in the current frame, and updates the position information of the object through matching and verification. Through continuous target tracking, the movement trajectory of the object can be generated. This trajectory is usually represented by a series of coordinate points, each of which corresponds to the position of the object at a certain moment. By analyzing and processing these coordinate points, the motion parameters such as the object's movement speed and acceleration can be obtained. The specific tracking is as follows:

[0066] Initialize position information: In the first frame, the object is detected using the target detection algorithm to obtain the initial position.

[0067] The initial position includes the initial position information of the target (bounding box coordinates and category).

[0068] Initializing location information can provide initial location information for tracking.

[0069] Reinforcement learning algorithm: In subsequent frames, the position information of the previous frame is used to predict the possible position of the object in the current frame. A Kalman filter is used to make the prediction of moving targets smoother and more stable.

[0070] The obtained position information includes: the position information of the previous frame and the predicted current position information.

[0071] The reinforcement learning algorithm predicts the possible location of the target in the current frame, improving the robustness of tracking.

[0072] Match and correct: Based on the predicted position, the object’s position information is updated by matching the possible target area in the current frame with the previous detection results.

[0073] Among them, predicted position: the possible position of the target in the current frame predicted by the Kalman filter.

[0074] Among them, the detection result of the current frame: the location and category information of the target obtained by analyzing the target monitoring.

[0075] Among them, updated location information: the final location information obtained through matching and correction.

[0076] By matching the possible target area in the current frame with the previous detection results, the object's position information is updated to ensure the accuracy of tracking.

[0077] Tracking update: Maintain the tracking status of the target object by updating the detection results of each frame.

[0078] The tracking update result includes the updated location information and the moving trajectory of the target.

[0079] Generate movement trajectory: By recording the target position information of each frame, the movement trajectory of the target object can be generated.

[0080] The moving trajectory data includes target position information of each frame, and each position point can be expressed in the form of coordinates.

[0081] S4, using a modeling tool to generate a twin scene of the target object, and in the twin scene, calling the model library to pre-store the model to track the target position information according to the target tracking data, so as to match the target in the real-time monitoring scene with the virtual target in the twin scene to obtain the target data matching result of the real-time monitoring scene and the twin scene.

[0082] Specifically, the model library model is called according to the target identified in the on-site monitoring screen to generate a twin scene, and the model tracking label is called according to the coordinate generation label generated by target tracking, so that the model matches the on-site screen content.

[0083] In project site construction, 3D tracking technology can be used to create a virtual twin scene to simulate and track targets in the real world. This not only improves the visualization of monitoring, but also optimizes the decision-making process through prediction and advance display. The 3D tracking of the present invention includes the following steps:

[0084] Use 3D modeling tools (such as Unreal Engine (UE), Unity, etc.) to generate a twin scene of the target. Extract target information from the real-time video stream through visual recognition technology (such as target detection and recognition algorithm). Call the 3D model pre-stored in the model library and apply it to the twin scene to create a virtual target.

[0085] 3D modeling tools: Unreal Engine (UE) provides powerful graphics rendering capabilities and physics engines, suitable for creating high-quality virtual scenes. Unity has extensive community support and rich plug-in resources, suitable for rapid prototyping and cross-platform publishing.

[0086] Object Detection: Use deep learning models (such as YOLO, SSD, etc.) to detect objects from video streams.

[0087] Target recognition: further identify the category of the target (such as workers, vehicles, equipment, etc.).

[0088] Model Library: 3D Models: Pre-created and stored 3D models, including workers, vehicles, equipment, etc.

[0089] Model matching: Based on the visual recognition results, select the appropriate 3D model and place it at the corresponding position in the twin scene.

[0090] Furthermore, in the twin scene, the 3D model moves according to the actual position information of the target, forming a model movement tracking effect.

[0091] Target location information: Target location information obtained from real-time video stream.

[0092] Model position update: Update the position of the 3D model according to the target position information.

[0093] Location updates include: real-time updates and smooth movement.

[0094] Real-time update: The position of the 3D model is updated every frame to ensure that the model is consistent with the position of the actual target.

[0095] Smooth movement: Use interpolation techniques (such as linear interpolation, spline interpolation, etc.) to make the movement of 3D models smoother and more natural.

[0096] Optimize performance: Adjust the level of detail of 3D models based on the distance and viewing angle of the target to optimize rendering performance. Merge draw calls for multiple 3D models to reduce rendering overhead.

[0097] Furthermore, the target in the real-time monitoring screen is matched with the virtual target in the twin scene and displayed in the twin scene. Through the target prediction results, the possible location of the target within the next 3 seconds can be displayed in advance so that decisions can be made in advance.

[0098] The embodiment of the present invention obtains the real-time video stream from the camera, displays the 3D model in the twin scene, and obtains the possible position of the target in the next 3 seconds through the Kalman filter or other prediction algorithms.

[0099] Specifically, use feature matching algorithms (such as SIFT, SURF, etc.) to match the target in the real-time monitoring screen with the virtual target in the twin scene. Ensure that the virtual target in the twin scene is consistent with the target position in the real-time monitoring screen. Use a Kalman filter or other prediction algorithm to predict the possible position of the target in the next 3 seconds. In the twin scene, the predicted position is displayed by a dotted line or other mark to remind the user.

[0100] Furthermore, when the target information disappears or the program ends, the camera resources are released and all models are closed. The relevant cache data is cleaned up to ensure that the system resources are effectively released.

[0101] Camera resources: resources of camera devices.

[0102] Model resources: 3D model resources.

[0103] Cached data: temporarily stored data, such as target location information, prediction results, etc.

[0104] Resource release: Camera: Turn off the camera device and release related hardware resources. Model: Destroy all 3D model instances and release related rendering resources. Cache cleanup: Memory cleanup: Clear temporary data such as target position information and prediction results stored in the memory. File cleanup: Delete temporary files to ensure that disk space is effectively managed.

[0105] In this way, the accuracy and real-time performance of three-dimensional tracking can be achieved, providing strong support for the monitoring and management of project sites.

[0106] According to the three-dimensional mapping method of digital twin space based on visual recognition of the present invention, real-time video monitoring of the construction site is provided, remote access is supported, and management efficiency is improved. Advanced deep learning models are used for high-precision target detection, providing real-time target location and category information. Through target tracking technology, continuous tracking of the target is ensured, and detailed movement trajectories and motion parameters are generated. Combined with 3D modeling tools and visual recognition technology, digital twin simulation of the construction site is realized to provide augmented reality effects. The target position is displayed in advance through target prediction results, which effectively fixes the delay problem and improves the real-time performance of the system. Automatically manage camera resources and model resources to ensure efficient use of system resources.

[0107] In order to implement the above embodiment, Figure 2 As shown, this embodiment also provides a three-dimensional mapping device 10 for digital twin space based on visual recognition, including:

[0108] The video data acquisition module 100 is used to acquire video data containing a target object to be detected in a real-time monitoring scene;

[0109] The image target detection module 200 is used to input the preprocessed video data into the YOLO target detection model based on the deep neural network to perform target detection to obtain the target detection result of the current frame image;

[0110] The target tracking generation module 300 is used to obtain the initial target position information of the first frame image according to the target detection result of the first frame image in the current frame image, and predict, match and correct the target position of the current frame image according to the initial target position information to generate target tracking data of the target object;

[0111] The target matching prediction module 400 is used for the target content matching module, which is used to generate a twin scene of the target object using a modeling tool, and call the model library pre-stored model to track the target position information according to the target tracking data in the twin scene, so as to match the target in the real-time monitoring scene with the virtual target in the twin scene to obtain the target matching prediction result between the real-time monitoring scene and the twin scene.

[0112] Furthermore, after the video data acquisition module 100, it also includes: a pre-processing module, which is used to:

[0113] Extracting frame images from the video data at a preset frame rate;

[0114] Adjust the frame image to the input size data required by the YOLO target detection model;

[0115] The input size data is normalized to obtain a normalized image sequence.

[0116] Furthermore, the image object detection module 200 is also used for:

[0117] Use OpenCV's DNN module to load the YOLO target detection model and weight file;

[0118] The normalized image sequence is input into the YOLO target detection model to perform target detection and obtain the detection results;

[0119] The detection results are parsed to obtain the target position information and category information of the current frame image.

[0120] Furthermore, the target tracking generation module 300 is also used for:

[0121] Use the YOLO target detection model to detect the object in the first frame image to obtain the corresponding initial position information;

[0122] Use the initial position information of the object in the previous frame image to predict the possible position information of the object in the current frame image;

[0123] By matching the predicted possible position information in the current frame image with the target detection result of the previous frame image, the position information of the object in the current frame image is updated;

[0124] The tracking status of the target object is determined by updating the position information of each frame of the image;

[0125] The target position information of each frame of image is recorded to generate the moving trajectory data of the target object.

[0126] Furthermore, after the target matching prediction module 400, it also includes: a data release module, which is used to release camera resources and close all models to clean up related cache data when the program ends after the target information disappears.

[0127] According to the three-dimensional mapping device of the digital twin space based on visual recognition of the present invention, real-time video monitoring of the construction site is provided, remote access is supported, and management efficiency is improved. Advanced deep learning models are used for high-precision target detection, providing real-time target position and category information. Through target tracking technology, continuous tracking of the target is ensured, and detailed movement trajectories and motion parameters are generated. Combining 3D modeling tools and visual recognition technology, digital twin simulation of the construction site is realized, providing augmented reality effects. The target position is displayed in advance through the target prediction results, which effectively fixes the delay problem and improves the real-time performance of the system. Automatically manage camera resources and model resources to ensure efficient use of system resources.

[0128] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0129] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

Claims

1. A three-dimensional mapping method of digital twin space based on visual recognition, characterized in that: include: Acquire video data containing the target object to be detected in the real-time monitoring scene; The preprocessed video data is input into the YOLO target detection model based on the deep neural network for target detection to obtain the target detection result of the current frame image; Acquire initial target position information of the first frame image according to the target detection result of the first frame image in the current frame image, and predict, match and correct the target position of the current frame image according to the initial target position information to generate target tracking data of the target object; A twin scene of the target object is generated using a modeling tool, and in the twin scene, the model library pre-stored model is called according to the target tracking data to track the target position information, so as to match the target in the real-time monitoring scene with the virtual target in the twin scene to obtain the target matching prediction result between the real-time monitoring scene and the twin scene.

2. The method according to claim 1, characterized in that After acquiring video data containing the target object to be detected in the real-time monitoring scene, the method further includes: Extracting frame images from the video data at a preset frame rate; Adjust the frame image to the input size data required by the YOLO target detection model; The input size data is normalized to obtain a normalized image sequence.

3. The method according to claim 2, characterized in that The pre-processed video data is input into a YOLO target detection model based on a deep neural network to perform target detection to obtain a target detection result of the current frame image, including: Use OpenCV's DNN module to load the YOLO target detection model and weight file; The normalized image sequence is input into the YOLO target detection model to perform target detection and obtain the detection results; The detection results are parsed to obtain the target position information and category information of the current frame image.

4. The method according to claim 3, characterized in that The method comprises: obtaining initial target position information of the first frame image according to the target detection result of the first frame image in the current frame image, and predicting, matching and correcting the target position of the current frame image according to the initial target position information to generate target tracking data of the target object, including: Use the YOLO target detection model to detect the object in the first frame image to obtain the corresponding initial position information; Use the initial position information of the object in the previous frame image to predict the possible position information of the object in the current frame image; By matching the predicted possible position information in the current frame image with the target detection result of the previous frame image, the position information of the object in the current frame image is updated; The tracking status of the target object is determined by updating the position information of each frame of the image; The target position information of each frame of image is recorded to generate the moving trajectory data of the target object.

5. The method according to claim 1, characterized in that The method further includes: when the target information disappears and the program ends, releasing the camera resources and closing all models to clean up related cache data.

6. A three-dimensional mapping device for digital twin space based on visual recognition, characterized in that: include: A video data acquisition module is used to acquire video data containing the target object to be detected in the real-time monitoring scene; The image target detection module is used to input the preprocessed video data into the YOLO target detection model based on the deep neural network to perform target detection to obtain the target detection result of the current frame image; A target tracking generation module is used to obtain the initial target position information of the first frame image according to the target detection result of the first frame image in the current frame image, and predict, match and correct the target position of the current frame image according to the initial target position information to generate target tracking data of the target object; The target matching prediction module is used for the target content matching module, which is used to generate a twin scene of the target object using a modeling tool, and call the model library pre-stored model to track the target position information according to the target tracking data in the twin scene, so as to match the target in the real-time monitoring scene with the virtual target in the twin scene to obtain the target matching prediction result between the real-time monitoring scene and the twin scene.

7. The device according to claim 6, characterized in that After the video data acquisition module, it also includes: a pre-processing module for: Extracting frame images from the video data at a preset frame rate; Adjust the frame image to the input size data required by the YOLO target detection model; The input size data is normalized to obtain a normalized image sequence.

8. The device according to claim 7, characterized in that The image object detection module is further used for: Use OpenCV's DNN module to load the YOLO target detection model and weight file; The normalized image sequence is input into the YOLO target detection model to perform target detection and obtain the detection results; The detection results are parsed to obtain the target position information and category information of the current frame image.

9. The device according to claim 8, characterized in that The target tracking generation module is also used for: Use the YOLO target detection model to detect the object in the first frame image to obtain the corresponding initial position information; Use the initial position information of the object in the previous frame image to predict the possible position information of the object in the current frame image; By matching the predicted possible position information in the current frame image with the target detection result of the previous frame image, the position information of the object in the current frame image is updated; The tracking status of the target object is determined by updating the position information of each frame of the image; The target position information of each frame of image is recorded to generate the moving trajectory data of the target object.

10. The device according to claim 6, characterized in that After the target matching prediction module, it also includes: a data release module, which is used to release camera resources and close all models to clean up related cache data when the program ends after the target information disappears.

Images