Close contact determination system for 3D scene structure reconstruction and 3D human identity re-identification

Abstract

This invention discloses a close contact determination system for 3D scene structure reconstruction and 3D human identity re-identification, including: selecting positive patients; obtaining a 3D human model of the positive patient; performing pedestrian detection and masking on the video using a target detection neural network, and generating a 3D pedestrian pre-model using a 3D reconstruction neural network; performing 3D reconstruction of the entire video scene; and performing matching calculations between the pedestrian 3D model and the positive patient 3D model using a ternary loss function neural network. By using video stream for 3D scene reconstruction, the correlation between adjacent video frames is fully utilized, minimizing distance misjudgments caused by camera distortion. This system also uses 3D human identity recognition, which can effectively utilize the correlation between adjacent video frames, verify the accuracy of the 3D model through multi-frame cross-verification, and then perform similarity matching using the obtained 3D model, which can significantly improve recognition accuracy and increase the accuracy of close contact determination.

Description

Technical Field

[0001] This invention belongs to the field of identity re-identification technology, and in particular relates to a close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification. Background Technology

[0002] Existing methods for determining close contacts mostly involve extracting a single image from a video, outlining the area of ​​the person in the image, and then performing frame-by-frame similarity matching of the people in the image. However, this approach fails to take into account the distance errors that may be caused by factors such as camera distortion. In addition, a single image carries far less information than continuous dynamic actions, which limits the accuracy of similarity matching. Summary of the Invention

[0003] The purpose of this invention is to provide a close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification in order to solve the above-mentioned problems.

[0004] The present invention achieves the above objectives through the following technical solutions: A close contact determination system for 3D scene structure reconstruction and 3D human identity re-identification includes the following steps: Step 1: Obtain videos containing positive patients and select the positive patients from the videos; Step 2: The video is preprocessed using a mask neural network to hide parts other than positive patients. The processed video is then input into a 3D reconstruction neural network to obtain the corresponding 3D human body model of the positive patient. Step 3: First, use a target detection neural network to detect pedestrians in the video and apply a mask. Then, use a 3D reconstruction neural network to reconstruct the 3D pedestrian pre-model. Step 4: Perform 3D reconstruction of the entire video scene. After reconstruction, use a generative adversarial neural network to correct 3D scene distortion. Based on the reconstructed scene, confirm the inter-frame pedestrian and human body information relationship in the video and use a diffusion model to reconstruct the 3D pedestrian model. Step 5: Use a ternary loss function neural network to match the 3D model of pedestrians with the 3D model of positive patients. If there is a pedestrian in the surveillance video whose calculated value is less than the threshold of the positive patient, then mark this pedestrian as a positive patient. The trajectory of each pedestrian is obtained through the previous 3D model of pedestrians. The trajectory is compared to obtain the trajectory of close contacts. Finally, the close contact video is output and handed over to the human reviewer.

[0005] The names of the neural networks mentioned above are all optional. As long as they contain the specified functions, replacing them with neural networks with similar functions will not affect the present invention.

[0006] Compared with the prior art, the beneficial effects of the present invention are as follows: This system reconstructs 3D scenes using video streams, fully leveraging the correlation between adjacent video frames to minimize distance misjudgments caused by camera distortion. Furthermore, it employs 3D human identification, effectively utilizing the correlation between adjacent video frames to verify the accuracy of the 3D model across multiple frames. The resulting 3D model is then used for similarity matching, significantly improving recognition accuracy and enhancing the precision of close contact determination. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a schematic diagram of the close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification described in this invention; Figure 2 This is a schematic diagram of the mask neural network of the close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification described in this invention; Figure 3 This is a schematic diagram of the three-dimensional reconstruction neural network of the close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification described in this invention; Figure 4 This is a schematic diagram of the generative adversarial neural network of the close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification described in this invention; Figure 5 This is a schematic diagram of the diffusion model of the close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification described in this invention; Figure 6 This is a schematic diagram of the ternary loss function neural network of the close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification described in this invention. Detailed Implementation

[0009] The present invention will be further described below with reference to the accompanying drawings: like Figure 1 As shown, the close contact determination system for 3D scene structure reconstruction and 3D human identity re-identification includes the following: Step 1: Obtain videos containing positive patients and select the positive patients from the videos; Step 2: The video is preprocessed using a mask neural network yolov7-mask to hide parts other than positive patients. The processed video is then input into a 3D reconstruction neural network NeuralRecon to obtain the corresponding 3D human body model of the positive patient. Among them: such as Figure 2 As shown, the process of the mask neural network is as follows: first, the image is processed by the neural network to obtain feature vectors; then, the feature vectors are used to perform target detection to obtain the target category and contour detection to obtain the target contour; finally, the results are merged to form the mask neural network. Among them: such as Figure 3 As shown, the 3D reconstruction neural network first processes the video streams from different cameras to obtain three surface models with different granularities. Then, the multiple camera frames are compared and evaluated with each other, and finally the 3D model is output. Step 3: First, use a target detection neural network such as yolov7-mask to detect pedestrians and apply a mask to the video. Then, use a 3D reconstruction neural network to reconstruct the 3D pedestrian pre-model. Due to occlusion, the generated model is not accurate enough and requires additional processing. Step 4: Perform 3D reconstruction of the entire video scene. After reconstruction, use a generative adversarial neural network such as DCGAN to correct 3D scene distortion. Based on the reconstructed scene, confirm the inter-frame pedestrian and human body information relationship in the video and use a diffusion model such as DDPM to reconstruct the 3D pedestrian model. Among them: such as Figure 4 As shown, the Generative Adversarial Neural Network (GAN) continuously trains a Generative Neural Network (GNN) and an Adversarial Neural Network (ANN) separately. If the result generated by the GNN is detected by the ANN, the GNN is trained; otherwise, the ANN is trained, so that the result generated by the GNN is close to the real sample. Among them: such as Figure 5 As shown, the diffusion model is an image reconstruction neural network that has been initially trained. It adds noise or blank areas to the image to be reconstructed, trains the neural network to fill in the additional missing areas, and repeats the above process to achieve the overall effect of reconstructing a small number of images. Step 5: Use a ternary loss function neural network such as OSNet to match the 3D model of pedestrians with the 3D model of positive patients. If there is a pedestrian in the surveillance video whose calculated value is less than the threshold of the positive patient, then mark this pedestrian as a positive patient. The trajectory of each pedestrian is obtained through the previous 3D modeling of pedestrians. The trajectory is compared to obtain the trajectory of close contacts. Finally, the close contact video is output and handed over to the human for review. Among them, such as Figure 6As shown, the ternary loss function neural network is a method that allows the neural network to directly output similarity calculated using cosine distance by setting a loss function. The process involves continuously training between and within groups to make the feature vectors of images generated in the same group similar.

[0010] Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. A close contact determination system for 3D scene structure reconstruction and 3D human identity re-identification, characterized in that: Includes the following steps: Step 1: Obtain videos containing positive patients and select the positive patients from the videos; Step 2: The video is preprocessed using a mask neural network to hide parts other than positive patients. The processed video is then input into a 3D reconstruction neural network to obtain the corresponding 3D human body model of the positive patient. Step 3: First, use a target detection neural network to detect pedestrians in the video and apply a mask. Then, use a 3D reconstruction neural network to reconstruct the 3D pedestrian pre-model. Step 4: Perform 3D reconstruction of the entire video scene. After reconstruction, use a generative adversarial neural network to correct 3D scene distortion. Based on the reconstructed scene, confirm the inter-frame pedestrian and human body information relationship in the video and use a diffusion model to reconstruct the 3D pedestrian model. Step 5: Use a ternary loss function neural network to match the 3D model of pedestrians with the 3D model of positive patients. If there is a pedestrian in the surveillance video whose calculated value is less than the threshold of the positive patient, then mark this pedestrian as a positive patient. The trajectory of each pedestrian is obtained through the previous 3D model of pedestrians. The trajectory is compared to obtain the trajectory of close contacts. Finally, the close contact video is output and handed over to the human reviewer.

2. The close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification according to claim 1, characterized in that: The process of the mask neural network in step two is as follows: first, the image is processed by a neural network to obtain feature vectors; then, the feature vectors are used to perform target detection to obtain the target category and contour detection to obtain the target contour; finally, the results are merged to form a mask neural network.

3. The close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification according to claim 1, characterized in that: In step two, the 3D reconstruction neural network first processes the video streams from different cameras to obtain three different granularity surface models. Then, the multiple camera frames are compared and evaluated with each other, and finally the 3D model is output.

4. The close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification according to claim 1, characterized in that: In step four, the generative adversarial neural network (GAN) is continuously trained separately with the GAN and the adversarial neural network. If the result generated by the GAN is detected by the adversarial neural network, the GAN is trained; otherwise, the adversarial neural network is trained, so that the result generated by the GAN is close to the real sample.

5. The close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification according to claim 1, characterized in that: In step four, the diffusion model is a pre-trained image reconstruction neural network. Noise or blank areas are added to the image to be reconstructed, and the neural network is trained to fill in the additional missing areas. The above process is repeated to achieve the overall effect of reconstructing a small number of images.

6. The close contact determination system for three-dimensional scene structure reconstruction and three-dimensional human identity re-identification according to claim 1, characterized in that: In step five, the ternary loss function neural network is a method that allows the neural network to directly output similarity calculated using cosine distance by setting a loss function. The process involves continuously training between and within groups to make the feature vectors of images generated in the same group similar.

Images