Social distancing detection methods, devices, electronic equipment and storage media

Abstract

This invention provides a social distance detection method, apparatus, electronic device, and storage medium. The method includes: acquiring a first target image and a second target image, wherein the first target image and the second target image are acquired at the same time from different angles; inputting the first target image and the second target image into a pre-trained neural network model, and outputting a first head detection result, a second head detection result, a first head feature, and a second head feature through the pre-trained neural network model; performing target head matching based on the first head detection result, the second head detection result, the first head feature, and the second head feature, and extracting target head information based on the matching result; performing three-dimensional reconstruction of the target head based on the target head information, and calculating the social distance between target individuals based on the reconstructed three-dimensional target head. This improves the accuracy of social distance detection between target individuals.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more particularly to a social distance detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] In some scenarios, it is necessary to control the distance between people, such as ticket windows, bank counters, public places during flu season, and special locations with restricted gatherings. Current methods for detecting distance between people mainly rely on visual estimation by staff from the scene or surveillance video, or on image processing techniques. However, visual estimation by staff is not only time-consuming and labor-intensive, with high labor costs, but also susceptible to subjective factors, resulting in low accuracy. Existing image processing methods primarily measure the approximate actual distance by detecting the distance between people in two-dimensional images, which also has low accuracy. Therefore, current methods for detecting distance between people suffer from low accuracy. Summary of the Invention

[0003] This invention provides a social distance detection method that can improve the accuracy of social distance detection between people.

[0004] In a first aspect, embodiments of the present invention provide a social distance detection method, the method comprising:

[0005] Acquire a first target image and a second target image, wherein the first target image and the second target image are acquired at the same time from different angles;

[0006] The first target image and the second target image are input into a pre-trained neural network model, and the first head detection result, the second head detection result, the first head feature, and the second head feature are output through the pre-trained neural network model. The first head detection result and the first head feature correspond to the first target image, and the second head detection result and the second head feature correspond to the second target image.

[0007] The first head detection result and the second head detection result of the target person are obtained by matching the first head feature and the second head feature, and the target head information is extracted based on the first head detection result and the second head detection result of the target person.

[0008] The target head is reconstructed in three dimensions based on the target head information, and the social distance between the target people is calculated based on the reconstructed three-dimensional target head.

[0009] Optionally, the pre-trained neural network model includes a common network, a first branch network, and a second branch network. The common network is connected to both the first and second branch networks. The first branch network is used to output a first head detection result and a second head detection result, and the second branch network is used to output a first head feature and a second head feature.

[0010] Optionally, the step of inputting the first target image and the second target image into a pre-trained neural network model, and outputting a first head detection result, a second head detection result, a first head feature, and a second head feature through the pre-trained neural network model, includes:

[0011] The first target image and the second target image are stitched together to obtain the input image;

[0012] The input image is fed into a pre-trained neural network model;

[0013] The first head detection result and the second head detection result are calculated and output using the common network and the first branch network in the pre-trained neural network model; and

[0014] The first head feature and the second head feature are calculated and output using the common network and the second branch network in the pre-trained neural network model.

[0015] Optionally, the training of the neural network model includes:

[0016] The first training was performed on the public network and the first branch network using the first dataset;

[0017] A second training was performed on the public network and the second branch network using the second dataset;

[0018] The first training and the second training are performed alternately until the first training and the second training converge to obtain a trained neural network model.

[0019] Optionally, a first head detection result corresponds to a first head feature, and a second head detection result corresponds to a second head feature. The step of matching the first head feature and the second head feature to obtain the first head detection result and the second head detection result of the target person, and extracting the target head information based on the first head detection result and the second head detection result of the target person, includes:

[0020] Based on the similarity between the first head feature and the second head feature, the first head feature and the second head feature are matched to obtain a head feature pair belonging to the same target person. The head feature pair includes the first head feature and the second head feature.

[0021] Based on the head feature pairs, the corresponding head detection result pairs are obtained, and the head detection result pairs include the first head detection result and the second head detection result;

[0022] Based on the head detection results, the target head information is extracted.

[0023] Optionally, extracting target head information based on the head detection result pair includes:

[0024] Based on the multi-target tracking algorithm, IDs are assigned to the head detection results of different target personnel to obtain head detection result pairs with IDs;

[0025] Based on the head detection results with IDs, extract the target head information.

[0026] Optionally, the target head information includes head depth information, and the step of performing three-dimensional reconstruction of the target head based on the target head information includes:

[0027] Based on the head depth information, the target head is reconstructed in three dimensions to obtain a three-dimensional target head.

[0028] Optionally, the calculation of social distance between target individuals based on the reconstructed three-dimensional target heads includes:

[0029] The reconstructed three-dimensional target heads are projected onto a preset plane in three-dimensional space to obtain multiple target head projections;

[0030] Calculate the distance between the projected heads of different targets to obtain the projected distance between the target individuals;

[0031] The projected distance between the target individuals is converted into a social distance between them according to a preset ratio.

[0032] Secondly, embodiments of the present invention also provide a social distancing detection device, the device comprising:

[0033] The acquisition module is used to acquire a first target image and a second target image, wherein the first target image and the second target image are acquired at the same time from different angles;

[0034] The first processing module is used to input the first target image and the second target image into a pre-trained neural network model, and output a first head detection result, a second head detection result, a first head feature, and a second head feature through the pre-trained neural network model, wherein the first head detection result and the first head feature correspond to the first target image, and the second head detection result and the second head feature correspond to the second target image;

[0035] The second processing module is used to obtain the first head detection result and the second head detection result of the target person by matching the first head feature and the second head feature, and to extract the target head information based on the first head detection result and the second head detection result of the target person;

[0036] The third processing module is used to perform three-dimensional reconstruction of the target head based on the target head information, and to calculate the social distance between the target people based on the reconstructed three-dimensional target head.

[0037] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the social distance detection method provided in embodiments of the present invention.

[0038] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the social distancing detection method provided in the embodiments of the present invention.

[0039] In this embodiment of the invention, a first target image and a second target image are acquired, which are captured from different angles at the same time. The first and second target images are input into a pre-trained neural network model, and the model outputs a first head detection result, a second head detection result, a first head feature, and a second head feature. The first head detection result and the first head feature correspond to the first target image, and the second head detection result and the second head feature correspond to the second target image. Target head matching is performed based on the first head detection result, the second head detection result, the first head feature, and the second head feature, and target head information is extracted based on the matching results. The target head is then reconstructed in three dimensions based on the target head information, and the social distance between target individuals is calculated based on the reconstructed three-dimensional target head. By extracting more accurate target head information from head images of target individuals at different angles for three-dimensional reconstruction, the position of the three-dimensional target head in three-dimensional space is more accurate, thereby improving the accuracy of the social distance calculation between target individuals. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of a social distance detection method provided in an embodiment of the present invention;

[0042] Figure 2 This is a flowchart of a method for processing target images using a neural network model, provided in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of a neural network model structure provided in an embodiment of the present invention;

[0044] Figure 4 This is a flowchart of a neural network model training method provided in an embodiment of the present invention;

[0045] Figure 5 This is a flowchart of a method for extracting target head information provided in an embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of the structure of a social distancing detection device provided in an embodiment of the present invention;

[0047] Figure 7 This is a schematic diagram of the structure of a first processing module provided in an embodiment of the present invention;

[0048] Figure 8 This is a schematic diagram of another social distancing detection device provided in an embodiment of the present invention;

[0049] Figure 9 This is a schematic diagram of the structure of a second processing module provided in an embodiment of the present invention;

[0050] Figure 10 This is a schematic diagram of the structure of an extraction submodule provided in an embodiment of the present invention;

[0051] Figure 11 This is a schematic diagram of the structure of a third processing module provided in an embodiment of the present invention;

[0052] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Please see Figure 1 , Figure 1 This is a flowchart of a social distance detection method provided in an embodiment of the present invention, such as... Figure 1 As shown, this method is used for real-time social distance detection and includes the following steps:

[0055] 101. Obtain the first target image and the second target image.

[0056] In this embodiment of the invention, the first target image and the second target image were acquired at the same time from different angles. The first target image and the second target image include at least two target individuals.

[0057] Two cameras with different shooting angles can be used to capture images of the first and second targets respectively. The two cameras can be calibrated and linked during installation to ensure they operate on the same coordinate system and capture images simultaneously. Alternatively, a calibrated binocular camera can be used to capture the first and second target images. In this embodiment, it is preferable to use a calibrated binocular camera to capture the first and second target images, where the first and second target images can be the left-eye image and the right-eye image, respectively.

[0058] The first target image and the second target image mentioned above can be consecutive frame images (video stream images) or independent frame images (photographs).

[0059] 102. Input the first target image and the second target image into a pre-trained neural network model, and output the first head detection result, the second head detection result, the first head feature, and the second head feature through the pre-trained neural network model.

[0060] In this embodiment of the invention, the first head detection result and the first head feature correspond to the first target image, and the second head detection result and the second head feature correspond to the second target image.

[0061] Specifically, the head detection result can be a head detection bounding box (x, y, h, w), where x and y are the coordinates of the center point of the head detection bounding box, h is the height of the head detection bounding box, and w is the width of the head detection bounding box. The head detection bounding box can be understood as a small image within the target image, with each small image containing the head image of a target person.

[0062] The above-mentioned head features can be extracted from the head image using a head feature vector.

[0063] In one possible embodiment, the pre-trained neural network model can be multiple, including a head detection network model that performs head detection on the first target image and the second target image respectively, and a head feature extraction network model that performs head feature extraction on the first target image and the second target image respectively.

[0064] In this embodiment of the invention, to improve the speed of acquiring head detection results and head features, an optional neural network model is provided. The input is a first target image and a second target image, and the output is a first head detection result and a first head feature corresponding to the first target image, and a second head detection result and a second head feature corresponding to the second target image. That is, the head detection result and head features are obtained by processing the same neural network model. The input to the pre-trained neural network model is the first target image and the second target image. The neural network model includes a common network, a first branch network, and a second branch network. The common network is connected to both the first and second branch networks. The first branch network outputs the first head detection result and the second head detection result, and the second branch network outputs the first head feature and the second head feature.

[0065] For details, please see Figure 2 , Figure 2 This is a flowchart of a method for processing target images using a neural network model, as provided in an embodiment of the present invention. Figure 2 As shown, it includes the following steps:

[0066] 201. The first target image and the second target image are stitched together to obtain the input image.

[0067] The stitching of the first target image and the second target image can be done by stitching the first target image and the second target image vertically or by stitching them end-to-end, such as... Figure 3 As shown. The stitched first target image and the second target image can be used as a single input image and fed into the neural network model described above for processing.

[0068] 202. Input the input image into the pre-trained neural network model.

[0069] In this embodiment of the invention, the neural network model includes a common network, a first branch network, and a second branch network. The common network is connected to both the first and second branch networks. Figure 3 As shown.

[0070] The aforementioned public network is used to extract the basic features that can be shared between the first target image and the second target image in the input image, and obtain the basic feature map.

[0071] 203. Calculate and output the first head detection result and the second head detection result using the common network and the first branch network in the pre-trained neural network model.

[0072] In this embodiment of the invention, the aforementioned public network and the first branch network are used to process and output the first head detection result and the second head detection result.

[0073] The input image is processed through a public network to extract basic feature maps. These basic feature maps are then input into a first branch network, which performs head detection on them, outputting a first head detection result and a second head detection result. Specifically, the head detection result can be a head detection bounding box (x, y, h, w), where x and y are the coordinates of the center point of the head detection bounding box, h is the height of the head detection bounding box, and w is the width of the head detection bounding box. The first head detection result corresponds to the first target image, and the second head detection result corresponds to the second target image.

[0074] More specifically, the aforementioned first branch network includes a first common branch network, a first sub-network, and a second sub-network. The first sub-network detects and outputs a first head detection result, and the second sub-network detects and outputs a second head detection result. The first common branch network extracts shareable basic head detection features from the basic feature map, thus obtaining a basic head detection feature map. The first sub-network processes the basic head detection feature map to output the first head detection result, and the second sub-network processes the same basic head detection feature map to output the second head detection result. In essence, the first and second sub-networks process the basic head detection feature map using different weight matrices to obtain the first and second head detection results.

[0075] 204. Calculate and output the first head feature and the second head feature using the common network and the second branch network in the pre-trained neural network model.

[0076] In this embodiment of the invention, the aforementioned public network and second branch network are used to process and output the first head feature and the second head feature.

[0077] The input image is processed by a public network to extract a basic feature map. This basic feature map is then fed into a second branch network, which extracts head features from the basic feature map, resulting in the output of a first head feature and a second head feature. Specifically, these head features can be feature vectors of a preset dimension. The first head feature corresponds to the first target image, and the second head feature corresponds to the second target image.

[0078] More specifically, the aforementioned second branch network includes a second common branch network, a third sub-network, and a fourth sub-network. The third sub-network extracts and outputs the first head feature, and the fourth sub-network extracts and outputs the second head feature. The second common branch network extracts shareable basic head features from the basic feature map, thus obtaining a basic head feature map. The third sub-network processes the basic head feature map to output the first head feature, and the fourth sub-network processes the basic head feature map to output the second head feature. It can be understood that the first and fourth sub-networks process the basic head feature image using different weight matrices to obtain the first and second head features.

[0079] In this embodiment of the invention, a neural network model is used to simultaneously perform head detection and head feature extraction on the first target image and the second target image, which can improve the processing speed of head detection results and head features.

[0080] Optionally, the first branch network and the second branch network in the above neural network model are two different types of networks with different forward propagation structures. Simultaneous training during this process may lead to conflicts. Therefore, this embodiment of the invention also provides a neural network training method, which can be found in the following details. Figure 4 , Figure 4 This is a flowchart of a neural network model training method provided in an embodiment of the present invention, such as... Figure 4 As shown, it includes the following steps:

[0081] 401. Perform the first training on the public network and the first branch network using the first dataset.

[0082] In this embodiment of the invention, the first dataset mentioned above can be a dataset for head detection, which includes sample head images and corresponding head label data.

[0083] Furthermore, the aforementioned sample head images may include two images captured at the same time from different angles, both images containing the head images of the same person. For example, it could be a first sample head image and a second sample head image (i.e., a left-eye sample image and a right-eye sample image) captured by a binocular camera. Even further, the aforementioned sample head image is a head image stitched together from the first sample head image and the second sample head image.

[0084] The aforementioned first training refers to training the public network and the first branch network for head detection. In the overall head detection training, it is expected that all head features are as similar as possible. During the training of the first branch network, the distinction between the head and the background is trained. In the first sub-network and the second sub-network, the distinction between the first head detection result and the second head detection result is trained.

[0085] After obtaining the output results of the sample head images, the loss function of the result and the head label data is calculated, and the parameters of the common network and the first branch network are adjusted in reverse according to the loss function.

[0086] 402. Perform a second training on the public network and the second branch network using the second dataset.

[0087] In this embodiment of the invention, the second dataset mentioned above can be a dataset for extracting human head features, and the second dataset includes sample human head images and corresponding human head label data.

[0088] Furthermore, the aforementioned sample head images may include two images captured at the same time from different angles, both images containing the head images of the same person. For example, it could be a first sample head image and a second sample head image (i.e., a left-eye sample image and a right-eye sample image) captured by a binocular camera. Even further, the aforementioned sample head image is a head image stitched together from the first sample head image and the second sample head image.

[0089] The second training mentioned above refers to training the public network and the second branch network to extract head features. In the overall head feature extraction training, it is expected that the head features of different people are as dissimilar as possible in order to better distinguish different heads and thus extract different head features. In the training process of the second branch network, the differentiation and extraction of all head features are trained. In the third and fourth sub-networks, the differentiation and extraction of the first and second head features are trained.

[0090] After obtaining the output results of the sample head images, the loss function of the result and the head label data is calculated, and the parameters of the common network and the second branch network are adjusted in reverse according to the loss function.

[0091] It should be noted that the first training and the second training mentioned above are only to distinguish the training processes of the first branch network and the second branch network, and are not a limitation on the training sequence.

[0092] 403. Alternate between the first and second training sessions until the first and second training sessions converge to obtain a trained neural network model.

[0093] In this embodiment of the invention, the first training and the second training are performed alternately. This alternation can be understood as follows: after training the neural network model n times through the first training, the neural network model is then trained n times through the second training, and then trained n times through the first training again, in a cyclical alternation. Here, n is greater than or equal to 1. This improves the fitting effect of the common network, thereby simultaneously improving the accuracy of head detection and head feature extraction. The convergence of the first training can be understood as the minimum loss function corresponding to the common network and the first branch network, and the convergence of the second training can be understood as the minimum loss function corresponding to the common network and the second branch network, thus obtaining a well-trained neural network model.

[0094] Optionally, after the first and second training iterations converge, the converged neural network model can be fine-tuned to obtain a well-trained neural network model. Specifically, this fine-tuning can be achieved by adjusting the resolution to fine-tune the parameters of all networks in the neural network model.

[0095] 103. Based on the first head feature and the second head feature, obtain the first head detection result and the second head detection result of the target person, and extract the target head information based on the first head detection result and the second head detection result of the target person.

[0096] In this embodiment of the invention, the first head detection result and the first head feature are obtained based on the first target image, and the second head detection result and the second head feature are obtained based on the second target image. Therefore, it is necessary to match the heads in the first target image and the second target image to distinguish the target heads of different people, so as to better extract the target head information.

[0097] For details, please see Figure 5 , Figure 5 This is a flowchart of a method for extracting target head information provided in an embodiment of the present invention, such as... Figure 5 As shown, it includes the following steps:

[0098] 501. Based on the similarity between the first head feature and the second head feature, match the first head feature and the second head feature to obtain head feature pairs belonging to the same target person.

[0099] The aforementioned facial features of the same target person include the first and second facial features of the same target person.

[0100] In this embodiment of the invention, since the first head feature and the second head feature are extracted and output simultaneously through the same neural network model, the waiting time for head features can be reduced when calculating their similarity. The aforementioned similarity can be cosine similarity or Euclidean similarity.

[0101] In one possible embodiment, the loss function used by the aforementioned public network and second branch network is MSE loss (Mean Squared Error loss), also known as L2 loss. Since the calculation of MSE loss is similar to that of Euclidean distance, both of which involve calculating the mean squared error, using MSE loss to train the public network and second branch network, and using Euclidean distance to calculate the first and second head features output by the public network and second branch network, can yield a better performance and a better similarity metric. Moreover, the calculation is relatively simple and can improve the speed of similarity calculation.

[0102] By matching the first and second head features using similarity, the head features of the same person in the first and second target images are matched to obtain the corresponding head features. For example, by extracting the head features of the first and second target images through the public network and the second branch network, the first head feature of target person A is a1, and the second head feature is a2. The first head feature of target person B is b1, and the second head feature is b2. Then, the similarity S(a1, a2), S(a1, b2), S(b1, a2), and S(b1, b2) are calculated. If the similarity S(a1, a2) satisfies the preset similarity condition, then a1 and a2 are the head feature pair of the same target person (target person A). If S(b1, b2) satisfies the preset similarity condition, then b1 and b2 are the head feature pair of the same target person (target person B).

[0103] 502. Based on the head feature pairs, find the corresponding head detection result pairs.

[0104] The above head detection results include the detection results of the first person and the second person who are the same target.

[0105] In this embodiment of the invention, each first head feature corresponds to a first head detection result, and each second head feature corresponds to a second head detection result. The aforementioned head feature pairs include the first and second head features of the same target person, thereby enabling the retrieval of the first head detection result corresponding to the first head feature of the target person, and the retrieval of the second head detection result corresponding to the second head feature of the target person, based on the head feature pairs.

[0106] In one possible embodiment, the first target image and the second target image are consecutive frame images, and the first target image and the second target image include multiple target persons. According to a multi-target tracking algorithm, IDs can be assigned to the head detection result pairs of different target persons to obtain head detection result pairs with IDs. Different target persons correspond to different ID head detection result pairs. In this embodiment of the invention, the above-mentioned multi-target tracking algorithm can employ the deep sort tracking algorithm, trained using a head dataset, so that the deep sort tracking algorithm can learn to track the heads of target persons in the first target image and the second target image, and to assign IDs to the same target person in the first target image and the second target image. The above-mentioned deep sort tracking algorithm can track the heads of multiple target persons in real time and assign different IDs.

[0107] It should be noted that in multi-target tracking algorithms, since the head features corresponding to the head detection results have already been extracted through the neural network model, it is only necessary to reuse the first head features and the second head features. Therefore, in multi-target tracking algorithms, it is not necessary to extract head features again.

[0108] 503. Extract target head information based on head detection results.

[0109] In this embodiment of the invention, the head detection result includes a first head detection result and a second head detection result for the same target person. The first and second head detection results can be head detection boxes. Since the first and second target images were acquired from different angles, the head detection boxes corresponding to the first and second head detection results are also based on head images from different angles. It is understood that, based on research on the human eye, two images taken by a camera from two different perspectives can be obtained. If the correspondence of a real object point P in the two images is known, the three-dimensional coordinate information of point P can be accurately calculated. The real object point P can be various key points in the head, or it can be a SIFT (Scale-invariant feature transform) feature point, etc.

[0110] The aforementioned target head information can be the correspondence between keypoint pairs in the first head detection result and the second head detection result, or the correspondence between scale-invariant feature point pairs. The aforementioned correspondence can be the coordinate relationship between the two keypoints in the head detection box of the keypoint pair, or the coordinate relationship between the two scale-invariant feature points in the head detection box of the scale-invariant feature point pair.

[0111] 104. Perform three-dimensional reconstruction of the target head based on the target head information, and calculate the social distance between the target personnel based on the reconstructed three-dimensional target head.

[0112] In this embodiment of the invention, the aforementioned target head information can be the correspondence between keypoint pairs in the first head detection result and the second head detection result, or the correspondence between scale-invariant feature point pairs. Based on the head depth information, the target head can be reconstructed in three dimensions to obtain a three-dimensional target head.

[0113] Specifically, based on the principle of triangulation, depth information can be calculated by centering key points on two corresponding key points with different coordinates, thereby obtaining the depth information of each key point. Based on the depth information of each key point, a 3D reconstruction of the target head can be performed.

[0114] Calculating the social distance between target individuals based on the reconstructed 3D target heads can be achieved by projecting multiple reconstructed 3D target heads onto a preset plane (3D projection plane) in 3D space to obtain multiple target head projections; calculating the distance between different target head projections to obtain the projected distance between target individuals; and converting the projected distance between target individuals into the social distance between target individuals according to a preset ratio. The aforementioned 3D space can be constructed based on calibrated camera coordinates, and the origin of the 3D space can be the optical center point of the camera, or the center point of two optical centers.

[0115] Specifically, the geometric center of the target person's head projection can be calculated as the head projection point. The distances between each head projection point on the 3D projection surface are then calculated using Euclidean distance. The aforementioned preset ratio is a 3D spatial measurement compared to the actual spatial measurement, which can be determined based on depth information; the greater the depth value, the larger the matching ratio. The distances of the head projection points are then converted into the actual distances of the target person's head using the matched preset ratio, thus obtaining the social distance of the target person in the current scene.

[0116] In this embodiment of the invention, a first target image and a second target image are acquired, which are captured from different angles at the same time. The first and second target images are input into a pre-trained neural network model, and the model outputs a first head detection result, a second head detection result, a first head feature, and a second head feature. The first head detection result and the first head feature correspond to the first target image, and the second head detection result and the second head feature correspond to the second target image. Target head matching is performed based on the first head detection result, the second head detection result, the first head feature, and the second head feature, and target head information is extracted based on the matching results. The target head is then reconstructed in three dimensions based on the target head information, and the social distance between target individuals is calculated based on the reconstructed three-dimensional target head. By extracting more accurate target head information from head images of target individuals at different angles for three-dimensional reconstruction, the position of the three-dimensional target head in three-dimensional space is more accurate, thereby improving the accuracy of the social distance calculation between target individuals.

[0117] It should be noted that the social distance detection method provided in this embodiment of the invention can be applied to devices such as mobile phones, monitors, computers, and servers that are capable of social distance detection.

[0118] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a social distancing detection device provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the device includes:

[0119] The acquisition module 601 is used to acquire a first target image and a second target image, wherein the first target image and the second target image are acquired at the same time from different angles;

[0120] The first processing module 602 is used to input the first target image and the second target image into a pre-trained neural network model, and output a first head detection result, a second head detection result, a first head feature, and a second head feature through the pre-trained neural network model, wherein the first head detection result and the first head feature correspond to the first target image, and the second head detection result and the second head feature correspond to the second target image;

[0121] The second processing module 603 is used to obtain the first head detection result and the second head detection result of the target person by matching the first head feature and the second head feature, and to extract the target head information based on the first head detection result and the second head detection result of the target person;

[0122] The third processing module 604 is used to perform three-dimensional reconstruction of the target head based on the target head information, and to calculate the social distance between the target people based on the reconstructed three-dimensional target head.

[0123] Optionally, the pre-trained neural network model includes a common network, a first branch network, and a second branch network. The common network is connected to both the first and second branch networks. The first branch network is used to output a first head detection result and a second head detection result, and the second branch network is used to output a first head feature and a second head feature.

[0124] Optional, such as Figure 7 As shown, the first processing module 602 includes:

[0125] The preprocessing submodule 6021 is used to stitch the first target image and the second target image together to obtain the input image;

[0126] The input submodule 6022 is used to input the input image into a pre-trained neural network model;

[0127] The first processing submodule 6023 is used to calculate and output the first head detection result and the second head detection result through the common network and the first branch network in the pre-trained neural network model; and

[0128] The second processing submodule 6024 is used to calculate and output the first head feature and the second head feature through the common network and the second branch network in the pre-trained neural network model.

[0129] Optional, such as Figure 8 As shown, the device further includes:

[0130] The first training module 605 is used to perform the first training on the public network and the first branch network using the first dataset;

[0131] The second training module 606 is used to perform a second training on the public network and the second branch network using the second dataset;

[0132] The third training module 607 is used to alternate between the first training and the second training until the first training and the second training converge to obtain a trained neural network model.

[0133] Optional, such as Figure 9 As shown, a first head detection result corresponds to a first head feature, and a second head detection result corresponds to a second head feature. The second processing module 603 includes:

[0134] The first matching submodule 6031 is used to match the first head feature and the second head feature based on the similarity between the first head feature and the second head feature to obtain a head feature pair belonging to the same target person. The head feature pair includes the first head feature and the second head feature.

[0135] The second matching submodule 6032 is used to find the corresponding head detection result pair based on the head feature pair, wherein the head detection result pair includes the first head detection result and the second head detection result;

[0136] The extraction submodule 6033 is used to extract target head information based on the head detection result pair.

[0137] Optional, such as Figure 10 As shown, the extraction submodule 6033 includes:

[0138] The allocation unit 60331 is used to assign IDs to head detection result pairs of different target personnel according to the multi-target tracking algorithm, so as to obtain head detection result pairs with IDs;

[0139] Extraction unit 60332 is used to extract target head information based on the head detection result pair with ID.

[0140] Optionally, the third processing module 604 is further configured to perform three-dimensional reconstruction of the target head based on the head depth information to obtain a three-dimensional target head.

[0141] Optional, such as Figure 11 As shown, the third processing module 604 includes:

[0142] The projection submodule 6041 is used to project the reconstructed multiple three-dimensional target heads onto a preset plane in three-dimensional space to obtain multiple target head projections.

[0143] The calculation submodule 6042 is used to calculate the distance between the projections of different target heads, and to obtain the projection distance between the target people.

[0144] The conversion submodule 6043 is used to convert the projected distance between the target personnel into the social distance between the target personnel according to a preset ratio.

[0145] It should be noted that the social distancing detection device provided in this embodiment of the invention can be applied to devices such as mobile phones, monitors, computers, and servers that can perform social distancing detection.

[0146] The social distancing detection device provided in this embodiment of the invention can implement all the processes of the social distancing detection method in the above-described method embodiments, and can achieve the same beneficial effects. To avoid repetition, further details are omitted here.

[0147] See Figure 12 , Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 12 As shown, it includes: a memory 1202, a processor 1201, and a computer program stored in the memory 1202 and executable on the processor 1201, wherein:

[0148] Processor 1201 is used to call the computer program stored in memory 1202 and perform the following steps:

[0149] Acquire a first target image and a second target image, wherein the first target image and the second target image are acquired at the same time from different angles;

[0150] The first target image and the second target image are input into a pre-trained neural network model, and the first head detection result, the second head detection result, the first head feature, and the second head feature are output through the pre-trained neural network model. The first head detection result and the first head feature correspond to the first target image, and the second head detection result and the second head feature correspond to the second target image.

[0151] The first head detection result and the second head detection result of the target person are obtained by matching the first head feature and the second head feature, and the target head information is extracted based on the first head detection result and the second head detection result of the target person.

[0152] The target head is reconstructed in three dimensions based on the target head information, and the social distance between the target people is calculated based on the reconstructed three-dimensional target head.

[0153] Optionally, the pre-trained neural network model includes a common network, a first branch network, and a second branch network. The common network is connected to both the first and second branch networks. The first branch network is used to output a first head detection result and a second head detection result, and the second branch network is used to output a first head feature and a second head feature.

[0154] Optionally, the step of processor 1201 inputting the first target image and the second target image into a pre-trained neural network model, and outputting a first head detection result, a second head detection result, a first head feature, and a second head feature through the pre-trained neural network model, includes:

[0155] The first target image and the second target image are stitched together to obtain the input image;

[0156] The input image is fed into a pre-trained neural network model;

[0157] The first head detection result and the second head detection result are calculated and output using the common network and the first branch network in the pre-trained neural network model; and

[0158] The first head feature and the second head feature are calculated and output using the common network and the second branch network in the pre-trained neural network model.

[0159] Optionally, the processor 1201 also performs training on the neural network model, specifically by performing the following steps:

[0160] The first training was performed on the public network and the first branch network using the first dataset;

[0161] A second training was performed on the public network and the second branch network using the second dataset;

[0162] The first training and the second training are performed alternately until the first training and the second training converge to obtain a trained neural network model.

[0163] Optionally, a first head detection result corresponds to a first head feature, and a second head detection result corresponds to a second head feature. The processor 1201 executes the process of matching the first head feature and the second head feature to obtain the first head detection result and the second head detection result of the target person, and extracting the target head information based on the first head detection result and the second head detection result of the target person, including:

[0164] Based on the similarity between the first head feature and the second head feature, the first head feature and the second head feature are matched to obtain a head feature pair belonging to the same target person. The head feature pair includes the first head feature and the second head feature.

[0165] Based on the head feature pairs, the corresponding head detection result pairs are obtained, and the head detection result pairs include the first head detection result and the second head detection result;

[0166] Based on the head detection results, the target head information is extracted.

[0167] Optionally, the process of extracting target head information based on the head detection result pair executed by processor 1201 includes:

[0168] Based on the multi-target tracking algorithm, IDs are assigned to the head detection results of different target personnel to obtain head detection result pairs with IDs;

[0169] Based on the head detection results with IDs, extract the target head information.

[0170] Optionally, the target head information executed by the processor 1201 includes head depth information, and the step of performing three-dimensional reconstruction of the target head based on the target head information includes:

[0171] Based on the head depth information, the target head is reconstructed in three dimensions to obtain a three-dimensional target head.

[0172] Optionally, the calculation of social distance between target individuals based on the reconstructed three-dimensional target heads, performed by processor 1201, includes:

[0173] The reconstructed three-dimensional target heads are projected onto a preset plane in three-dimensional space to obtain multiple target head projections;

[0174] Calculate the distance between the projected heads of different targets to obtain the projected distance between the target individuals;

[0175] The projected distance between the target individuals is converted into a social distance between them according to a preset ratio.

[0176] It should be noted that the aforementioned electronic devices can be mobile phones, monitors, computers, servers, and other devices that can be used for social distancing detection.

[0177] The electronic device provided in this embodiment of the invention can implement all the processes of the social distance detection method in the above method embodiments and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0178] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the social distancing detection method provided in this invention and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0179] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0180] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A social distance detection method, characterized in that, Includes the following steps: Acquire a first target image and a second target image, wherein the first target image and the second target image are acquired at the same time from different angles; The first target image and the second target image are input into a pre-trained neural network model, and the first head detection result, the second head detection result, the first head feature, and the second head feature are output through the pre-trained neural network model. The first head detection result and the first head feature correspond to the first target image, and the second head detection result and the second head feature correspond to the second target image. Based on the first head feature and the second head feature, the first head detection result and the second head detection result belonging to the same target person are obtained by matching, and the target head information is extracted based on the first head detection result and the second head detection result belonging to the same target person; The target head is reconstructed in three dimensions based on the target head information, and the social distance between the target people is calculated based on the reconstructed three-dimensional target head.

2. The method as described in claim 1, characterized in that, The pre-trained neural network model includes a common network, a first branch network, and a second branch network. The common network is connected to both the first and second branch networks. The first branch network is used to output a first head detection result and a second head detection result, and the second branch network is used to output a first head feature and a second head feature.

3. The method as described in claim 2, characterized in that, The step of inputting the first target image and the second target image into a pre-trained neural network model, and outputting a first head detection result, a second head detection result, a first head feature, and a second head feature through the pre-trained neural network model, includes: The first target image and the second target image are stitched together to obtain the input image; The input image is fed into a pre-trained neural network model; The first head detection result and the second head detection result are calculated and output using the common network and the first branch network in the pre-trained neural network model; and The first head feature and the second head feature are calculated and output using the common network and the second branch network in the pre-trained neural network model.

4. The method as described in claim 2, characterized in that, The training of the neural network model includes: The first training was performed on the public network and the first branch network using the first dataset; A second training was performed on the public network and the second branch network using the second dataset; The first training and the second training are performed alternately until the first training and the second training converge to obtain a trained neural network model.

5. The method as described in claim 1, characterized in that, A first head detection result corresponds to a first head feature, and a second head detection result corresponds to a second head feature. The process of matching the first head feature and the second head feature to obtain the first head detection result and the second head detection result of the target person, and extracting target head information based on the first head detection result and the second head detection result of the target person, includes: Based on the similarity between the first head feature and the second head feature, a pair of head features belonging to the same target person is obtained, and the pair of head features includes the first head feature and the second head feature. Based on the head feature pairs, the corresponding head detection result pairs are obtained, and the head detection result pairs include a first head detection result and a second head detection result; Based on the head detection results, the target head information is extracted.

6. The method as described in claim 5, characterized in that, The step of extracting target head information based on the head detection result pair includes: Based on the multi-target tracking algorithm, IDs are assigned to the head detection results of different target personnel to obtain head detection result pairs with IDs; Based on the head detection results with IDs, extract the target head information.

7. The method as described in claim 5, characterized in that, The target head information includes head depth information, and the step of performing three-dimensional reconstruction of the target head based on the target head information includes: Based on the head depth information, the target head is reconstructed in three dimensions to obtain a three-dimensional target head.

8. The method as described in claim 7, characterized in that, The calculation of social distance between target individuals based on the reconstructed 3D target heads includes: The reconstructed three-dimensional target heads are projected onto a preset plane in three-dimensional space to obtain multiple target head projections; Calculate the distance between the projected heads of different targets to obtain the projected distance between the target individuals; The projected distance between the target individuals is converted into a social distance between them according to a preset ratio.

9. A social distancing detection device, characterized in that, The device includes: The acquisition module is used to acquire a first target image and a second target image, wherein the first target image and the second target image are acquired at the same time from different angles; The first processing module is used to input the first target image and the second target image into a pre-trained neural network model, and output a first head detection result, a second head detection result, a first head feature, and a second head feature through the pre-trained neural network model, wherein the first head detection result and the first head feature correspond to the first target image, and the second head detection result and the second head feature correspond to the second target image; The second processing module is used to obtain the first head detection result and the second head detection result belonging to the same target person by matching the first head feature and the second head feature, and to extract the target head information based on the first head detection result and the second head detection result belonging to the same target person; The third processing module is used to perform three-dimensional reconstruction of the target head based on the target head information, and to calculate the social distance between the target people based on the reconstructed three-dimensional target head.

10. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the social distancing detection method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the social distancing detection method as described in any one of claims 1 to 8.

Images