Bird-eye view peer identification method, device and equipment and medium

Abstract

The invention relates to a bird's-eye view angle peer identification method, device and equipment and a medium, and belongs to the field of computer vision and deep learning, the bird's-eye view angle peer identification method comprises the steps of obtaining a multi-scene monitoring image, preprocessing the multi-scene monitoring image, obtaining a depth image and a segmentation image, and obtaining the depth image and the segmentation image; a constructed improved YOLOv8s-seg model is trained on the basis of the depth map and the segmentation map, the improved YOLOv8s-seg model comprises a backbone network layer, a Neck layer, a Head layer and a depth output layer, a monitoring image to be recognized is predicted on the basis of the improved YOLOv8s-seg model which is completely trained, depth information of a human body target in a human body detection frame is obtained, and the depth information of the human body target in the human body detection frame is obtained. According to the method, the human body target is projected into the aerial view, the walking direction and speed of the human body target in the aerial view are analyzed based on the depth information of the human body target so as to identify the peers, and the recognition precision of the peers is enhanced under the aerial view angle.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and deep learning, and in particular to a method, device, equipment and medium for identifying pedestrians from a bird's-eye view. Background Art

[0002] In intelligent monitoring systems, pedestrian detection and tracking are crucial tasks and are widely used in urban security, intelligent transportation and other fields.

[0003] Traditional pedestrian detection methods rely on multi-camera systems and cannot provide accurate depth information from a single perspective. Existing technologies often have difficulty identifying pedestrians in complex scenes, especially from a bird's-eye view. The lack of comprehensive analysis of target depth and walking direction makes it difficult to achieve efficient and accurate recognition. Summary of the invention

[0004] In view of this, it is necessary to provide a method, device, equipment and medium for identifying pedestrians from a bird's-eye view to solve the technical problems of low efficiency and accuracy in identifying pedestrians from a bird's-eye view.

[0005] In order to solve the above problems, the present invention provides a method for identifying pedestrians from a bird's-eye view, comprising: Acquire a multi-scene monitoring image, pre-process the multi-scene monitoring image, and obtain a depth map and a segmentation map; Based on the depth map and the segmentation map, the constructed improved YOLOv8s-seg model is trained, wherein the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer and a depth output layer; Based on the well-trained improved YOLOv8s-seg model, the surveillance image to be identified is predicted to obtain the depth information of the human target in the human detection frame; The human target is projected into a bird's-eye view, and the walking direction and speed of the human target in the bird's-eye view are analyzed based on the depth information of the human target to identify the fellow travelers.

[0006] In a possible implementation, preprocessing the multi-scene surveillance images to obtain a depth map and a segmentation map includes: Processing the multi-scene surveillance image using a Depth-Anything-V2-Large model to generate a depth map corresponding to the multi-scene surveillance image; The SAM-vit_h model is used to segment the multi-scene monitoring images to generate a segmentation map.

[0007] In a possible implementation, the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer, and a depth output layer, and the backbone network layer includes a CBS module, a C2f module, an SPPF module, and a BiFPN; the training of the constructed improved YOLOv8s-seg model based on the depth map and the segmentation map includes: The depth map and the segmentation map are input into the improved YOLOv8s-seg model, and the image is subjected to feature extraction through the backbone network layer to obtain feature maps of multiple scales, wherein the CBS module convolves the input image, the C2f module extracts deep features from the convolved image, and the SPPF module extracts multi-scale features from the image after deep feature extraction to obtain feature maps of multiple scales, and the feature maps of multiple scales are bidirectionally connected and weightedly fused through the BiFPN; The weighted fused feature maps of multiple scales are input into the Neck layer. After the feature maps are fused through the Neck layer, the image after the feature fusion is input into the Head layer and the depth output layer. The image after the feature fusion is semantically segmented through the Head layer, and the depth of the image after the feature fusion is estimated through the depth output layer.

[0008] In a possible implementation, the training of the improved YOLOv8s-seg model constructed based on the depth map and the segmentation map further includes: The scale and smoothness of the depth map of the improved YOLOv8s-seg model are optimized by multiple loss functions, wherein the multiple loss functions include Scale-invariant and Smoothness.

[0009] In a possible implementation, the improved YOLOv8s-seg model based on the complete training predicts the surveillance image to be identified to obtain the depth information of the human target in the human detection frame, including: The surveillance image to be identified is input into the well-trained improved YOLOv8s-seg model, and after feature extraction of the backbone network layer, feature fusion of the Neck layer, image segmentation of the Head layer, and depth estimation of the depth output layer, a human detection frame, depth information, and segmentation information of the human target are generated; Based on the human body detection frame, depth information and segmentation information of the human body target, an average depth value of pixels in the human body detection frame is calculated, and the depth information of the human body target in the human body detection frame is obtained based on the average depth value.

[0010] In a possible implementation, projecting the human target into a bird's-eye view, and analyzing the walking direction and speed of the human target based on the depth information of the human target to identify a fellow traveler, includes: The DeepSORT algorithm and the human ReID model are used to track the human target to obtain a human target tracking ID; The human target is projected into a bird's-eye view, and based on the depth information of the human target, the depth information of the human target is updated using an exponential smoothing algorithm. Then, based on the updated depth information of the human target and the human target tracking ID, the walking direction and speed of the human target are analyzed to identify fellow travelers.

[0011] In a possible implementation, the calculation formula of the updated depth information of the human target is: , in, for The depth prediction value of the human target at the moment, is the output depth of the current human target, is the coefficient, for The depth of the human target at all times.

[0012] On the other hand, the present invention also provides a pedestrian recognition device from a bird's-eye view, comprising: An image preprocessing module is used to obtain multi-scene surveillance images, preprocess the multi-scene surveillance images, and obtain a depth map and a segmentation map; A model training module, used for training the constructed improved YOLOv8s-seg model based on the depth map and the segmentation map, wherein the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer and a depth output layer; A human depth information acquisition module is used to predict the surveillance image to be identified based on the improved YOLOv8s-seg model that has been trained to obtain the depth information of the human target in the human detection frame; The companion recognition module is used to project the human target into the bird's-eye view, and analyze the walking direction and speed of the human target in the bird's-eye view based on the depth information of the human target to recognize the companion.

[0013] In another aspect, the present invention further provides an electronic device, comprising: a processor and a memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the method for identifying pedestrians from a bird's-eye view as described above are implemented.

[0014] On the other hand, the present invention also provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the bird's-eye view pedestrian recognition method as described above.

[0015] The beneficial effects of the present invention are as follows: an improved YOLOv8s-seg model is trained based on a depth map and a segmentation map, wherein the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer and a depth output layer, the efficiency of the model is improved by sharing features between the Head layer and the depth output layer in the improved YOLOv8s-seg model, the scale and smoothness of the depth map of the improved YOLOv8s-seg model are optimized through multiple loss functions, the accuracy and stability of the depth information are improved, a human target is projected into a bird's-eye view, the walking direction and speed of the human target in the bird's-eye view are analyzed based on the depth information of the human target, so as to identify fellow travelers, and in a complex scene from a bird's-eye view, the depth estimation of the human target is combined with tracking, so as to enhance the recognition accuracy of fellow travelers. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flow chart of an embodiment of a method for identifying pedestrians from a bird's-eye view provided by the present invention; Figure 2 A schematic structural diagram of an embodiment of a pedestrian recognition device from a bird's-eye view provided by the present invention; Figure 3 A schematic structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0017] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0018] The present invention discloses a method, device, equipment and medium for identifying pedestrians from a bird's eye view, which can be used in a computer. The method, device or computer-readable storage medium involved in the present invention can be integrated with the above-mentioned device or can be relatively independent.

[0019] A specific embodiment of the present invention discloses a method for identifying pedestrians from a bird's-eye view, which can be executed by a computer, and specifically by one or more processors of the computer. Figure 1 As shown, the method for identifying pedestrians from a bird's-eye view includes: S101, acquiring multi-scene surveillance images, preprocessing the multi-scene surveillance images, and obtaining a depth map and a segmentation map; S102, training the constructed improved YOLOv8s-seg model based on the depth map and the segmentation map, wherein the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer, and a depth output layer; S103, predicting the surveillance image to be identified based on the well-trained improved YOLOv8s-seg model, and obtaining the depth information of the human target in the human detection frame; S104: Project the human target into the bird's-eye view, and analyze the walking direction and speed of the human target in the bird's-eye view based on the depth information of the human target to identify the walking person.

[0020] Among them, the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer and a depth output layer. The backbone network layer includes a CBS module, a C2f module, a SPPF module and a BiFPN. On the basis of the YOLOv8s-seg model, a depth output layer (Depth Head) for depth estimation is added. The depth output layer and the Head layer share the parameters of the Neck layer. The depth information of the input image is processed through the depth output layer, and a depth map is generated for each pixel in combination with the target detection and segmentation results. On the basis of the backbone network layer of the YOLOv8s-seg model, a cross-scale feature fusion technology (BiFPN) is introduced to enhance the perception of multi-scale objects.

[0021] Compared with the prior art, the method for pedestrian recognition from a bird's-eye view provided in this embodiment configures a human body detection frame based on the acquired multi-scene monitoring images, pre-processes the multi-scene monitoring images to obtain a depth map and a segmentation map; trains the constructed improved YOLOv8s-seg model based on the depth map and the segmentation map, wherein the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer and a depth output layer; the efficiency of the model is improved by sharing features between the depth estimation and segmentation tasks in the improved YOLOv8s-seg model, and the depth estimation and segmentation tasks are improved by using multiple loss functions. The loss function optimizes the scale and smoothness of the depth map of the improved YOLOv8s-seg model, thereby improving the accuracy and stability of the depth information. Based on the well-trained improved YOLOv8s-seg model, the surveillance image to be identified is predicted to obtain the depth information of the human target in the human detection frame, and the human target is projected into the bird's-eye view. Based on the depth information of the human target, the walking direction and speed of the human target in the bird's-eye view are analyzed to identify peers. In complex scenes from a bird's-eye view, the depth estimation of human targets is combined with tracking to enhance the recognition accuracy of peers.

[0022] In some embodiments, in step S101, a multi-scene surveillance image is obtained, three million multi-scene surveillance images are obtained, a high-precision human body detection frame is configured for each image, and the human body detection frame includes the human body target in the image. The image is annotated with high precision through the human body detection frame to obtain annotation data, and the annotation data provides a basis for subsequent model training; the multi-scene surveillance image is pre-processed to obtain a depth map and a segmentation map, that is, the three million multi-scene surveillance images are processed using a high-precision Depth-Anything-V2-Large model to generate a depth map corresponding to the multi-scene surveillance image, and the depth information of the image is obtained based on the depth map, and SAM-vit_h (Segment Anything The SAM-vit_h model performs segmentation processing on multi-scene surveillance images, generates segmentation maps, obtains image segmentation information based on the segmentation maps, and processes images through the Depth-Anything-V2-Large model and the SAM-vit_h model, providing accurate depth information and segmentation information for each image. The Depth-Anything-V2-Large model is a model for monocular depth estimation, which aims to solve the problem of monocular depth estimation. It performs well in providing diversified scene learning by using synthetic data sets for training, and also supports migration learning from synthetic data to real data sets to enhance the adaptability of the model under actual conditions. The SAM-vit_h model is a model based on visual Transformer. The (ViT) model generates a mask for any object in any image or video, that is, it segments the object area and generates the corresponding binary mask image. The model uses deep learning technology to train a large number of images and corresponding mask data sets, so that it can automatically identify objects in the image and generate corresponding masks. The advantage of the model is that it can handle any type of object, whether it is a natural object or an artificial object, and can generate accurate masks. In addition, it can also handle complex scenes, such as situations where multiple objects overlap or are occluded.

[0023] In some embodiments, in step S102, an improved YOLOv8s-seg model is constructed based on the YOLOv8s-seg model, the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer and a depth output layer, the backbone network layer includes a CBS module, a C2f module, an SPPF module and BiFPN, in addition to the classification and bounding box output of the YOLOv8s-seg model, a depth output layer is added, and the depth estimation task is processed separately through the depth output layer, that is, the depth information of the input image is processed through the depth output layer, and the target detection and segmentation results output by other branches of the YOLOv8s-seg model are combined to generate a depth map for each pixel. In addition, a cross-scale feature fusion technology (BiFPN) is introduced on the basis of the backbone network layer of the YOLOv8s-seg model; the improved YOLOv8s-seg The model integrates the tasks of detection, segmentation and depth estimation, and uses the Head layer and the depth output layer to share the features extracted by the convolution layer, thereby improving the efficiency of the model and reducing repeated calculations. The improved YOLOv8s-seg model constructed is trained based on the depth map and the segmentation map to obtain a fully trained improved YOLOv8s-seg model. The depth map and the segmentation map are input into the improved YOLOv8s-seg model, and the image is subjected to feature extraction through the backbone network layer to obtain feature maps of multiple scales. That is, after the input image is convolved by the CBS module of the backbone network layer, the deep feature extraction is performed by the C2f module of the backbone network layer, and the features are further extracted by the SPPF module to obtain feature maps of multiple scales. The feature maps of multiple scales are then bidirectionally connected and weightedly fused by the BiFPN of the backbone network layer to further improve the diversity and expression ability of the features.The weighted fused feature maps of multiple scales are input to the Neck layer. After the feature maps are fused through the Neck layer, the fused image is input to the Head layer and the depth output layer. The Head layer performs semantic segmentation on the fused image, and the depth output layer performs depth estimation on the fused image. The depth map of the improved YOLOv8s-seg model is optimized for scale and smoothness through multiple loss functions. Its multiple loss functions include Scale-invariant and Smoothness. The scale of depth estimation is optimized through the Scale-invariant loss function to reduce the difference between the predicted depth map and the actual depth map. The scale difference of the degree map enables the model to estimate the relative depth more accurately. The image gradient information is introduced through the Smoothness loss function to constrain the depth change of adjacent pixels, keep the depth edge clear, reduce noise and enhance geometric details, so that the improved YOLOv8s-seg model is smoother on the decision boundary and the improved YOLOv8s-seg model is optimized. The use of Scale-invariant and Smoothness loss functions at the same time can take into account the scale and smoothness of the improved YOLOv8s-seg model, thereby improving the performance and generalization ability of the model. The calculation formula of the Scale-invariant loss function is: , in, is the Scale-invariant loss function, that is, the loss function with unchanged scale and offset, is the number of valid pixels in the image, is the predicted value after scale and offset adjustment, is the ground truth value after scale and offset adjustment, is the loss function of the defined type; The calculation formula of its Smoothness loss function is: , in, is the Smoothness loss function, is the pixel in the image, Pixel The depth value at Pixel The image intensity value (brightness or color change information) at is the gradient of the depth map, is the gradient of image brightness (edge ​​information of the original image, indicating geometric details), It is a weight coefficient for controlling edge preservation, which is used to balance the smoothness and edge preservation of the depth map.

[0024] In some embodiments, in step S103, the surveillance image to be identified is predicted based on the well-trained improved YOLOv8s-seg model to obtain the depth information of the human target in the human detection frame, and the surveillance image to be identified is input into the well-trained improved YOLOv8s-seg model. After feature extraction of the backbone network layer, feature fusion of the Neck layer, image segmentation of the Head layer, and depth estimation of the depth output layer, a human detection frame, depth information, and segmentation information of the human target are generated under a monocular camera. Based on the human detection frame, depth information, and segmentation information of the human target, the average depth value of the pixels in the human detection frame is calculated, and the depth information of the human target in the human detection frame is obtained based on the average depth value. After directly generating the human detection frame, depth information, and segmentation information of the human target under a monocular camera using the improved YOLOv8s-seg model, the average depth value of the pixels in each human detection frame is calculated through the depth map matched with the human body area, and the average depth value of the pixels is used as the depth of the human target.

[0025] In some embodiments, in step S104, a human target is projected into a bird's-eye view, and the walking direction and speed of the human target in the bird's-eye view are analyzed based on the depth information of the human target to identify fellow travelers, and the DeepSORT algorithm and the human ReID model are used to track the human target to obtain a human target tracking ID, so that each human target has a tracking ID, and the corresponding human target is identified according to the tracking ID of the human target to ensure that the previous and next frames are continuous with the same person, and the human target is projected into the bird's-eye view, that is, the calibration parameters and image processing algorithm of the monocular camera are used to include the human target. The two-dimensional image of the target is converted into a bird's-eye view in three-dimensional space. The specific steps are: use camera calibration technology to obtain the intrinsic and extrinsic parameters of the camera to ensure the accuracy of the projection, use image processing algorithms such as perspective transformation to convert the two-dimensional image into a bird's-eye view, and use ground markers or objects of known size for correction to ensure the scale and accuracy of the bird's-eye view. Based on the depth information of the human target, the exponential smoothing algorithm is used to update the depth information of the human target. In order to avoid the influence of depth outliers, the exponential smoothing algorithm is used to fuse the depth information of the historical frame and the current frame, update the depth of the human body, and update the depth of the human body based on the updated depth information of the human target. The walking direction and speed of human targets in the bird's-eye view are analyzed based on the updated depth information and human target tracking ID. By judging whether the walking directions and speeds of multiple targets are similar within a certain period of time, the walking people are identified. That is, based on the updated depth information and human target tracking ID of the human target, the distances between multiple human targets are calculated, the relative position relationship of the human targets is analyzed, the walking direction and speed of the human targets are extracted, their motion trajectories are analyzed, and the time window is used to analyze the changes in the behavioral characteristics of the human targets within a period of time. When multiple targets maintain similar behavioral characteristics within a certain period of time, such as similar walking directions and speeds, they are determined to be walking people. Its DeepSORT algorithm is a multi-target tracking algorithm based on deep learning, which combines deep learning and Kalman filtering technology to improve the accuracy and stability of tracking. The core process of the DeepSORT algorithm includes three main steps: prediction, observation and update. First, the algorithm uses the Kalman filter to predict the target position (tracks) of the next frame. Then, the target is detected in the current frame, and the detected target is associated with the previous trajectory through data association. Finally, the prediction and detection results are combined for update to improve the tracking accuracy.The human ReID model (Re-identification) is a technology that uses computer vision technology to determine whether there is a specific pedestrian in an image or video. It is referred to as ReID and is a sub-problem of image retrieval. ReID learns the visual features of pedestrians and identifies the same pedestrian under multiple cameras. It is usually used for human tracking and identification in surveillance videos. The core of the ReID model lies in feature extraction and similarity measurement. The model first learns features that can cope with changes in pedestrians under different cameras, and then maps these features to a new space, making the same person closer and different people farther away. Finally, the image features are sorted according to the distance between them and the search results are returned. The DeepSORT algorithm introduces the ReID (Re-identification) network and uses a deep learning model to extract the apparent features of the target to help distinguish different targets. It can also correctly identify the target even when it is blocked or leaves the field of view and reappears. The calculation formula for the updated depth information of the human target is: , in, for The depth prediction value of the human target at the moment, is the output depth of the current human target, is the coefficient, for The depth of the human target at all times.

[0026] In order to better implement the method for identifying pedestrians from a bird's-eye view in the embodiment of the present invention, based on the method for identifying pedestrians from a bird's-eye view, correspondingly, Figure 2 As shown, the embodiment of the present invention further provides a bird's-eye view pedestrian recognition device, and the bird's-eye view pedestrian recognition device 200 includes: An image preprocessing module 201 is used to obtain multi-scene surveillance images, preprocess the multi-scene surveillance images, and obtain a depth map and a segmentation map; A model training module 202 is used to train the constructed improved YOLOv8s-seg model based on the depth map and the segmentation map, wherein the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer and a depth output layer; A human depth information acquisition module 203 is used to predict the surveillance image to be identified based on the well-trained improved YOLOv8s-seg model to obtain the depth information of the human target in the human detection frame; The companion identification module 204 is used to project the human target into the bird's-eye view, and analyze the walking direction and speed of the human target based on the depth information of the human target to identify the companion.

[0027] like Figure 3 As shown, the present invention also provides an electronic device 300 , which can be a computing device such as a mobile terminal, a desktop computer, a notebook, a palmtop computer, a server, etc. The electronic device 300 includes a processor 301 , a memory 302 , and a display 303 . Figure 3 Only some components of the electronic device 300 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0028] In some embodiments, the memory 302 may be an internal storage unit of the electronic device 300, such as a hard disk or memory of the electronic device 300. In other embodiments, the memory 302 may also be an external storage device of the electronic device 300, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 300. Further, the memory 302 may also include both an internal storage unit of the electronic device 300 and an external storage device. The memory 302 is used to store application software and various data installed in the electronic device 300, such as program codes installed in the electronic device 300. The memory 302 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a bird's-eye view pedestrian recognition program is stored on the memory 302, and the bird's-eye view pedestrian recognition program can be executed by the processor 301, thereby realizing the bird's-eye view pedestrian recognition method of each embodiment of the present invention.

[0029] In some embodiments, the processor 301 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 302, such as a method for identifying pedestrians from a bird's-eye view.

[0030] In some embodiments, the display 303 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 303 is used to display the identification information of the bird's-eye view pedestrian identification program and to display a visual user interface. The components 301-303 of the electronic device 300 communicate with each other through a system bus.

[0031] In some embodiments, when the processor 301 executes the bird's-eye view pedestrian recognition program in the memory 302, the various steps in the bird's-eye view pedestrian recognition method described in the above embodiments are implemented. Since the bird's-eye view pedestrian recognition method has been described in detail above, it will not be repeated here.

[0032] Accordingly, the present invention also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, it can implement the steps or functions of the bird's-eye view pedestrian recognition method provided in the above-mentioned method embodiments.

[0033] In summary, the method, device, equipment and medium for identifying pedestrians from a bird's-eye view provided by the present invention obtain multi-scene surveillance images, pre-process the multi-scene surveillance images, obtain depth maps and segmentation maps, and train the constructed improved YOLOv8s-seg model based on the depth maps and segmentation maps, wherein the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer and a depth output layer, and predicts the surveillance image to be identified based on the well-trained improved YOLOv8s-seg model to obtain the depth information of the human target in the human detection frame, and projects the human target into the bird's-eye view, and analyzes the walking direction and speed of the human target in the bird's-eye view based on the depth information of the human target to identify the pedestrians, thereby enhancing the recognition accuracy of the pedestrians from a bird's-eye view.

[0034] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0035] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for identifying pedestrians from a bird's-eye view, characterized in that: include: Acquire multi-scene surveillance images, preprocess the multi-scene surveillance images, and obtain depth maps and segmentation maps; Based on the depth map and the segmentation map, the constructed improved YOLOv8s-seg model is trained, wherein the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer and a depth output layer; Based on the well-trained improved YOLOv8s-seg model, the surveillance image to be identified is predicted to obtain the depth information of the human target in the human detection frame; The human target is projected into a bird's-eye view, and the walking direction and speed of the human target in the bird's-eye view are analyzed based on the depth information of the human target to identify the fellow travelers.

2. The method for identifying pedestrians from a bird's eye view according to claim 1, characterized in that: The preprocessing of the multi-scene monitoring images to obtain a depth map and a segmentation map includes: Processing the multi-scene surveillance image using a Depth-Anything-V2-Large model to generate a depth map corresponding to the multi-scene surveillance image; The SAM-vit_h model is used to segment the multi-scene monitoring images to generate a segmentation map.

3. The method for identifying pedestrians from a bird's eye view according to claim 2, characterized in that: The backbone network layer includes a CBS module, a C2f module, a SPPF module and a BiFPN; The improved YOLOv8s-seg model constructed based on the depth map and the segmentation map is trained, including: The depth map and the segmentation map are input into the improved YOLOv8s-seg model, and the depth map and the segmentation map are subjected to feature extraction through the backbone network layer to obtain feature maps of multiple scales, wherein the CBS module convolves the input image, the C2f module extracts deep features from the convolved image, and the SPPF module extracts multi-scale features from the image after the deep feature extraction to obtain feature maps of multiple scales, and the feature maps of multiple scales are bidirectionally connected and weightedly fused through the BiFPN; The weighted fused feature maps of multiple scales are input into the Neck layer. After the feature maps are fused through the Neck layer, the image after the feature fusion is input into the Head layer and the depth output layer. The image after the feature fusion is semantically segmented through the Head layer, and the depth of the image after the feature fusion is estimated through the depth output layer.

4. The method for identifying pedestrians from a bird's-eye view according to claim 3, characterized in that: The improved YOLOv8s-seg model constructed based on the depth map and the segmentation map is trained, and further includes: The scale and smoothness of the depth map of the improved YOLOv8s-seg model are optimized by multiple loss functions, wherein the multiple loss functions include Scale-invariant and Smoothness.

5. The method for identifying pedestrians from a bird's eye view according to claim 4, characterized in that: The improved YOLOv8s-seg model based on the complete training predicts the surveillance image to be identified to obtain the depth information of the human target in the human detection frame, including: The surveillance image to be identified is input into the well-trained improved YOLOv8s-seg model, and after feature extraction of the backbone network layer, feature fusion of the Neck layer, image segmentation of the Head layer, and depth estimation of the depth output layer, a human detection frame, depth information, and segmentation information of the human target are generated; Based on the human body detection frame, depth information and segmentation information of the human body target, an average depth value of pixels in the human body detection frame is calculated, and the depth information of the human body target in the human body detection frame is obtained based on the average depth value.

6. The method for identifying pedestrians from a bird's eye view according to claim 5, characterized in that: The projecting the human target into the bird's-eye view and analyzing the walking direction and speed of the human target based on the depth information of the human target to identify the walking person includes: The DeepSORT algorithm and the human ReID model are used to track the human target to obtain a human target tracking ID; The human target is projected into a bird's-eye view, and based on the depth information of the human target, the depth information of the human target is updated using an exponential smoothing algorithm. Then, based on the updated depth information of the human target and the human target tracking ID, the walking direction and speed of the human target are analyzed to identify fellow travelers.

7. The method for identifying pedestrians from a bird's eye view according to claim 6, characterized in that: The calculation formula of the updated depth information of the human target is: ， in, for The depth prediction value of the human target at the moment, is the output depth of the current human target, is the coefficient, for The depth of the human target at all times.

8. A device for identifying pedestrians from a bird's-eye view, characterized in that: include: An image preprocessing module is used to obtain multi-scene surveillance images, preprocess the multi-scene surveillance images, and obtain a depth map and a segmentation map; A model training module, used for training the constructed improved YOLOv8s-seg model based on the depth map and the segmentation map, wherein the improved YOLOv8s-seg model includes a backbone network layer, a Neck layer, a Head layer and a depth output layer; A human depth information acquisition module is used to predict the surveillance image to be identified based on the improved YOLOv8s-seg model that has been trained to obtain the depth information of the human target in the human detection frame; The companion recognition module is used to project the human target into the bird's-eye view, and analyze the walking direction and speed of the human target in the bird's-eye view based on the depth information of the human target to recognize the companion.

9. An electronic device, characterized in that: including memory and processor; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps of the method for identifying pedestrians from a bird's-eye view as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the bird's-eye view pedestrian recognition method as described in any one of claims 1-7.

Images