An object tracking method and device across cameras, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a cross-camera object tracking method and device, electronic equipment and storage medium, the method comprising: acquiring original images collected by different cameras; in the overlapping area of the original images collected by different cameras, determining different weight information according to the distance between the object and the camera, and performing fusion processing on the overlapping area according to the weight information to obtain a fused image; determining the object identifier in the fused image and the position information corresponding to the object identifier according to a pre-trained recognition model; the recognition model is obtained by training an initial network model using sample image data, and the initial network model is obtained by improving the speed and accuracy of a neural network model; the position information of the object under different cameras is tracked using a preset tracking algorithm, improving the speed and accuracy of model recognition, reducing the workload, and reducing the problem that the object cannot be tracked in real time when moving across cameras.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a cross-camera object tracking method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the acceleration of urbanization process and the continuous rise of security needs, intelligent monitoring systems are increasingly important in public safety, traffic management, business operations and many other fields. As the core component of intelligent monitoring systems, the development of video monitoring technology plays a key role in improving monitoring efficiency, enhancing security and optimizing resource allocation.

[0003] Traditional indoor monitoring adopts a single-camera discrete deployment mode, and its core logic is to achieve visual coverage of a local area by separately deploying cameras at key points in the monitoring area. However, the shooting range of a single camera is limited, so multiple cameras can be used to track pedestrians. However, due to differences in camera installation location or lighting conditions, the same pedestrian may appear distorted in different camera images, making it difficult to accurately track the same pedestrian. Therefore, it is necessary to improve the accuracy of tracking the same pedestrian by different cameras. SUMMARY

[0004] Some embodiments of the present application aim to provide a cross-camera object tracking method and device, electronic equipment and storage medium. Through the technical solutions of the embodiments of the present application, the original images collected by different cameras are obtained; in the overlapping area of the original images collected by the different cameras, different weight information is determined according to the distance between the object and the camera, and the overlapping area is fused according to the weight information to obtain a fused image; the object identifier and the position information corresponding to the object identifier in the original image and the fused image are determined according to a pre-trained recognition model; wherein the pre-trained recognition model is obtained by training a sample image data on an initial network model, and the initial network model is obtained by improving the speed and accuracy of a neural network model; a preset tracking algorithm is used to track the position information of the object corresponding to the object identifier under different cameras. The overlapping area fusion method based on the position weight ratio allocation strategy in the embodiments of the present application fuses the information of the overlapping area of adjacent cameras according to the real-time change of the pixel weight, improves the continuity and accuracy of the cross-camera moving person tracking, realizes the continuous and smooth visual transition of the moving person under the multi-camera visual angle, and at the same time, the recognition model is obtained by improving the speed and accuracy of the neural network model. In this way, the overlapping area of the original images collected by different cameras is fused, then the object in the fused image is detected and recognized according to the recognition model, the position information of the object is determined, and then a preset tracking algorithm is used to track the position information of the object corresponding to the object identifier under different cameras. The speed and accuracy of model recognition are improved, the workload is reduced, and the problem that the object cannot be tracked in real time when the object moves across the camera is solved.

[0005] In a first aspect, some embodiments of the present application provide a cross-camera object tracking method, comprising:

[0006] Obtaining original images collected by different cameras;

[0007] In the overlapping area of the original images collected by the different cameras, different weight information is determined according to the distance between the object and the camera, and the overlapping area is fused according to the weight information to obtain a fused image;

[0008] Determining the object identifier and the position information corresponding to the object identifier in the original image and the fused image according to a pre-trained recognition model; wherein the pre-trained recognition model is obtained by training a sample image data on an initial network model, and the initial network model is obtained by improving the speed and accuracy of a neural network model;

[0009] Tracking the position information of the object corresponding to the object identifier under different cameras using a preset tracking algorithm.

[0010] Some embodiments of the application improve the continuity and accuracy of moving person tracking across cameras by fusing information of overlapping regions of adjacent cameras according to real-time changes in pixel weights using an overlapping region fusion method based on a position weight ratio allocation strategy, and achieve continuous and smooth visual transition of a moving person under multi-camera visual angles. In addition, the recognition model is obtained by improving the speed and accuracy of the neural network model. In this way, the overlapping regions of the original images collected by different cameras are fused, and then the objects in the fused images are detected and recognized according to the recognition model to determine the position information of the objects. Then, a preset tracking algorithm is used to track the position information of the objects corresponding to the object identifiers under different cameras, thereby improving the speed and accuracy of model recognition, reducing the workload, and reducing the problem of inability to track in real time when the object moves across cameras.

[0011] Optionally, the different weight information is determined according to the distance between the object and the camera in the overlapping region of the original images collected by the different cameras, and the overlapping region is fused according to the weight information to obtain a fused image, including:

[0012] An overlapping region image in a first image and a second image collected by a first camera is obtained.

[0013] Weight information corresponding to the position information of the object in the overlapping region image is determined according to the position information of the object in the overlapping region image.

[0014] The fused image is determined according to the weight information, first pixels of the overlapping region image in the first image, and second pixels of the overlapping region image in the second image.

[0015] Some embodiments of the application improve the continuity and accuracy of moving person tracking across cameras by fusing information of overlapping regions of adjacent cameras according to real-time changes in pixel weights using an overlapping region fusion method based on a position weight ratio allocation strategy, and achieve continuous and smooth visual transition of a moving person under multi-camera visual angles.

[0016] Optionally, the weight information corresponding to the position information of the object in the overlapping region image is determined according to the position information of the object in the overlapping region image, including:

[0017] The distance of the object to a preset edge is determined based on the overlapping region.

[0018] The weight information is determined according to the distance of the object to the preset edge and a preset change rate.

[0019] Some embodiments of the present application are fused, which can weaken the small proportion of camera image pixels on the pixel points, reduce the appearance of redundant bounding boxes, and ensure that the fused image can present a smooth transition change.

[0020] Optionally, the pre-trained identification model is obtained by the following method:

[0021] Obtain sample image data, the sample image data being original images collected by different cameras, the sample image data including a preset object identifier and first position information corresponding to the preset object identifier;

[0022] Determine an overlapping region image of the original images collected by the different cameras, and fuse the overlapping region image to obtain a sample fusion image;

[0023] Train the initial network model according to the sample fusion image to obtain second position information of the preset object identifier, wherein the initial network model is obtained by improving the speed and accuracy of a YOLOv5 model;

[0024] Compare the first position information and the second position information, and continuously update the initial network model, and in the case that the first position information and the second position information match, determine the initial network model corresponding to the second position information as the identification model.

[0025] Optionally, the initial network model is obtained by improving the speed and accuracy of a YOLOv5 model, including:

[0026] The main network of the YOLOv5 model is replaced with a ShuffleNetv2 main network by modifying a network structure configuration file of the YOLOv5 to obtain an improved model;

[0027] An attention mechanism is added to the improved model to obtain the initial network model.

[0028] Some embodiments of the present application use a lightweight convolutional neural network ShuffleNetv2 to replace the CSPDarknet structure in the YOLOv5s main network to greatly improve the speed of the detection algorithm, and an attention mechanism is added to the model to improve the accuracy of object detection.

[0029] Optionally, the preset tracking algorithm is used to track the position information of the object corresponding to the object identifier under different cameras, including:

[0030] The DeepSort tracking algorithm is used to track the position information of the object corresponding to the object identifier under different cameras.

[0031] In a second aspect, some embodiments of the present application provide a cross-camera object tracking device, comprising:

[0032] An acquisition module configured to acquire original images captured by different cameras;

[0033] A fusion module configured to determine different weight information according to distances between an object and the cameras in an overlapping area of the original images captured by the different cameras, and perform fusion processing on the overlapping area according to the weight information to obtain a fused image;

[0034] A determination module configured to determine an object identifier and position information corresponding to the object identifier in the original images and the fused image according to a pre-trained recognition model, wherein the pre-trained recognition model is obtained by training a sample image data on an initial network model, and the initial network model is obtained by improving a speed and accuracy of a neural network model;

[0035] A tracking module configured to track position information of an object corresponding to the object identifier under different cameras using a preset tracking algorithm.

[0036] Some embodiments of the present application improve the continuity and accuracy of cross-camera moving person tracking by using an overlapping area fusion method based on a position weight ratio allocation strategy, fuse information of overlapping areas of adjacent cameras according to real-time changes of pixel weights, realize continuous and smooth visual transition of moving persons under multi-camera visual angles, and improve the speed and accuracy of model recognition by improving a speed and accuracy of a neural network model, thereby reducing workload and solving the problem that an object cannot be tracked in real time when moving across cameras.

[0037] Optionally, the fusion module is configured to:

[0038] acquire an overlapping area image in the first image and the second image captured by the first camera;

[0039] determine weight information corresponding to the position information according to the position information of the object in the overlapping area image;

[0040] determine the fused image according to the weight information, first pixels of the overlapping area image in the first image, and second pixels of the overlapping area image in the second image.

[0041] Some embodiments of the present application improve the continuity and accuracy of cross-camera moving person tracking by fusing the information of the overlapping areas of adjacent cameras according to the real-time changes of pixel weights based on a position weight ratio allocation strategy, and realize continuous and smooth visual transition of the moving person under multi-camera visual angle.

[0042] Optionally, the fusion module is configured to:

[0043] determine the distance of the object to a preset edge based on the overlapping area;

[0044] determine the weight information according to the distance of the object to the preset edge and a preset change rate.

[0045] Some embodiments of the present application weaken the small proportion of camera image pixels on the pixel points after fusion processing, reduce the appearance of redundant bounding boxes, and ensure that the fused image can present a smooth transition change.

[0046] Optionally, the device further comprises a training module configured to:

[0047] obtain sample image data, the sample image data being original images collected by different cameras, the sample image data comprising a preset object identifier and first position information corresponding to the preset object identifier;

[0048] determine an overlapping area image of the original images collected by the different cameras, and fuse the overlapping area image to obtain a sample fusion image;

[0049] train the initial network model according to the sample fusion image to obtain second position information of the preset object identifier, wherein the initial network model is obtained by improving the speed and accuracy of a YOLOv5 model;

[0050] compare the first position information and the second position information, continuously update the initial network model, and determine the initial network model corresponding to the second position information as the recognition model in the case that the first position information and the second position information match.

[0051] Optionally, the initial network model is obtained by improving the speed and accuracy of a YOLOv5 model, comprising:

[0052] modifying a network structure configuration file of the YOLOv5 to replace the backbone network of the YOLOv5 model with a ShuffleNetv2 backbone network to obtain an improved model;

[0053] adding an attention mechanism in the improved model to obtain the initial network model.

[0054] Some embodiments of the present application use a lightweight convolutional neural network ShuffleNetv2 to replace the CSPDarknet structure in the YOLOv5s backbone network, which can greatly improve the speed of the detection algorithm, and at the same time, increase the attention mechanism in the model to improve the accuracy of object detection.

[0055] Optionally, the tracking module is configured to:

[0056] The DeepSort tracking algorithm is used to track the position information of the object corresponding to the object identifier under different cameras.

[0057] In a third aspect, some embodiments of the present application provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the cross-camera object tracking method according to any one of the embodiments of the first aspect.

[0058] In a fourth aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, wherein the program is executable by a processor to implement the cross-camera object tracking method according to any one of the embodiments of the first aspect.

[0059] In a fifth aspect, some embodiments of the present application provide a computer program product, comprising a computer program, wherein the computer program is executable by a processor to implement the cross-camera object tracking method according to any one of the embodiments of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the following will briefly introduce the drawings needed to be used in some embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0061] Figure 1 A flowchart of a cross-camera object tracking method provided by an embodiment of the present application is shown in the figure;

[0062] Figure 2 A schematic diagram of object tracking in the prior art is shown in the figure;

[0063] Figure 3 A schematic diagram of image fusion provided by an embodiment of the present application is shown in the figure;

[0064] Figure 4 A schematic diagram of model improvement provided by an embodiment of the present application is shown in the figure;

[0065] Figure 5 A schematic diagram of cross-camera object tracking provided for an embodiment of the present application;

[0066] Figure 6 A structural schematic diagram of a cross-camera object tracking device provided for an embodiment of the present application;

[0067] Figure 7 A schematic diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in some embodiments of the present application will be described below in combination with the drawings in some embodiments of the present application.

[0069] It should be noted that similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms “first”, “second”, and the like are only used for differentiation in description, and cannot be understood as indicating or implying relative importance.

[0070] With the acceleration of urbanization and the continuous rise of security needs, intelligent monitoring systems are increasingly important in public safety, traffic management, commercial operations and many other fields. As the core component of intelligent monitoring systems, the development of video monitoring technology plays a key role in improving monitoring efficiency, enhancing security, and optimizing resource allocation.

[0071] Traditional indoor monitoring mostly adopts a single-camera discrete deployment mode, whose core logic is to achieve visual coverage of a local area by separately deploying cameras at key points in the monitoring area.

[0072] When tracking pedestrians in a low-altitude large scene (such as a large shopping mall atrium, a traffic hub hall, etc.), this mode needs to complete the cross-camera continuous tracking of the same pedestrian in steps: first, a single camera obtains real-time pictures through an image acquisition module, extracts the appearance features (such as clothing color, contour structure), behavior features (such as gait regularity), and local features (such as hairstyle, carried items) of the pedestrian with the help of HOG (Histogram of Oriented Gradients), CNN (Convolutional Neural Network), etc. algorithms, and establishes a temporary feature template for the target pedestrian; then, when the pedestrian moves to the monitoring range of the adjacent camera, the new camera repeats the above feature extraction process to generate new feature data; finally, the system compares the features extracted by the new camera with the pedestrian feature template established in the early stage through a feature matching algorithm (such as cosine similarity calculation, Euclidean distance comparison), and if the matching degree exceeds the preset threshold, it is determined as the same target, completing the tracking connection.

[0073] However, this discrete tracking mode has significant defects in low-altitude large scenes: on the one hand, different cameras will cause feature distortion of the same pedestrian in different camera pictures due to differences in installation angle and lighting conditions (such as color deviation of clothes caused by backlight and deformation of contour proportion caused by low-angle shooting), directly reducing the feature matching accuracy; on the other hand, when the density of pedestrians in the scene is high, occlusion and crossing walking are easy to occur, resulting in incomplete feature information extracted by a single camera, and then causing problems such as "losing" and "wrong tracking" (such as misjudging pedestrians with similar clothes as the same target) in cross-camera matching.

[0074] These tracking error problems directly promote the fusion application of centralized video stitching and cross-camera tracking technology. The centralized scheme arranges the discrete video streams collected by multiple cameras in a matrix according to the relative position information, generates a panoramic picture covering the entire large scene, and then extracts and detects the features of pedestrians in multiple camera pictures to find the same pedestrian in multiple cameras, realizing continuous tracking across cameras.

[0075] When multiple discrete camera videos are arranged in a matrix, due to the low-angle shooting of the cameras, in the overlapping area between adjacent cameras, the low viewing angle causes the edge of the upper body of the pedestrian to be significantly missing when the pedestrian moves from the field of view of one camera to the field of view of another camera. The tracking algorithm has difficulty in extracting features in consecutive frames, and when the viewing angles of adjacent cameras are different, the front and back of the pedestrian may be captured respectively. Therefore, the edge features of the pedestrian are lost when crossing cameras, and the tracking algorithm is easy to misidentify new targets, affecting continuous monitoring.

[0076] As shown in Figure 2 , different cameras shoot the same pedestrian at the same position, and due to the difference in angle and the large amount of missing body parts of the pedestrian, the tracking algorithm is easy to misidentify it as a new target, and cannot accurately track it.

[0077] In view of this, some embodiments of the present application provide a cross-camera object tracking method, which comprises: acquiring original images collected by different cameras; in an overlapping area of the original images collected by different cameras, determining different weight information according to the distance between an object and a camera, and performing fusion processing on the overlapping area according to the weight information to obtain a fused image; determining an object identifier in the fused image and position information corresponding to the object identifier according to a pre-trained recognition model; wherein the pre-trained recognition model is obtained by training a sample image data on an initial network model, and the initial network model is obtained by improving the speed and accuracy of a neural network model; and tracking the position information of the object corresponding to the object identifier under different cameras using a preset tracking algorithm. The overlapping area fusion method based on the position weight ratio allocation strategy in the embodiments of the present application fuses the information of the overlapping areas of adjacent cameras according to the real-time change of the pixel weight, improves the continuity and accuracy of the tracking of the moving person across the cameras, realizes the continuous and smooth visual transition of the moving person under the multi-camera visual angle, and improves the speed and accuracy of the model recognition while reducing the workload and the problem that the object cannot be tracked in real time when moving across the cameras.

[0078] As Figure 1 indicated, the embodiments of the present application provide a cross-camera object tracking method, which comprises:

[0079] S101, acquiring original images collected by different cameras;

[0080] The embodiments of the present application are applied to an indoor low-altitude large scene (usually a monitoring height of 3-8 meters from the ground) as the core area with the most intensive personnel activities and the most frequent interactions, and this kind of scene puts forward higher requirements for the continuous tracking of pedestrian trajectories, the early warning of abnormal behaviors, the regulation of crowd density and other functions.

[0081] In the low-altitude indoor environment, a plurality of cameras are arranged in a matrix arrangement, the cameras are installed in a top-down posture at a fixed distance, two adjacent cameras collect original images under a certain scene, and the collected original images are sent to a terminal device.

[0082] S102, in an overlapping area of the original images collected by different cameras, determining different weight information according to the distance between an object and a camera, and performing fusion processing on the overlapping area according to the weight information to obtain a fused image;

[0083] Specifically, the terminal device identifies the original images collected by different cameras, obtains an overlapping area of the original images collected by different cameras, and for the overlapping area, converts image processing from different cameras into processing of a single fused image by performing pixel fusion on the pictures of different cameras at the same position. The overlapping area is directly fused, that is, different weight information is determined according to the distance between the object and the camera, and the overlapping area is fused according to the weight information to obtain a fused image.

[0084] S103, determining, according to a pre-trained recognition model, an object label in the fused image and position information corresponding to the object label; wherein the pre-trained recognition model is obtained by training a sample image data on an initial network model, and the initial network model is obtained by improving the speed and accuracy of a neural network model.

[0085] Specifically, a neural network, for example, a YOLOv5s model, is pre-set on the terminal device. Since the present application is applied to real-time tracking of objects, for example, pedestrians, the requirement for processing speed is high, so the neural network model is improved in speed, that is, a lightweight network ShuffleNetV2 is used to replace the backbone network of YOLOv5s, and then in order to accurately detect the position information of the pedestrians, the neural network model is improved in accuracy, for example, an attention mechanism is added, to obtain an initial network model, and then the initial network model is trained by using sample image data to obtain a recognition model, which is used to detect objects in an image.

[0086] After the terminal device obtains the fused image, the original image and the fused image are input into the recognition model to obtain the object label in the original image and the object label in the fused image, as well as the position information corresponding to the object label.

[0087] S104, using a preset tracking algorithm to track the position information of the object corresponding to the object label under different cameras.

[0088] Specifically, the terminal device uses a preset tracking algorithm to track the recognized object label and position information.

[0089] In the embodiments of the present application, indoor multiple cameras are arranged in a matrix, the overlapping area fusion based on the position weight ratio allocation strategy is used, and a detection algorithm is used for pedestrian tracking detection. Here, the improved YOLOv5 algorithm is used to extract the features of the target pedestrians, and the DeepSort tracking algorithm is used for pedestrian tracking. Thus, when tracking and detecting pedestrians across cameras, unlike the traditional method of simultaneously extracting pedestrian features from multiple cameras, the embodiments of the present application first perform pixel fusion on the edge overlapping areas of adjacent cameras, assign different weight values to each pixel point in the overlapping area according to the distance of the pixel point from the center of the field of view of the two cameras, and then perform pixel superposition. Among them, the pixel points closer to the center are given higher weights, while the pixel points farther away are given lower weights, so as to ensure that the overlapping area can achieve a smooth transition effect, thereby improving the accuracy of the identity label. Then, a detection algorithm (based on the requirement of real-time, the improved YOLOv5_DeepSort algorithm is used) is used to track and label pedestrians across cameras.

[0090] Some embodiments of the present application use the overlapping area fusion method based on the position weight ratio allocation strategy to fuse the information of the overlapping areas of adjacent cameras according to the real-time changes of the pixel weights, improve the continuity and accuracy of tracking moving people across cameras, and achieve continuous and smooth visual transition of moving people under the view of multiple cameras. At the same time, the recognition model is obtained by improving the speed and accuracy of the neural network model. In this way, the overlapping areas of the original images collected by different cameras are fused, then the objects in the fused image are detected and recognized according to the recognition model to determine the position information of the objects, and then a preset tracking algorithm is used to track the position information of the objects corresponding to the object labels under different cameras, thereby improving the speed and accuracy of model recognition, reducing the workload, and reducing the problem of unable to track in real time when the object moves across cameras.

[0091] Another embodiment of the present application further supplements the cross-camera object tracking method provided in the above embodiments.

[0092] Optionally, in the overlapping area of the original images collected by different cameras, different weight information is determined according to the distance between the object and the camera, and the overlapping area is fused according to the weight information to obtain a fused image, including:

[0093] Obtain the overlapping area image in the first image and the second image collected by the first camera;

[0094] According to the position information of the object in the overlapping area image, determine the weight information corresponding to the position information;

[0095] According to the weight information, the first pixel of the overlapping area image in the first image and the second pixel of the overlapping area image in the second image, determine the fused image.

[0096] Some embodiments of this application employ an overlapping region fusion method based on a position weight ratio allocation strategy. By fusing information from overlapping regions of adjacent cameras according to real-time changes in pixel weights, the continuity and accuracy of moving person tracking across cameras are improved, enabling continuous and smooth visual transitions of moving persons under multi-camera perspectives.

[0097] Optionally, weight information corresponding to the position information is determined based on the position information of the object in the overlapping region image, including:

[0098] Based on the overlapping area, determine the distance from the object to the preset edge;

[0099] Weight information is determined based on the distance of the object to the preset edge and the preset rate of change.

[0100] like Figure 3 As shown in the embodiment of this application, the camera is installed in a top-down position and fixed at a certain distance. When a pedestrian steps into the overlapping area of ​​adjacent cameras, different cameras will capture images with different perspectives and a large number of missing upper body features. In subsequent pedestrian detection and tracking, different identity marker boxes will appear in the different camera images of the same pedestrian in the overlapping area, resulting in identity recognition errors and making continuous tracking impossible.

[0101] For overlapping regions, pixel fusion is performed on images from different cameras at the same location. The image processing from different cameras is converted into processing of the fused single image. The overlapping regions are directly fused. When the pixel weight ratio of multiple cameras is the same for each pixel, it is difficult to achieve a smooth transition between the overlapping and non-overlapping regions. When a person crosses the overlapping region, redundant bounding boxes may still appear when detecting pedestrians in the fused overlapping region because the pixels of multiple cameras are the same. This paper proposes a pixel fusion strategy based on position weight ratio for overlapping regions. The calculation method is shown in the following formula (1).

[0102]

[0103] Where: I represents the fused image, and I1 and I2 are the two overlapping images that need to be fused, specifically referring to the pixel values ​​of different images at the same location. This is a parameter that controls the weights, with a value range of [0,1]. It is used to adjust the contribution ratio of each image in the final fusion result. r is the distance of the pixel in the direction of pedestrian movement from the edge l of the overlapping region. It is an adjustment parameter used to control the rate at which the weights change with distance.

[0104] Within the overlapping region, for each pixel, based on its position relative to the edge of the overlapping region... The distance is used to dynamically adjust the weights.

[0105] Specifically, as shown in formula (2), when the position point is close to the edge of the overlapping area Distance is , the weight value Changes dynamically with After the above fusion operation, the pixels of the camera image with a small proportion are weakened, the appearance of redundant bounding boxes is reduced, and the fused image can present a smooth transition change.

[0106] Some embodiments of the application weaken the pixels of the camera image with a small proportion after fusion processing, reduce the appearance of redundant bounding boxes, and ensure that the fused image can present a smooth transition change.

[0107] Optionally, the pre-trained recognition model is obtained by the following method:

[0108] Obtain sample image data, the sample image data being original images collected by different cameras, the sample image data including a preset object identifier and first position information corresponding to the preset object identifier;

[0109] Determine an overlapping area image of the original images collected by different cameras, and fuse the overlapping area image to obtain a sample fusion image;

[0110] Train an initial network model according to the sample fusion image to obtain second position information of the preset object identifier, wherein the initial network model is obtained by improving the speed and accuracy of a YOLOv5 model;

[0111] Compare the first position information and the second position information, and continuously update the initial network model, and in the case that the first position information and the second position information match, determine the initial network model corresponding to the second position information as the recognition model.

[0112] Specifically, in the embodiment of the application, since the moving object, such as a pedestrian, is tracked in real time, the real-time pedestrian tracking requires that the processing speed should be not greater than 50 ms (not less than 20 frames per second), including video frame fusion time and tracking algorithm execution time. The lightweight improvement of the tracking algorithm is the key to ensure the processing speed. Using a lightweight convolutional neural network ShuffleNetv2 to replace the CSPDarknet structure in the YOLOv5s backbone network can greatly improve the speed of the detection algorithm.

[0113] Specifically, using ShuffleNetv2 to replace CSPDarknet in the YOLOv5s backbone network includes the following steps:

[0114] In the source code directory of YOLOv5, find the YAML file (such as yolov5s.yaml) that defines the network structure, and replace the original backbone network part (CSPDarknet) with the newly designed ShuffleNetv2 backbone network in the file.

[0115] Using the modified network structure configuration file, combine the training script of YOLOv5 to train the model, and in the training process, the model can be optimized according to the performance feedback of the validation set, such as adjusting the learning rate, optimizer parameters, etc.

[0116] After training is completed, the model is verified using the test set to evaluate its detection accuracy and inference speed, etc.

[0117] In order to improve the pedestrian detection accuracy when using YOLOv5 detector for pedestrian detection, a feasible strategy is to introduce attention mechanism. Compared with other attention mechanisms, the coordinate attention mechanism CA is used in the embodiment of the application, which embeds position information into channel attention, so that the lightweight network can pay attention to a larger area, and effectively avoids excessive computational overhead.

[0118] Optionally, the initial network model is obtained by improving the speed and accuracy of the YOLOv5 model, including:

[0119] The backbone network of the YOLOv5 model is replaced with a ShuffleNetv2 backbone network by modifying the network structure configuration file of YOLOv5 to obtain an improved model;

[0120] An attention mechanism is added to the improved model to obtain an initial network model.

[0121] As shown in Figure 4 , replacing CSPDarknet in the YOLOv5s backbone network with ShuffleNetv2 includes the following steps:

[0122] In the source code directory of YOLOv5, find the YAML file (such as yolov5s.yaml) that defines the network structure, i.e. the model configuration file, and replace the original backbone network part (CSPDarknet) with the newly designed ShuffleNetv2 backbone network in the file.

[0123] Inserting the coordinate attention mechanism CA in YOLOV5 includes the following steps:

[0124] In the source code directory of YOLOv5, find the YAML file (such as yolov5s.yaml) that defines the network structure, i.e. the model configuration file, and replace the original backbone network part (CSPDarknet) with the newly designed ShuffleNetv2 backbone network in the file.

[0125] modify the network structure part to ensure that the CA module can be correctly connected with the front and rear modules. This may involve adjusting the size, channel number and other parameters of the feature map to maintain the consistency of the data flow;

[0126] Using the modified model configuration file for training, during the training process, the loss function and accuracy of the model can be monitored to evaluate the convergence of the model, and the performance of the YOLOv5 model with CA introduced is evaluated on the test set.

[0127] Some embodiments of the application use a lightweight convolutional neural network ShuffleNetv2 to replace the CSPDarknet structure in the YOLOv5s backbone network, which improves the detection speed of YOLOv5s; that is, it greatly improves the speed of the detection algorithm, while adding an attention mechanism CA in the model to improve the accuracy of object detection, enhance the detection accuracy of YOLOv5s, and reduce the identity confusion problem when the person crosses the camera.

[0128] As Figure 5 To provide the result graph of the object tracking algorithm of the embodiment of the application, the pedestrian can smoothly transition from the monitoring range of the first camera to the monitoring range of the second camera, and the identity ID (object identifier) frame is always continuously tracked and the ID value remains unchanged, which fully indicates that the identity information of the pedestrian remains consistent during the cross-camera tracking process, and the cross-camera continuous seamless tracking of the pedestrian is successfully realized.

[0129] Optionally, a preset tracking algorithm is used to track the position information of the object corresponding to the object identifier under different cameras, including:

[0130] The DeepSort tracking algorithm is used to track the position information of the object corresponding to the object identifier under different cameras.

[0131] It should be noted that each implementable manner in the embodiment can be implemented alone or in any combination without conflict.

[0132] Another embodiment of the application provides a cross-camera object tracking device for executing the cross-camera object tracking method provided by the above-mentioned embodiment.

[0133] As Figure 6 As shown in the structure schematic diagram of the cross-camera object tracking device provided by the embodiment of the application. The cross-camera object tracking device includes an acquisition module 601, a fusion module 602, a determination module 603 and a tracking module 604, wherein:

[0134] The acquisition module 601 is configured to acquire original images collected by different cameras;

[0135] The fusion module 602 is configured to determine different weight information according to the distance between the object and the camera in the overlapping area of the original images collected by different cameras, and perform fusion processing on the overlapping area according to the weight information to obtain a fused image.

[0136] The determination module 603 is configured to determine the object label and the position information corresponding to the object label in the fused image according to a pre-trained recognition model, wherein the pre-trained recognition model is obtained by training a sample image data on an initial network model, and the initial network model is obtained by improving the speed and accuracy of a neural network model.

[0137] The tracking module 604 is configured to track the position information of the object corresponding to the object label under different cameras by using a preset tracking algorithm.

[0138] As to the device in the embodiment, the specific manners in which the modules perform operations have been described in detail in the embodiments of the method, and will not be described in detail here.

[0139] Some embodiments of the present application improve the continuity and accuracy of cross-camera moving person tracking by using an overlapping area fusion method based on a position weight ratio allocation strategy, fuse the information of the overlapping areas of adjacent cameras according to the real-time change of the pixel weight, realize the continuous and smooth visual transition of the moving person under the multi-camera visual angle, and improve the speed and accuracy of model recognition while reducing the workload and the problem that the object cannot be tracked in real time and accurately when moving across cameras.

[0140] Another embodiment of the present application further supplements the cross-camera object tracking device provided in the above-mentioned embodiments.

[0141] Optionally, the fusion module is configured to:

[0142] obtain the overlapping area image in the first image and the second image collected by the first camera;

[0143] determine the weight information corresponding to the position information according to the position information of the object in the overlapping area image;

[0144] determine the fused image according to the weight information, the first pixel of the overlapping area image in the first image, and the second pixel of the overlapping area image in the second image.

[0145] Some embodiments of the present application improve the continuity and accuracy of cross-camera moving person tracking by fusing the information of the overlapping areas of adjacent cameras according to the real-time changes of pixel weights based on a position weight ratio allocation strategy, and realize continuous and smooth visual transition of the moving person under multi-camera visual angle.

[0146] Optionally, the fusion module is configured to:

[0147] determine the distance of the object to the preset edge based on the overlapping area;

[0148] determine the weight information according to the distance of the object to the preset edge and the preset change rate.

[0149] Some embodiments of the present application weaken the camera image pixels with small proportion on the pixel points after fusion processing, reduce the appearance of redundant bounding boxes, and ensure that the fused image can present a smooth transition change.

[0150] Optionally, the device further comprises a training module configured to:

[0151] obtain sample image data, the sample image data being original images collected by different cameras, the sample image data comprising a preset object identifier and first position information corresponding to the preset object identifier;

[0152] determine an overlapping area image of the original images collected by different cameras, and fuse the overlapping area image to obtain a sample fusion image;

[0153] train the initial network model according to the sample fusion image to obtain second position information of the preset object identifier, wherein the initial network model is obtained by improving the speed and accuracy of a YOLOv5 model;

[0154] compare the first position information and the second position information, and continuously update the initial network model, and in the case that the first position information and the second position information match, determine the initial network model corresponding to the second position information as the recognition model.

[0155] Optionally, the initial network model is obtained by improving the speed and accuracy of a YOLOv5 model, comprising:

[0156] by modifying the network structure configuration file of YOLOv5, replacing the backbone network of YOLOv5 model with a ShuffleNetv2 backbone network to obtain an improved model;

[0157] add an attention mechanism in the improved model to obtain the initial network model.

[0158] Some embodiments of the present application use a lightweight convolutional neural network ShuffleNetv2 to replace the CSPDarknet structure in the YOLOv5s backbone network, which can greatly improve the speed of the detection algorithm, and at the same time, increase the attention mechanism in the model to improve the accuracy of object detection.

[0159] Optionally, a tracking module is configured to:

[0160] The DeepSort tracking algorithm is used to track the position information of the object corresponding to the object identifier under different cameras.

[0161] As to the device in the present embodiment, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0162] It should be noted that each implementable manner in the present embodiment can be implemented alone or in any combination manner without conflict.

[0163] The present embodiment further provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the operations of the method corresponding to any embodiment of the cross-camera object tracking method provided by the above embodiment.

[0164] The present embodiment further provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the operations of the method corresponding to any embodiment of the cross-camera object tracking method provided by the above embodiment.

[0165] As shown in FIG. 7, Figure 7 Some embodiments of the present application provide an electronic device 700, which includes a memory 710, a processor 720, and a computer program stored in the memory 710 and executable on the processor 720, wherein the processor 720 reads the program from the memory 710 through the bus 730 and executes the program to implement the method of any embodiment included in the cross-camera object tracking method.

[0166] The processor 720 can process digital signals and can include various computing structures, such as a complex instruction set computer structure, a reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, the processor 720 can be a microprocessor.

[0167] The memory 710 can be used to store instructions executed by the processor 720 or data related to the execution of the instructions. The instructions and / or data can include code for implementing some or all of the functions of one or more modules described in the embodiments of the present application. The processor 720 of the embodiments of the present disclosure can be configured to execute the instructions in the memory 710 to implement the methods shown in the above. The memory 710 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memories well known to those skilled in the art.

[0168] The above merely provides an example of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, and thus, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0169] The above merely provides an example of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, and thus, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0170] It should be noted that, in the present document, the relationship terms such as first and second are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms “include”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement “including a…” does not exclude the presence of another identical element in the process, method, article or device including the element.

Claims

1. A cross-camera object tracking method, characterized in that, The method for tracking people in large, low-altitude indoor scenes includes: Acquire raw images captured by different cameras, which are arranged in a matrix and installed at certain intervals in a top-down orientation; In the overlapping area of ​​the original images captured by different cameras, the images from different cameras at the same location are fused pixel by pixel, and the image processing from different cameras is converted into processing of the fused single image. Different weight information is determined according to the distance between the object and the camera, and the overlapping area is fused according to the weight information to obtain the fused image. Based on a pre-trained recognition model, object identifiers and corresponding location information in the original image and the fused image are determined; wherein, the pre-trained recognition model is obtained by training an initial network model using sample image data, and the initial network model is obtained by improving the speed and accuracy of the neural network model; A preset tracking algorithm is used to track the position information of the object corresponding to the object identifier under different cameras; Wherein: in the overlapping region of the original images captured by different cameras, different weight information is determined according to the distance between the object and the camera, and the overlapping region is fused according to the weight information to obtain a fused image, including: Acquire the overlapping region image between the first image and the second image captured by the first camera; Based on the position information of the object in the overlapping region image, determine the weight information corresponding to the position information; Based on the weight information, the first pixel of the overlapping region image in the first image and the second pixel of the overlapping region image in the second image are used to determine the fused image; The step of determining weight information corresponding to the position information of the object in the overlapping region image includes: Based on the overlapping area, the distance from the object to the preset edge is determined; The weight information is determined based on the distance from the object to the preset edge and the preset rate of change.

2. The cross-camera object tracking method according to claim 1, characterized in that, The pre-trained recognition model is obtained in the following way: Acquire sample image data, which are original images captured by different cameras, and the sample image data includes a preset object identifier and a first position information corresponding to the preset object identifier; The overlapping region images of the original images captured by the different cameras are determined, and the overlapping region images are fused to obtain a sample fused image; Based on the sample fusion image, the initial network model is trained to obtain the second location information of the preset object identifier, wherein the initial network model is obtained by improving the speed and accuracy of the YOLOv5 model; The first location information and the second location information are compared, and the initial network model is continuously updated. When the first location information and the second location information match, the initial network model corresponding to the second location information is determined as the recognition model.

3. The cross-camera object tracking method according to claim 2, characterized in that, The initial network model was obtained by improving the speed and accuracy of the YOLOv5 model, including: By modifying the YOLOv5 network structure configuration file, the backbone network of the YOLOv5 model was replaced with the ShuffleNetv2 backbone network, resulting in an improved model. An attention mechanism is added to the improved model to obtain the initial network model.

4. The cross-camera object tracking method according to claim 1, characterized in that, The step of employing a preset tracking algorithm to track the position information of the object corresponding to the object identifier under different cameras includes: The DeepSort tracking algorithm is used to track the position information of the object corresponding to the object identifier under different cameras.

5. A cross-camera object tracking device, characterized in that, The device for tracking people in large, low-altitude indoor scenes includes: The acquisition module is used to acquire raw images captured by different cameras, which are arranged in a matrix and installed at certain intervals in a top-down orientation; The fusion module is used to perform pixel fusion of images from different cameras at the same location in the overlapping area of ​​the original images captured by different cameras, convert the image processing from different cameras into a single fused image, determine different weight information according to the distance between the object and the camera, and perform fusion processing on the overlapping area according to the weight information to obtain a fused image. The determination module is used to determine object identifiers and corresponding location information in the original image and the fused image based on a pre-trained recognition model; wherein the pre-trained recognition model is obtained by training an initial network model using sample image data, and the initial network model is obtained by improving the speed and accuracy of a neural network model; The tracking module is used to track the position information of the object corresponding to the object identifier under different cameras using a preset tracking algorithm; The fusion module is used for: Acquire the overlapping region image between the first image and the second image captured by the first camera; Based on the position information of the object in the overlapping region image, determine the weight information corresponding to the position information; Based on the weight information, the first pixel of the overlapping region image in the first image and the second pixel of the overlapping region image in the second image are used to determine the fused image; The step of determining weight information corresponding to the position information of the object in the overlapping region image includes: Based on the overlapping area, the distance from the object to the preset edge is determined; The weight information is determined based on the distance from the object to the preset edge and the preset rate of change.

6. The cross-camera object tracking device according to claim 5, characterized in that, The device further includes a training module, the training module being used for: Acquire sample image data, which are original images captured by different cameras, and the sample image data includes a preset object identifier and a first position information corresponding to the preset object identifier; The overlapping region images of the original images captured by the different cameras are determined, and the overlapping region images are fused to obtain a sample fused image; Based on the sample fusion image, the initial network model is trained to obtain the second location information of the preset object identifier, wherein the initial network model is obtained by improving the speed and accuracy of the YOLOv5 model; The first location information and the second location information are compared, and the initial network model is continuously updated. When the first location information and the second location information match, the initial network model corresponding to the second location information is determined as the recognition model.

7. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it can implement the cross-camera object tracking method according to any one of claims 1-4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, characterized in that, when the program is executed by a processor, it can implement the cross-camera object tracking method according to any one of claims 1-4.

Images