Multi-camera target tracking method and device, equipment and medium

Abstract

The invention provides a multi-camera target tracking method and device, equipment and a medium, relates to the technical field of security and protection monitoring, and is used for solving the problem that the target tracking effect of an existing monitoring system is relatively poor. The method comprises the following steps: acquiring image information of a target pedestrian uploaded by any pan-tilt camera; according to the image information of the target pedestrian, obtaining a pedestrian ID of the target pedestrian and a human shape feature of the target pedestrian; associating the human shape features of the target pedestrian with the pedestrian ID of the target pedestrian and then storing the human shape features and the pedestrian ID in an ReID base library; obtaining human-shaped moving tracks of a plurality of pedestrians uploaded by any panoramic camera; and associating the human shape moving track of each pedestrian with the corresponding ID. According to the invention, the wide-coverage panorama camera and the high-quality shooting pan-tilt camera are combined, and the target tracking capability of the monitoring system is improved.

Description

Technical Field

[0001] The present application relates to the field of security monitoring technology and provides a multi-camera target tracking method, device, equipment and medium. Background Art

[0002] The existing monitoring system mainly relies on traditional fixed cameras and a small number of pan-tilt cameras. Fixed cameras have limited viewing angles, while pan-tilt cameras are flexible but have limited coverage. Moreover, these cameras work independently and lack an effective coordination mechanism, and cannot fully cover the monitoring area. Therefore, the existing monitoring system has blind spots, resulting in poor target tracking effects. Summary of the invention

[0003] The present application provides a multi-camera target tracking method, device, equipment and medium for solving the problem of poor target tracking effect of existing monitoring systems.

[0004] In a first aspect, a multi-camera target tracking method is provided, which is applied to a cloud server of a monitoring system, wherein the monitoring system further includes a plurality of panoramic cameras and a pan / tilt camera bound to each panoramic camera, and the cloud server includes a ReID base database, wherein the ReID base database stores a plurality of pedestrian IDs, a human feature associated with each pedestrian ID, a collection time of each human feature, and a collection camera ID; the method includes:

[0005] Get the image information of the target pedestrian uploaded by any gimbal camera;

[0006] Acquire the pedestrian ID and the human shape features of the target pedestrian according to the image information of the target pedestrian;

[0007] After associating the human shape features of the target pedestrian with the pedestrian ID of the target pedestrian, the human shape features are saved in the ReID database;

[0008] Acquire the human-shaped movement trajectories of multiple pedestrians uploaded by any panoramic camera; wherein the human-shaped movement trajectory of any pedestrian includes the positions of any pedestrian at multiple times;

[0009] If there is a target human feature in the ReID database whose collection time is any of the multiple time moments, and the collection camera ID of the target human feature indicates a gimbal camera bound to any of the panoramic cameras, then the pedestrian ID corresponding to the target human feature is determined as the pedestrian ID of any of the pedestrians, and the pedestrian ID of any of the pedestrians is associated with the human movement trajectory of any of the pedestrians.

[0010] Optionally, the cloud server further includes a face database, wherein the face database stores a plurality of pedestrian IDs and a face image associated with each pedestrian ID; and obtaining the pedestrian ID and the human shape features of the target pedestrian according to the image information of the target pedestrian includes:

[0011] If the image information of the target pedestrian is a face image of the target pedestrian and a video clip of the target pedestrian, extracting a human shape feature of the target pedestrian from the video clip of the target pedestrian;

[0012] Calculating the similarity between the face image of the target pedestrian and the face image associated with each pedestrian ID respectively to obtain multiple similarities;

[0013] If the maximum similarity among the multiple similarities is greater than or equal to a preset threshold, the pedestrian ID corresponding to the maximum similarity is determined as the pedestrian ID of the target pedestrian.

[0014] Optionally, after respectively calculating the similarities between the face image of the target pedestrian and the face image associated with each pedestrian ID to obtain a plurality of similarities, the method further includes:

[0015] If the maximum similarity is less than the preset threshold, generating a pedestrian ID of the target pedestrian;

[0016] After associating the pedestrian ID of the target pedestrian with the face image of the target pedestrian, the pedestrian is saved in the face database.

[0017] Optionally, acquiring the pedestrian ID and the human shape features of the target pedestrian according to the image information of the target pedestrian includes:

[0018] If the image information of the target pedestrian is a full-body image of the target pedestrian, the full-body image of the target pedestrian is input into the ReID model, and the pedestrian ID of the target pedestrian and the human shape features of the target pedestrian are output; wherein the ReID model is trained based on multiple pedestrian sample images with pedestrian IDs.

[0019] Optionally, the step of inputting the full-body image of the target pedestrian into a ReID model and outputting the pedestrian ID of the target pedestrian and the human shape features of the target pedestrian includes:

[0020] The full-body image of the target pedestrian is input into the ReID model, and N pedestrian IDs and human features of the target pedestrian are outputted in order of similarity; N is a positive integer;

[0021] According to the human-shaped movement trajectories of the multiple pedestrians, determining that the pedestrian ID ranked first among the N pedestrian IDs has a space-time constraint conflict;

[0022] If the first-ranked pedestrian ID has the time-space constraint conflict, continue to judge whether the second-ranked pedestrian ID among the N pedestrian IDs has the time-space constraint conflict, until the target pedestrian ID among the N pedestrian IDs does not have the time-space constraint conflict, and determine the target pedestrian ID as the pedestrian ID of the target pedestrian.

[0023] Optionally, the space-time constraint conflict is that the same pedestrian ID appears at different locations within a preset time period or the speed of the same pedestrian ID appearing at different locations is greater than a preset speed.

[0024] Optionally, after associating the human shape feature of the target pedestrian with the pedestrian ID of the target pedestrian and saving them in the ReID base database, the method further includes:

[0025] Determine a plurality of human features associated with the pedestrian ID of the target pedestrian stored in the ReID base database;

[0026] If the number of the plurality of human-shaped features is greater than or equal to a preset number, the human-shaped feature with the earliest collection time is deleted from the plurality of human-shaped features.

[0027] In a second aspect, a multi-camera target tracking device is provided, wherein the device is arranged in a cloud server of a monitoring system, wherein the monitoring system further comprises a plurality of panoramic cameras and a pan / tilt camera bound to each panoramic camera, wherein the cloud server comprises a ReID base database, wherein the ReID base database stores a plurality of pedestrian IDs, a human shape feature associated with each pedestrian ID, a collection time of each human shape feature and a collection camera ID; the device comprises:

[0028] An acquisition module, used to acquire the image information of the target pedestrian uploaded by the PTZ camera;

[0029] The acquisition module is also used to acquire the pedestrian ID of the target pedestrian and the human shape features of the target pedestrian according to the image information;

[0030] A saving module, used to associate the human shape features of the target pedestrian with the pedestrian ID of the target pedestrian and save them in the ReID base database;

[0031] The acquisition module is also used to acquire the human-shaped movement trajectories of multiple pedestrians uploaded by the panoramic camera; wherein the human-shaped movement trajectory of any pedestrian includes the position of any pedestrian at multiple times;

[0032] An association module is used to determine the pedestrian ID corresponding to the target human feature as the pedestrian ID of any pedestrian, and associate the pedestrian ID of any pedestrian with the human movement trajectory of any pedestrian if there is a target human feature in the ReID base database whose collection time is any of the multiple time moments, and the collection camera ID of the target human feature indicates a pan-tilt camera bound to any panoramic camera.

[0033] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program file, and the processor executes the computer program file to implement the multi-camera target tracking method described in the first aspect.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program file stored thereon, and a processor executes the computer program file to implement the multi-camera target tracking method described in the first aspect.

[0035] Compared with the prior art, the beneficial effects of this application are as follows:

[0036] The present application provides a multi-camera target tracking method, which is applied to a cloud server of a monitoring system, wherein the monitoring system also includes multiple panoramic cameras and a pan-tilt camera bound to each panoramic camera, and the cloud server includes a ReID base database, which stores the multiple pedestrian IDs, human features associated with each pedestrian ID, and the collection time and camera ID of each human feature; the method includes: obtaining image information of a target pedestrian uploaded by any pan-tilt camera; obtaining a pedestrian ID and a human feature of the target pedestrian based on the image information of the target pedestrian; associating the human feature of the target pedestrian with the pedestrian ID of the target pedestrian and saving them in the ReID base database; obtaining human movement trajectories of multiple pedestrians uploaded by any panoramic camera; the human movement trajectory of any pedestrian includes the position of any pedestrian at multiple times; if there is a target human feature in the ReID base database whose collection time is any time among the multiple times, and the collection camera ID of the target human feature indicates a pan-tilt camera bound to any panoramic camera, then the pedestrian ID corresponding to the target human feature is determined as the pedestrian ID of any pedestrian, and the pedestrian ID of any pedestrian is associated with the human movement trajectory of any pedestrian. This application associates the pedestrian ID with the human movement trajectory by combining a wide-coverage panoramic camera with a high-quality gimbal camera. That is, the human activity trajectory uploaded by the panoramic camera and the high-quality image taken by the gimbal camera are associated. In this way, the identity of the pedestrian in the human movement trajectory can be determined, and target recognition and tracking within a large range can be achieved, thereby improving the target recognition and tracking capabilities of the monitoring system and reducing the blind spots of traditional monitoring systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of a flow chart of a multi-camera target tracking method provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of the structure of a multi-camera target tracking device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0042] In order to solve the problem of poor target tracking effect of existing monitoring systems, the present application embodiment provides a multi-camera target tracking method, which is applied to the cloud server of the monitoring system. Figure 1 , is a schematic diagram of an application scenario provided in an embodiment of the present application, or can be understood as a structural diagram of a monitoring system.

[0043] The monitoring system includes a cloud server, multiple panoramic cameras and multiple PTZ cameras. Each panoramic camera is bound to a PTZ camera, and the PTZ camera can be set above the panoramic camera. Each panoramic camera is connected to the cloud server for communication, and each PTZ camera is also connected to the cloud server for communication. The functions of each device are introduced below.

[0044] Among them, a panoramic camera is a camera device that can capture a 360-degree field of view. It can simultaneously capture scenes above, below, in front, behind, and around. It is usually used to create panoramic photos or videos, which can present a very wide field of view. A panoramic camera can detect human figures in real time and upload their movement trajectories to a cloud server.

[0045] A PTZ camera is usually a camera device mounted on a rotatable or tiltable bracket that can change direction and angle through remote or automatic control. A PTZ camera can capture faces and shoot videos, and upload them to a cloud server.

[0046] Cloud servers refer to server resources deployed on cloud computing platforms, which can process and store data. Cloud servers include ReID base database and face database. ReID base database stores multiple pedestrian IDs, human features associated with each pedestrian ID, the collection time of each human feature and the collection camera ID. The face database stores multiple pedestrian IDs and face images associated with each pedestrian ID. Pedestrian ID is a symbol, code or number used to uniquely identify each pedestrian. The collection time refers to the time when the image of each human feature is collected. The collection camera ID refers to the camera ID of the image of each human feature.

[0047] It should be noted that Figure 1 Two panoramic cameras and two gimbal cameras are taken as an example. In fact, there is no limit to the number of panoramic cameras and gimbal cameras.

[0048] As described above, the application scenarios are Figure 1 The monitoring system Figure 2 The multi-camera target tracking method shown in the figure is introduced.

[0049] S201, obtaining image information of a target pedestrian uploaded by any pan-tilt camera.

[0050] The target pedestrian refers to any pedestrian who enters the monitoring range of the PTZ camera. The image information of the target pedestrian can be a face image of the target pedestrian and a video clip of the target pedestrian. The video clip refers to a video recorded within a period of time before and after the capture time. The capture time refers to the time when the face image of the target pedestrian is captured. The image information of the target pedestrian can also be a full-body image of the target pedestrian.

[0051] In the specific implementation process, when the target pedestrian enters the monitoring range of the PTZ camera, the PTZ camera will identify and locate the face of the target pedestrian in the picture through the face detection module. Once the face of the target pedestrian is detected, the target face image of the target pedestrian is captured, and at the same time, a video clip of the target pedestrian is recorded for a period of time before and after the capture moment. If the PTZ camera has not detected the face of the target pedestrian, a full-body image of the target pedestrian is captured at a preset interval. Furthermore, after obtaining the image information of the target pedestrian, the PTZ camera uploads it to the cloud server.

[0052] S202: Obtain the pedestrian ID and human shape features of the target pedestrian according to the image information of the target pedestrian.

[0053] In a possible embodiment, the cloud server also includes a face library, which stores multiple pedestrian IDs and facial images associated with each pedestrian ID. If the image information of the target pedestrian is the facial image of the target pedestrian and the video clip of the target pedestrian, the human features of the target pedestrian are extracted from the video clip of the target pedestrian; the similarity between the facial image of the target pedestrian and the facial image associated with each pedestrian ID is calculated respectively to obtain multiple similarities; if the maximum similarity among the multiple similarities is greater than or equal to a preset threshold, the pedestrian ID corresponding to the maximum similarity is determined as the pedestrian ID of the target pedestrian.

[0054] In the specific implementation process, the human features of the target pedestrian can be extracted from the video clips of the target pedestrian through the ReID model. The human features are a set of features used to describe and identify the physical features of the pedestrian. The human features include appearance features, clothing features, posture features, and behavioral features. Appearance features can be height, body shape, skin color, hair color, eye color, etc. Clothing features can be the color, texture, style of clothing, etc. Posture features can be walking posture, standing posture, etc. Behavior features can be the pedestrian's gait, movement, etc.

[0055] Furthermore, facial image features of the target pedestrian and facial image features associated with each pedestrian ID in the face database are extracted respectively. Facial image features include key points, contours, color histograms and other information of the face. The similarity between the facial image features of the target pedestrian and the facial image features associated with each pedestrian ID is calculated. The similarity can be cosine similarity, Euclidean distance, etc., to obtain multiple similarities. Then, the maximum similarity among the multiple similarities is determined. If the maximum similarity is greater than or equal to a preset threshold, it means that the facial image corresponding to the maximum similarity and the facial image of the target pedestrian are likely to belong to the same face, and the pedestrian ID corresponding to the maximum similarity is determined as the pedestrian ID of the target pedestrian.

[0056] In the embodiment of the present application, considering that there may be multiple faces in the face library that are similar to the face of the target pedestrian, selecting the maximum similarity can reduce the possibility of misjudgment. By setting a preset threshold, the strictness of face matching can be flexibly controlled. For example, the preset threshold can be adjusted according to security requirements or actual application scenarios to balance the relationship between accuracy and fault tolerance.

[0057] In a possible embodiment, if the maximum similarity is less than a preset threshold, a pedestrian ID of the target pedestrian is generated; after the pedestrian ID of the target pedestrian is associated with the face image of the target pedestrian, the pedestrian ID is saved in a face database.

[0058] In the specific implementation process, if the maximum similarity is less than the preset threshold, it means that the face image corresponding to the maximum similarity and the face image of the target pedestrian are unlikely to belong to the same face, and a new pedestrian ID is generated as the pedestrian ID of the target pedestrian. There are many ways to generate pedestrian IDs. You can directly use the auto-increment field of the database to generate the pedestrian ID of the target pedestrian. This method can simplify the process of pedestrian ID generation and does not require additional logic to generate a unique pedestrian ID, reducing the complexity of development and the possibility of errors. You can also use the current timestamp as the pedestrian ID of the target pedestrian, accurate to milliseconds or higher, to ensure the uniqueness of the pedestrian ID generated at the same time. This method records the storage time of each face while storing the pedestrian ID.

[0059] In a possible embodiment, if the maximum similarity is greater than or equal to a preset threshold, the pedestrian ID corresponding to the maximum similarity is determined as the pedestrian ID of the target pedestrian. If the maximum similarity is greater than or equal to the preset similarity, the face image corresponding to the maximum similarity in the face library is replaced with the face image of the target pedestrian.

[0060] In a specific implementation process, the preset similarity is greater than a preset threshold value, the preset threshold value is, for example, 80%, and the preset similarity is, for example, 90%. If the maximum similarity among multiple similarities is greater than or equal to the preset similarity, the face image corresponding to the maximum similarity is deleted from the face database, the face image of the target pedestrian is associated with the pedestrian ID corresponding to the maximum similarity, and saved in the face database.

[0061] In an embodiment of the present application, when the face similarity is high enough, the face images in the face library are updated in a timely manner to maintain the timeliness and accuracy of the face library and improve the accuracy of subsequent face image matching.

[0062] In a possible embodiment, if the image information of the target pedestrian is a full-body image of the target pedestrian, the full-body image of the target pedestrian is input into the ReID model, and the pedestrian ID of the target pedestrian and the human shape features of the target pedestrian are output.

[0063] In the specific implementation process, the ReID model is trained based on multiple sample images of pedestrians with pedestrian IDs; the sample pedestrian images include multiple full-body images of the same pedestrian captured from different angles of the same gimbal camera. The ReID model can be used to extract the human features of the target pedestrian from the full-body image of the target pedestrian, and the similarity between the human features of the target pedestrian and the pedestrian features associated with each pedestrian ID in the ReID database is calculated respectively to obtain multiple similarities, and the pedestrian ID corresponding to the maximum similarity is output, that is, the pedestrian ID of the target pedestrian.

[0064] This involves the ReID model. The training process of the ReID model is introduced below.

[0065] The pedestrian sample image is used as the input of the initial model, and the error between the pedestrian ID output by the initial model and the pedestrian ID of each pedestrian sample image is used as the feedback data of the initial model. The model parameter values ​​are continuously adjusted through the feedback data. After training with a large number of pedestrian sample images, the model parameters of the initial model are continuously updated until the preset conditions are met, such as: the error is within the preset range or the maximum number of iterations is reached, thereby obtaining a ReID model.

[0066] In an embodiment of the present application, by comparing the full-body image of the target pedestrian with the data in the ReID database, the identity of the target pedestrian can be quickly identified, thereby improving the efficiency, accuracy and practicality of the security monitoring system.

[0067] In a possible embodiment, a full-body image of a target pedestrian is input into a ReID model, and a pedestrian ID and a human shape feature of the target pedestrian are output, including:

[0068] The full-body image of the target pedestrian is input into the ReID model, and N pedestrian IDs and human features of the target pedestrian sorted by similarity are output; N is a positive integer; based on the human movement trajectories of multiple pedestrians, it is determined whether the first-ranked pedestrian ID among the N pedestrian IDs has a space-time constraint conflict; if the first-ranked pedestrian ID has a space-time constraint conflict, it is continued to be determined whether the second-ranked pedestrian ID among the N pedestrian IDs has a space-time constraint conflict, until the target pedestrian ID among the N pedestrian IDs has no space-time constraint conflict, and the target pedestrian ID is determined as the pedestrian ID of the target pedestrian.

[0069] In the specific implementation process, after obtaining the similarities of multiple pedestrians through the ReID model, the pedestrian ID corresponding to the maximum similarity is not directly output. The multiple pedestrian IDs in the ReID base database are sorted in order of similarity from large to small, and the sorted N pedestrian IDs and the human features of the target pedestrian are output. N is a positive integer, and the value of N can be preset, for example, N = 5. According to the human movement trajectories of multiple pedestrians, it is judged in turn whether each pedestrian ID in the N pedestrian IDs has a time-space constraint conflict. If the first-ranked pedestrian ID does not have a time-space constraint conflict, the first-ranked pedestrian ID is determined as the pedestrian ID of the target pedestrian. If the first-ranked pedestrian ID has a time-space constraint conflict, it is continued to be judged whether the second-ranked pedestrian ID has a time-space constraint conflict, and so on. If each pedestrian ID in the N pedestrian IDs has a time-space constraint conflict, the pedestrian features of the target pedestrian are deleted.

[0070] Among them, the space-time constraint conflict is that the same pedestrian ID appears at different locations within a preset time period or the speed of the same pedestrian ID appearing at different locations is greater than the preset speed.

[0071] For example, the preset time period is 2 seconds and the preset speed is 10m / s. If a pedestrian ID appears at two locations within 2 seconds, or the speed of the pedestrian ID at the two locations is greater than 10m / s, it means that the pedestrian ID has a space-time constraint conflict. If a pedestrian ID does not appear at two locations within 2 seconds, and the speed of the pedestrian ID at the two locations is less than 10m / s, it means that the pedestrian ID does not have a space-time constraint conflict.

[0072] In an embodiment of the present application, time and space information are taken into consideration, and the N pedestrian IDs sorted by similarity are further screened through a time-space constraint algorithm, thereby improving the accuracy of the identity recognition result, which helps to reduce misidentification.

[0073] S203: After associating the human shape features of the target pedestrian with the pedestrian ID of the target pedestrian, save them in the ReID database.

[0074] During the specific implementation process, after obtaining the human features of the target pedestrian and the pedestrian ID of the target pedestrian, the human features of the target pedestrian are associated with the pedestrian ID of the target pedestrian. At the same time, the collection time of the graphic information is used as the collection time of the human features of the target pedestrian. The identification of any pan-tilt camera is used as the camera ID for collecting the human features of the target pedestrian and saved in the ReID database.

[0075] In a possible embodiment, after associating the human shape features of the target pedestrian with the pedestrian ID of the target pedestrian and saving them in the ReID base database, multiple human shape features associated with the pedestrian ID of the target pedestrian stored in the ReID base database can also be determined; if the number of the multiple human shape features is greater than or equal to a preset number, the human shape features with the earliest collection time are deleted from the multiple human shape features.

[0076] In the specific implementation process, considering that a large amount of human feature data may accumulate in the ReID database over time, deleting the earliest human feature at the time of collection can save storage space, ensure that the data in the ReID database is always updated, avoid outdated information affecting the accuracy of human feature matching, and reduce the computational burden during human feature matching, thereby improving the speed and efficiency of human feature matching.

[0077] S204: Obtain the moving trajectories of multiple pedestrians uploaded by any panoramic camera.

[0078] In the specific implementation process, the human-shaped movement trajectory of any pedestrian includes the position of any pedestrian at multiple times. The panoramic camera can perform human-shaped detection on each frame in the panoramic picture. Once a pedestrian is detected, the panoramic camera can track the position of the pedestrian at multiple times based on the pixel changes between consecutive frames, thereby obtaining the movement trajectory of the pedestrian, and then converting the various positions of the pedestrian into a polar coordinate system, thereby obtaining the human-shaped movement trajectory of the pedestrian in polar coordinates and uploading it to the cloud server. Polar coordinates usually use angles and distances to represent positions, which helps to more intuitively understand the movement path of a human figure.

[0079] S205. If there is a target human shape feature in the ReID database whose collection time is any of the multiple time moments, and the collection camera ID of the target human shape feature indicates a pan-tilt camera bound to any panoramic camera, then the pedestrian ID corresponding to the target human shape feature is determined as the pedestrian ID of any pedestrian, and the pedestrian ID of any pedestrian is associated with the human shape movement trajectory of any pedestrian.

[0080] In the specific implementation process, after obtaining the position of any pedestrian at multiple times, the target human shape features collected at any of the multiple times in the ReID database are queried. Since there are multiple gimbal cameras, it is possible that multiple gimbal cameras have collected human shape features of different pedestrians at the same time. Therefore, after determining that the acquisition camera ID of the target human shape features is indicated as the gimbal camera bound to any panoramic camera, the pedestrian ID corresponding to the target human shape features is determined as the pedestrian ID of any pedestrian, and the pedestrian ID of any pedestrian is associated with the human shape movement trajectory of any pedestrian.

[0081] In a possible embodiment, after the pedestrian ID corresponding to the target human feature is determined as the pedestrian ID of any pedestrian and the pedestrian ID of any pedestrian is associated with the human movement trajectory of any pedestrian, the earliest time and the latest time when any pedestrian ID appears at different locations can be counted to generate an activity event.

[0082] In an embodiment of the present application, by counting the earliest time and the latest time that each pedestrian ID appears at different locations, the behavior pattern and activity range of each pedestrian can be analyzed, which helps to identify each person's regular activity area, activity time period, and possible abnormal behavior. Based on the activity events of each pedestrian ID, some abnormal behaviors can be detected, such as staying in a certain location for a long time, appearing in a specific area during an unconventional time period, etc., which helps to promptly discover and deal with potential safety risks and abnormal events. Understanding the activity patterns and habits of each pedestrian ID can help optimize resource allocation, such as: strengthening personnel patrols during peak hours, adjusting equipment operating hours, etc., thereby improving resource utilization efficiency and service quality. It can also provide personalized services and support, such as: recommending activities, products or services that are suitable for their activity range and time. According to the statistical results of activity events, the scene layout and design can be optimized to better meet people's activity needs and behavioral habits, and enhance the attractiveness and comfort of the venue.

[0083] To summarize, the embodiments of the present application provide a multi-camera target tracking method, which achieves close spatial connection by combining a panoramic camera with wide coverage and almost no blind spots and a pan-tilt camera with high-quality shooting effects, thereby improving the target recognition and tracking capabilities of the monitoring system, reducing the blind spots of traditional monitoring systems, reducing security costs, and improving monitoring efficiency and accuracy.

[0084] Based on the same inventive concept, Figure 3 As shown, the embodiment of the present application also provides a multi-camera target tracking device; the device is arranged in a cloud server of a monitoring system, the monitoring system further includes a plurality of panoramic cameras, a pan / tilt camera bound to each panoramic camera, the cloud server includes a ReID base database, the ReID base database stores a plurality of pedestrian IDs, a humanoid feature associated with each pedestrian ID, a collection time of each humanoid feature and a collection camera ID; the device includes:

[0085] The acquisition module is used to obtain the image information of the target pedestrian uploaded by the PTZ camera;

[0086] The acquisition module is also used to acquire the pedestrian ID and the human shape features of the target pedestrian according to the image information;

[0087] A saving module is used to associate the human features of the target pedestrian with the pedestrian ID of the target pedestrian and save them to the ReID base database;

[0088] The acquisition module is also used to acquire the human-shaped movement trajectories of multiple pedestrians uploaded by the panoramic camera; wherein the human-shaped movement trajectory of any pedestrian includes the position of any pedestrian at multiple times;

[0089] An association module is used to determine the pedestrian ID corresponding to the target human feature as the pedestrian ID of any pedestrian if there is a target human feature in the ReID base database whose collection time is any of multiple time moments, and the collection camera ID of the target human feature indicates a pan-tilt camera bound to any panoramic camera, and associate the pedestrian ID of any pedestrian with the human movement trajectory of any pedestrian.

[0090] It should be noted that each module in the multi-camera target tracking device in this embodiment is one-to-one associated with each step in the multi-camera target tracking method in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned multi-camera target tracking method, which will not be repeated here.

[0091] In addition, in one embodiment, the present application also provides a computer device, which includes a processor, a memory, and a computer program file stored in the memory, and the computer program file implements the aforementioned multi-camera target tracking method when executed by the processor.

[0092] In addition, in one embodiment, the present application also provides a computer storage medium, on which a computer program file is stored, and when the computer program file is executed by a processor, the aforementioned multi-camera target tracking method is implemented.

[0093] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0094] In some embodiments, executable instructions may be in the form of program files, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone program files or as modules, components, subroutines or other units suitable for use in a computing environment.

[0095] As an example, executable instructions may, but are not necessarily associated with a file in a file system, may be stored as part of a file that stores other program files or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program file in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutine files, or code portions).

[0096] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0097] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0098] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0100] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A multi-camera target tracking method, characterized in that: In a cloud server applied to a monitoring system, the monitoring system further includes a plurality of panoramic cameras and a pan / tilt camera bound to each panoramic camera, the cloud server includes a ReID base database, and the ReID base database stores a plurality of pedestrian IDs, human features associated with each pedestrian ID, the collection time of each human feature and the collection camera ID; the method includes: Get the image information of the target pedestrian uploaded by any gimbal camera; Acquire the pedestrian ID and the human shape features of the target pedestrian according to the image information of the target pedestrian; After associating the human shape features of the target pedestrian with the pedestrian ID of the target pedestrian, the human shape features are saved in the ReID database; Acquire the human-shaped movement trajectories of multiple pedestrians uploaded by any panoramic camera; wherein the human-shaped movement trajectory of any pedestrian includes the positions of any pedestrian at multiple times; If there is a target human feature in the ReID database whose collection time is any of the multiple time moments, and the collection camera ID of the target human feature indicates a gimbal camera bound to any of the panoramic cameras, then the pedestrian ID corresponding to the target human feature is determined as the pedestrian ID of any of the pedestrians, and the pedestrian ID of any of the pedestrians is associated with the human movement trajectory of any of the pedestrians.

2. The target tracking method of multiple cameras as claimed in claim 1, characterized in that: The cloud server also includes a face database, which stores multiple pedestrian IDs and face images associated with each pedestrian ID; the step of obtaining the pedestrian ID and the human shape features of the target pedestrian according to the image information of the target pedestrian includes: If the image information of the target pedestrian is a face image of the target pedestrian and a video clip of the target pedestrian, extracting a human shape feature of the target pedestrian from the video clip of the target pedestrian; Calculating the similarity between the face image of the target pedestrian and the face image associated with each pedestrian ID respectively to obtain multiple similarities; If the maximum similarity among the multiple similarities is greater than or equal to a preset threshold, the pedestrian ID corresponding to the maximum similarity is determined as the pedestrian ID of the target pedestrian.

3. The target tracking method of multiple cameras as claimed in claim 2, characterized in that: After respectively calculating the similarities between the face image of the target pedestrian and the face image associated with each pedestrian ID to obtain a plurality of similarities, the method further includes: If the maximum similarity is less than the preset threshold, generating a pedestrian ID of the target pedestrian; After associating the pedestrian ID of the target pedestrian with the face image of the target pedestrian, the pedestrian is saved in the face database.

4. The target tracking method of multiple cameras as claimed in claim 1, characterized in that: The step of acquiring the pedestrian ID and the human shape features of the target pedestrian according to the image information of the target pedestrian includes: If the image information of the target pedestrian is a full-body image of the target pedestrian, the full-body image of the target pedestrian is input into the ReID model, and the pedestrian ID of the target pedestrian and the human shape features of the target pedestrian are output; wherein the ReID model is trained based on multiple pedestrian sample images with pedestrian IDs.

5. The target tracking method using multiple cameras as claimed in claim 4, characterized in that: The step of inputting the full-body image of the target pedestrian into the ReID model and outputting the pedestrian ID of the target pedestrian and the human shape features of the target pedestrian includes: The full-body image of the target pedestrian is input into the ReID model, and N pedestrian IDs and human features of the target pedestrian are outputted in order of similarity; N is a positive integer; According to the human-shaped movement trajectories of the multiple pedestrians, determining that the pedestrian ID ranked first among the N pedestrian IDs has a space-time constraint conflict; If the first-ranked pedestrian ID has the time-space constraint conflict, continue to judge whether the second-ranked pedestrian ID among the N pedestrian IDs has the time-space constraint conflict, until the target pedestrian ID among the N pedestrian IDs does not have the time-space constraint conflict, and determine the target pedestrian ID as the pedestrian ID of the target pedestrian.

6. The target tracking method using multiple cameras as claimed in claim 5, characterized in that: The space-time constraint conflict is that the same pedestrian ID appears at different locations within a preset time period or the speed of the same pedestrian ID appearing at different locations is greater than a preset speed.

7. The target tracking method using multiple cameras as claimed in claim 1, characterized in that: After associating the human shape feature of the target pedestrian with the pedestrian ID of the target pedestrian and saving them in the ReID base database, the method further includes: Determine a plurality of human features associated with the pedestrian ID of the target pedestrian stored in the ReID base database; If the number of the plurality of human-shaped features is greater than or equal to a preset number, the human-shaped feature with the earliest collection time is deleted from the plurality of human-shaped features.

8. A multi-camera target tracking device, characterized in that: The device is arranged in a cloud server of a monitoring system, the monitoring system further comprising a plurality of panoramic cameras and a pan / tilt camera bound to each panoramic camera, the cloud server comprising a ReID base database, the ReID base database storing a plurality of pedestrian IDs, human features associated with each pedestrian ID, the collection time of each human feature and the collection camera ID; the device comprises: An acquisition module, used to acquire the image information of the target pedestrian uploaded by the PTZ camera; The acquisition module is also used to acquire the pedestrian ID of the target pedestrian and the human shape features of the target pedestrian according to the image information; A saving module, used to associate the human shape features of the target pedestrian with the pedestrian ID of the target pedestrian and save them in the ReID base database; The acquisition module is also used to acquire the human-shaped movement trajectories of multiple pedestrians uploaded by the panoramic camera; wherein the human-shaped movement trajectory of any pedestrian includes the position of any pedestrian at multiple times; An association module is used to determine the pedestrian ID corresponding to the target human feature as the pedestrian ID of any pedestrian, and associate the pedestrian ID of any pedestrian with the human movement trajectory of any pedestrian if there is a target human feature in the ReID base database whose collection time is any of the multiple time moments, and the collection camera ID of the target human feature indicates a pan-tilt camera bound to any panoramic camera.

9. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores a computer program file, and the processor executes the computer program file to implement the multi-camera target tracking method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program file, and the processor executes the computer program file to implement the multi-camera target tracking method according to any one of claims 1 to 7.

Images