Cross-camera tracking method, device, electronic device and readable storage medium

Abstract

The present invention provides a cross-lens tracking method, device, electronic device and readable storage medium, the method comprising: if all cameras in the current camera monitoring circle have not captured the image of the target object within the theoretical appearance time period and the preset buffer time period, the current camera monitoring circle is expanded; based on all cameras in the expanded camera monitoring circle, a video search mechanism is started to determine the target recording time of the searched target object; if the target recording time is greater than the last appearance time of the target object in the current camera monitoring circle, the current camera monitoring circle is updated based on the camera when the target object appears at the target recording time; based on the updated current camera monitoring circle, the target object is tracked, and once the present invention determines that the current camera monitoring circle is invalid through timeout detection, the target object is re-tracked based on the video search mechanism, and the current camera monitoring circle is updated based on the camera that tracks the target object, thereby improving the target tracking accuracy.

Description

Technical Field

[0001] The present invention relates to the field of cross-camera tracking technology, and in particular to a cross-camera tracking method, device, electronic device and readable storage medium. Background Art

[0002] Cross-camera tracking technology refers to a method of effectively identifying and retrieving pedestrians or objects across cameras or scenes. That is, pedestrians or objects are re-identified or recognized under different camera lenses to achieve the effect of tracking targets under different camera lenses. With the near-full coverage of camera equipment, the improvement of target recognition and analysis technology efficiency, and the development and application of GIS technology, cross-camera tracking technology has been widely used in the security industry.

[0003] At present, there are many similar cross-camera tracking algorithm applications. Usually, it is necessary to first set up a control circle based on path planning, identify the target in the images captured by each device in the control circle, and compare the features with the tracked object. Finally, the comparison results are returned to the user and the historical travel trajectory corresponding to the target is generated.

[0004] However, constrained by the uncertainties of the actual on-site environment, when the equipment in the prevention and control circle fails to capture the tracked object or causes the target to be lost, the existing technology cannot self-correct the current prevention and control circle. Moreover, path planning is used to predict the movement trajectory of the tracked object, and target identification, comparison, and tracking tasks are issued to the camera on the trajectory. In scenarios such as shopping malls, stations, and mountain cities where the equipment relationship cannot be reflected by longitude and latitude, path planning may fail, and the equipment in the prevention and control circle cannot be delineated, so the equipment cannot be analyzed and continued to be tracked, and high-precision tracking cannot be achieved. Summary of the invention

[0005] One of the purposes of the present invention is to provide a cross-camera tracking method, device, electronic device and readable storage medium, which are used to adaptively correct the camera monitoring circle and improve the tracking accuracy. The embodiments of the present invention can be implemented as follows:

[0006] In a first aspect, the present invention provides a cross-camera tracking method, the method comprising: if all cameras within the current camera monitoring circle have not captured an image of the target object within a theoretical appearance time period and a preset buffer time period, the current camera monitoring circle is expanded; based on all cameras within the expanded camera monitoring circle, a video search mechanism is started to determine a target video recording time of the searched target object; if the target video recording time is greater than the last appearance time of the target object in the current camera monitoring circle, the current camera monitoring circle is updated based on the camera at which the target object appears at the target video recording time; and the target object is tracked based on the updated current camera monitoring circle.

[0007] In a second aspect, the present invention provides a cross-camera tracking device, comprising: a determination module, for expanding the current camera monitoring circle if all cameras in the current camera monitoring circle have not captured an image of the target object within a theoretical appearance time period and a preset buffer time period; a search module, for starting a video search mechanism based on all cameras in the expanded camera monitoring circle to determine a target video recording time of the searched target object; an update module, for updating the current camera monitoring circle based on the camera at which the target object appears at the target video recording time if the target video recording time is greater than the last appearance time of the target object in the current camera monitoring circle; and a tracking module, for tracking the target object based on the updated current camera monitoring circle.

[0008] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the method described in the first aspect.

[0009] In a fourth aspect, the present invention provides a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0010] The cross-lens tracking method, device, electronic device and readable storage medium provided by the present invention include: if all cameras in the current camera monitoring circle do not capture the image of the target object within the theoretical appearance time period and the preset buffer time period, the current camera monitoring circle is expanded; based on all cameras in the expanded camera monitoring circle, a video search mechanism is started to determine the target recording time of the searched target object; if the target recording time is greater than the last appearance time of the target object in the current camera monitoring circle, the current camera monitoring circle is updated based on the camera when the target object appears at the target recording time; based on the updated current camera monitoring circle, the target object is tracked. The present invention can determine whether the target object has been lost through timeout detection, thereby determining whether the current camera monitoring circle is invalid. After the current camera monitoring circle is invalid, the target object can be re-tracked based on the video search mechanism. Once the target reappears at a time greater than the last appearance time of the target object in the current prevention and control circle, the current prevention and control circle can be updated based on the camera that tracks the target object. The entire process can correct the camera monitoring circle and reduce the probability of target loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A schematic diagram of a scenario provided by an embodiment of the present invention;

[0013] Figure 2 A structural block diagram of an electronic device provided by an embodiment of the present invention;

[0014] Figure 3 A schematic flow chart of a cross-camera tracking method provided by an embodiment of the present invention;

[0015] Figure 4 A schematic diagram of a custom camera set provided by an embodiment of the present invention;

[0016] Figure 5 A schematic diagram of generating a custom camera set by selecting resource numbers and clicking on a map provided in an embodiment of the present invention;

[0017] Figure 6 An example diagram of multi-image feature fusion provided by an embodiment of the present invention;

[0018] Figure 7 A schematic flow chart of another cross-camera tracking method provided by an embodiment of the present invention;

[0019] Figure 8 An example diagram of an automatic robust camera monitoring circle provided by an embodiment of the present invention;

[0020] Fig. 9 A schematic diagram of a process flow of an automatic robust camera monitoring circle provided by an embodiment of the present invention;

[0021] Fig.10 A functional module diagram of the cross-mirror tracking device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0025] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0026] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0027] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0028] Please refer to Figure 1 , Figure 1 A scenario diagram is provided for an embodiment of the present invention, which includes a terminal device 101, multiple cameras (for example, camera 102-1 to camera 102-n) and a server 103, and the terminal device 101, cameras 102-1 to camera 102-n and server 103 are communicatively connected via a network 104.

[0029] Among them, the cameras 102-1 to 102-n are used to collect images or videos within the monitoring range, and send the collected images or videos to the server 103 through the network, and the server 103 stores the images in a database, or performs corresponding business processing on the images or videos. In addition, the terminal device 101 can send a business request to the server 103, and the server 103 can perform corresponding business processing on the images or videos after receiving the business request, and feed back the processing results to the terminal device 101 for display.

[0030] Taking cross-camera tracking as an example, cross-camera tracking uses target recognition technology to perform real-time analysis of the camera, extract snapshot images with target tracking objects, and push the snapshot images to users. It uses feature recognition and comparison technology to perform real-time screening of target tracking objects in the tracking process, and then combines GIS path planning technology to generate a dynamic prevention and control circle (i.e., the camera monitoring circle provided by an embodiment of the present invention) to predict the movement direction of the target tracking object and continuously track its trajectory. The target tracking object can be but is not limited to people, vehicles, animals, etc., which are not limited here.

[0031] For example, Figure 1 As an example, the target tracking object is a person. After the initial focus device sends the captured human image to the server 103, the server 103 feeds back the captured image of the person to the terminal device 101. The terminal device 101 displays the captured image on the screen. The user can select at least one human image. After the terminal device 101 feeds back the selected human image to the server 103, the server 103 can start cross-lens tracking, that is, generate a control circle according to the path planning, that is, Figure 1 Cameras 102-1 to 102-n belong to the same control circle, and then server 103 can send tracking tasks to cameras 102-1 to 102-n. After cameras 102-1 to 102-n receive the tracking tasks, they can feed back the collected images to server 103. After receiving the images, server 103 performs target recognition on the images, and matches the recognized human features with the human features of the person for similarity, and feeds back the comparison results to terminal device 101, and generates the trajectory of the person.

[0032] However, constrained by the uncertainty of the actual on-site environment, existing cross-camera tracking generally has the following shortcomings:

[0033] 1. Tracking only the human body can easily lead to misjudgment or loss of the target. For example, unclear human features can easily lead to misjudgment when encountering passers-by with similar body shape and clothing, or loss of the suspect who changes clothes; or, the target recognition task or the suspect in the camera is not reported due to obstructions, light, etc., resulting in the loss of the suspect.

[0034] 2. If the camera fails to capture the suspect or causes the target to be lost, the current control circle will no longer be able to capture the target and cannot perform self-correction.

[0035] 3. In indoor environments, due to inaccurate positioning or the longitude and latitude may be in the same vertical space, path planning fails, and the device in the control circle cannot be delineated, so the device cannot be analyzed and tracked. For example, path planning is used to predict the suspect's movement trajectory and generate a control circle for the camera on the trajectory. If the device relationship cannot be reflected in shopping malls, stations, mountain cities, etc., the dynamic control circle generated based on path planning will fail and tracking cannot be continued.

[0036] In order to solve the above technical problems, an embodiment of the present invention provides a cross-camera tracking method. First, in order to reduce the target loss rate, the cross-camera tracking method provided by the embodiment of the present invention can actively switch the real-time tracking camera to enable the system to update the control circle, check the timeout video, and correct the deviation with the latest trajectory, thereby reducing the target loss probability; secondly, in order to improve the target accuracy, the cross-camera tracking method provided by the embodiment of the present invention can also improve the readiness of tracking target reporting through double verification of face and body features, manual secondary confirmation of angle maps for feature fusion, etc.; finally, a joint indoor and outdoor tracking is proposed to adapt to the scenarios where longitude and latitude cannot adapt to the device relationship.

[0037] First see Figure 2 , Figure 2 The electronic device provided in the embodiment of the present invention can be used to perform the cross-lens tracking method provided in the embodiment of the present invention. For example, the electronic device can be but is not limited to Figure 1 The server 103 or the terminal device 101.

[0038] like Figure 2 As shown, the electronic device 200 includes a memory 201, a processor 202 and a communication interface 203, and the memory 201, the processor 202 and the communication interface 203 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0039] The memory 201 can be used to store software programs and modules, such as the instructions / modules of the cross-mirror tracking device 400 provided in the embodiment of the present invention, which can be stored in the memory 201 in the form of software or firmware or fixed in the operating system (OS) of the electronic device 200. The processor 202 executes various functional applications and data processing by executing the software programs and modules stored in the memory 201. The communication interface 203 can be used to communicate signaling or data with other node devices.

[0040] Among them, the memory 201 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0041] The processor 202 may be an integrated circuit chip with signal processing capability. The processor 202 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0042] Understandably, Figure 2 The structure shown is for illustration only. The electronic device 200 may also include Figure 2 More or fewer components as shown, or with Figure 2 Different configurations are shown. Figure 2 The components shown may be implemented in hardware, software or a combination thereof.

[0043] See also Figure 3 , Figure 3 A schematic flow chart of a cross-camera tracking method provided by an embodiment of the present invention. The execution subject of the method may be Figure 1 The server 103 in the embodiment comprises the following steps:

[0044] S301, if all cameras in the current camera monitoring circle have not captured an image of the target object within the theoretical occurrence time period and the preset buffer time period, the current camera monitoring circle is expanded;

[0045] S302, based on all cameras in the expanded camera monitoring circle, start a video search mechanism to determine a target video recording time of the searched target object;

[0046] S303, if the target recording time is greater than the last appearance time of the target object in the current camera monitoring circle, then updating the current camera monitoring circle based on the camera when the target object appears at the target recording time;

[0047] S304: Track the target object based on the updated current camera monitoring circle.

[0048] According to the cross-lens tracking method provided by an embodiment of the present invention, if all cameras in the current camera monitoring circle have not captured the image of the target object within the theoretical appearance time period and the preset buffer time period, it indicates that the camera in the control circle has not captured the target object, resulting in the loss of the target object, that is, the current camera monitoring circle is invalid, and the current camera monitoring circle needs to be adjusted, that is, the current camera monitoring circle is expanded, and based on all cameras in the expanded camera monitoring circle, a video search mechanism is started to determine the target recording time of the searched target object. If the target recording time is greater than the last appearance time of the target object in the current camera monitoring circle, it indicates that the expanded camera monitoring circle can track the target object. At this time, the current camera monitoring circle can be updated based on the camera when the target object appears at the recording time, and the target object is tracked based on the updated current camera monitoring circle.

[0049] It can be seen that the embodiment of the present invention can determine whether the target object has been lost through timeout detection, thereby determining whether the current camera monitoring circle has failed. After the current camera monitoring circle fails, the target object can be tracked again based on the video search mechanism. Once the time when the target appears again is greater than the last appearance time of the target object in the current prevention and control circle, the current prevention and control circle can be updated based on the camera that tracks the target object. The whole process can correct the camera monitoring circle and reduce the probability of target loss.

[0050] The following will be combined with the attached Figure 4 To Attachment Figure 8 The above steps S304 to S306 are described in detail.

[0051] In step S301, if all cameras in the current camera monitoring circle have not captured an image of the target object within the theoretical occurrence time period and the preset buffer time period, the current camera monitoring circle is expanded.

[0052] In the embodiment of the present invention, the theoretical occurrence time period can be achieved by the following steps:

[0053] a1, determine the maximum distance between the master camera and multiple slave cameras in the current camera monitoring circle.

[0054] a2, based on the maximum distance and the preset moving speed, determine the theoretical appearance time period;

[0055] Among them, in order to meet the tracking business, within the current camera monitoring circle, the main camera and the slave camera can switch with each other, and the main camera can be switched adaptively. For example, the main camera can be switched manually, or the server can adaptively switch the main camera according to the actual tracking results.

[0056] With respect to the above step a1, in an optional implementation manner, the distance between the master camera and each slave camera may be calculated first, and then the maximum distance may be determined from all the distances.

[0057] For the above step a2, the preset moving speed is used to simulate the walking speed of people, which can be set according to the actual scene. The maximum distance is divided by the preset moving speed, and the time obtained is the theoretical occurrence time period. The buffer time period can be set according to actual needs.

[0058] In an embodiment of the present invention, the current moment can be taken as the time starting point. If the target object is not found within the theoretical appearance time (that is, the target object is recognized in all images captured by the camera), it is continued to be determined whether the target object appears in the current camera monitoring circle within the buffer time period. If it still does not appear, it is considered that the target object is lost in the current camera monitoring circle.

[0059] For example, the current time is 9:00, the preset moving speed is 1 meter per second, the maximum distance is 50 meters, the buffer time is 2 minutes, and the theoretical appearance time is 50 seconds. Then, starting from 9 o'clock, if the target object appears within 2 minutes and 50 seconds after 9 o'clock, the current camera monitoring circle will still track the target object. If it still does not appear after 2 minutes and 50 seconds after 9 o'clock, the previous camera monitoring circle has expired and needs to be updated.

[0060] After determining that the current camera monitoring circle is invalid, the current camera monitoring circle can be expanded. The range of the expanded current camera monitoring circle can be several times the range of the current camera monitoring circle before expansion. For example, assuming that there are 5 cameras in the current camera monitoring circle, there can be 10 cameras in the expanded current camera monitoring circle.

[0061] In actual scenarios, when expanding the current camera monitoring circle, cameras located near the current prevention and control circle can be pulled into the current camera monitoring circle. The edge cameras of the camera monitoring circle can be used as a reference to pull other cameras close to the edge cameras into the current camera monitoring circle. The main camera can also be used as the center to pull cameras whose distance from the main camera meets a preset distance range into the monitoring circle. There is no limitation here.

[0062] In step S302, based on all cameras in the expanded camera monitoring circle, a video search mechanism is started to determine a target video recording time of the searched target object.

[0063] From the above content, it can be seen that there are some new cameras in the expanded current camera monitoring circle. These cameras are all cameras near the current camera monitoring circle. When the target is lost in the current camera monitoring circle, theoretically the target object should still be near the current camera monitoring circle. Therefore, based on the expanded current camera monitoring circle, a video search mechanism is performed to increase the probability of tracking the target object.

[0064] In the embodiment of the present invention, the video search mechanism represents searching for the target object in the video collected by each camera in the expanded current camera monitoring circle until the target object is found. The video analysis multiple can be evenly distributed using the current remaining graphics card computing power.

[0065] The last appearance time of the target object in the current camera monitoring circle is: the time when the image of the target object is collected in the current camera monitoring circle is the time closest to the current time. For example, assuming the current time is 9 o'clock, there are 5 cameras in the current camera monitoring circle that collect images of the target object at 8 o'clock, 8:10, 8:20, 8:30, and 8:40 respectively, and the target object does not appear afterwards, then 8:40 is the last appearance time of the target object.

[0066] The above recording time can be set to the time before the target object last appeared in the current camera monitoring circle to the current time. For example, assuming that the target object last appeared at 8:50 and the current time is 9:00, then the recording time can be set to 8:45 to 9:00.

[0067] Therefore, in an optional implementation, after starting the video search mechanism, the image of the target object that meets the facial feature similarity and body feature similarity can be pushed to the front-end device. The user can select "Confirm target trajectory" according to actual needs through the front-end device. After the device receives the user's selection instruction, it can determine the recording time corresponding to the image of the selected target object, and then compare it with the last appearance time of the target object in the current camera monitoring circle (that is, the latest trajectory time), and determine whether to update the current camera monitoring circle based on the comparison result.

[0068] In step S303, if the target recording time is greater than the last appearance time of the target object in the current camera monitoring circle, the current camera monitoring circle is updated based on the camera when the target object appears at the target recording time.

[0069] For example, assuming that the target object last appeared in the current camera monitoring circle at 8:50, and the target recording time of the latest search target is 8:55, then the camera monitoring circle needs to be updated at this time. If the target recording time of the latest search target is 8:45, the camera monitoring circle does not need to be updated.

[0070] Therefore, the above steps S302 and S303 can be understood as follows: based on all cameras within the expanded camera monitoring circle, the video search mechanism is started, and the image of the target object that meets the facial feature similarity and body feature similarity is pushed to the front-end device. The front-end device receives the user's selection instruction, determines the image of the selected target object, and determines the target video recording time and camera corresponding to the image of the target object. If the target video recording time is greater than the last appearance time of the target object in the current camera monitoring circle, the current camera monitoring circle is updated with the camera as the center camera; otherwise, the current camera monitoring circle is not updated.

[0071] It can be understood that, on the front-end device, each time the user performs a selection operation, a target recording time and camera can be determined. If the judgment condition in step S303 is met, the current camera monitoring circle is updated with the determined camera as the central camera. If it is not met, it will not be updated. The whole process can achieve close tracking of the target object and improve the monitoring effect of the camera monitoring circle.

[0072] Therefore, in an optional implementation manner, the above step S303 may be performed as follows:

[0073] b1, if the target object is located in an outdoor environment, the adjacent cameras corresponding to several cameras are determined based on the camera's position and road network data, and the current camera monitoring circle is updated based on the adjacent cameras and cameras.

[0074] In an optional implementation, the moving path of the target object may be predicted based on the camera position and road network data, and the current camera monitoring circle may be updated with the camera located on the moving path and the camera when the target object appears at the recording time.

[0075] b2: If the target object is located in an indoor environment, the current camera monitoring circle is updated based on a pre-generated custom camera set; wherein the custom camera set includes adjacent cameras corresponding to the camera.

[0076] It is understandable that in scenes such as indoors where there is no longitude and latitude or where the longitude and latitude overlap, the camera monitoring circle generated in the above manner cannot reflect the device relationship. Therefore, the embodiment of the present invention first provides an implementation method for establishing an indoor camera monitoring circle: that is, generating a custom camera set, which includes a central camera and multiple adjacent cameras near the central camera. Figure 4 , Figure 4 A schematic diagram of a custom camera set provided by an embodiment of the present invention, from Figure 4 As can be seen from the figure, the custom camera set can reflect the device relationship between the indoor 1st floor (1F) and 2nd floor (2F).

[0077] Customized camera collections can be realized through three methods: resource tree selection, map selection, and table import.

[0078] Among them, resource tree selection and map selection are both ways to manually select the central camera and multiple adjacent cameras around the central camera from the basic device library to establish the spatial relationship between the cameras. Figure 5 As shown, Figure 5 A schematic diagram of generating a custom camera set by selecting resource numbers and clicking on a map provided in an embodiment of the present invention.

[0079] The table import method is to manually fill in the data and import it according to the following template to generate a custom camera table. The imported device must exist in the basic device library. The generated custom camera table is shown in Table 1:

[0080] Table 1

[0081]

[0082] Therefore, when tracking a target object in an indoor environment, a camera monitoring circle can be generated based on a pre-generated custom camera set. Since the custom camera set contains adjacent cameras corresponding to the central camera, several adjacent cameras can be randomly determined to update the current camera monitoring circle.

[0083] It can also be understood that when the target object is located at the edge of the indoor environment and is entering the outdoor environment, that is, the target object is located at the junction of the outdoor environment and the indoor environment, then using the indoor custom camera set to define the camera monitoring circle will result in data loss and will not constitute a complete camera monitoring circle. In order to complete the prevention and control circle, the camera that captures the target object in the junction area can be used as the central camera; again, the latitude and longitude information of the device is combined with path planning to pull the adjacent cameras corresponding to the central camera in the outdoor environment to form a new camera monitoring circle that combines indoors and outdoors. Similarly, when the target object is entering the indoor environment, the camera that captures the target object in the junction area can be used as the central camera, and several adjacent connections can be determined from the custom camera set to generate a new camera monitoring circle.

[0084] In step S304, the target object is tracked based on the updated current camera monitoring circle.

[0085] In the embodiment of the present invention, considering that only using human features for target tracking may result in misjudgment or missed judgment, when the tracking target moves to the high-definition camera, the target recognition algorithm can recognize a clear face, and at this time, the face and the body are associated in real time. The face and the body are regarded as the same target, and the system automatically issues a face tracking task, performs a double comparison and verification on the tracking target, and when the face comparison result and the body comparison result are the same target, the target is determined to be identified, thereby improving the accuracy of the tracking target.

[0086] Therefore, after updating the current camera monitoring circle, the human body image of the target object in the target video can be fused with the existing human body image for multi-image features, and the face image of the target object in the target video can be fused with the face image for multi-image features, and tracking can be performed based on the fused face features and human body features. Therefore, in an optional implementation, the above step S304 can be performed as follows:

[0087] c1, obtaining multiple face images and human body images to be processed corresponding to the target object;

[0088] The target object in each human body image to be processed has a different human body angle. The face image to be processed meets the face feature similarity condition, and the human body image to be processed meets the human body feature similarity condition. In the real-time tracking process, the user can select multiple human body images at different angles and multiple face images from the face images and human body images corresponding to the pushed target object. The device uses multi-image feature fusion to improve the target feature based on the image selected by the user, and continues tracking with the fused features.

[0089] c2, based on the preset facial organ weights, feature fusion is performed on multiple facial images to be processed to obtain fused facial features, and based on the preset human body angle weights, feature fusion is performed on multiple human body images to be processed to obtain fused human body features.

[0090] In an embodiment of the present invention, weights can be set in advance for key facial organs (eye shape, nose, forehead, ears) in the face, and then, multiple face images to be processed are subjected to feature fusion, and the fused face features are subjected to weighted coefficient fusion to obtain fused face image features.

[0091] Similarly, for the human body image to be processed, the human body angle weights are set. For example, the weight of the front of the human body is 0.4, the weight of the front side of the human body is 0.4, and the weight of the back of the human body is 0. The calculation formula for the weights of different offset front and side angles is: offset front angle / 360°*side weight 0.4.

[0092] c3, based on the fused human features and fused face features, the target object is tracked.

[0093] For easier understanding, see Figure 6 , Figure 6 An example diagram of multi-image feature fusion provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown in FIG. 1 , there are 5 kinds of human body images at different angles, namely, front view, side view, and back view. There are offset angles relative to the front view and side view. Then, based on the weight corresponding to each human body angle, the human body images at these 5 kinds of angles can be fused to obtain the fused human body features. Similarly, Figure 6 The 18 face images can be combined with the preset face organ weights for feature fusion to obtain the fused face features.

[0094] Through the above implementation, the readiness of tracking target reporting can be improved by performing double verification with fused facial features and fused human bodies, and manually confirming the angle map for feature fusion.

[0095] In an optional implementation, assuming that the target object appears after the updated current camera monitoring circle, but the user does not receive the reporting information due to system delay, the following implementation can be performed to automatically correct the camera monitoring circle:

[0096] d1, if a main camera switching command input by the user is received, determine the main camera after switching within the current camera monitoring circle;

[0097] d2, update the current camera monitoring circle with the main camera as the center.

[0098] By actively switching the real-time tracking camera, the system adaptively updates the camera monitoring circle, reducing the probability of losing the target.

[0099] In an optional implementation, during the tracking process, a certain number of devices in the dynamic control circle must be reached to cover the moving path of the tracking target as comprehensively as possible, and even outdoors, there will be camera devices with the same longitude and latitude vertically in space. Therefore, the control circle should have an automatic robust mechanism. Therefore, an embodiment of the present invention provides an automatic robust implementation of the camera monitoring circle. The main idea of ​​this implementation is: adopt at least one of the following implementations.

[0100] See also Figure 7 , Figure 7 A schematic flow chart of another cross-camera tracking method provided by an embodiment of the present invention, the method may further include:

[0101] S305, if the total number of cameras in the current camera monitoring circle is less than the preset number, determining a target number of cameras to be pulled so that the total number of cameras is equal to the preset number;

[0102] S306, updating the target number of cameras to be pulled into the current camera monitoring circle;

[0103] The camera to be pulled is any one of the following cameras and their combination: a custom camera; a path planning camera; a straight-line distance camera;

[0104] Custom cameras represent cameras in the custom camera set; path planning cameras represent cameras determined based on road network data, specifically: path planning is performed based on the position of the central camera to obtain the target path, and then the cameras on or near the target path are obtained in combination with the road network data to determine several path planning cameras; straight-line distance cameras represent cameras that are closest to the central camera in a straight-line manner, specifically: among the cameras around the central camera, the distance between each camera and the central camera is calculated separately, and then several straight-line distance cameras are determined in the direction of distance from small to large.

[0105] In the embodiment of the present invention, the customized equipment is preferentially included in the control circle. When the number is insufficient, it is supplemented by the equipment planned by the path. If the number is still insufficient, the control circle is completed by cameras with similar horizontal straight-line distances calculated by GIS. Pulling cameras with similar straight-line distances also improves the vertical relationship of the spatial camera equipment, and also increases the possibility of tracking targets with strong anti-reconnaissance capabilities. The priority is: customized camera>path planning camera>straight-line distance camera.

[0106] Therefore, in an optional implementation manner, the above step S305 may be performed as follows:

[0107] e1, determine whether a custom camera set exists;

[0108] e2, if it exists, the target number of custom cameras in the custom camera set are determined as the cameras to be pulled;

[0109] e3, if not, determine whether there is road network data;

[0110] e4, if it exists, a number of path planning cameras are determined based on the road network data, and a target number of path planning cameras are selected as the cameras to be pulled;

[0111] e5, if it does not exist, then select the target number of straight-line distance cameras from a number of straight-line distance cameras in the direction from the smallest to the largest as the cameras to be pulled.

[0112] In an optional implementation, during the execution of the above implementation, it may happen that the number of custom cameras, path planning cameras, or straight-line distance cameras is less than the target number. At this time, at least two cameras can be determined to update the current camera monitoring circle, and the sum of the number of at least two cameras is the above target number. Therefore, the embodiment of the present invention also provides the following implementation method:

[0113] f1, determine whether the number of custom cameras in the custom camera set is greater than or equal to the target number;

[0114] f2, if yes, then the target number of custom cameras in the custom camera set are used as the cameras to be pulled;

[0115] f3, if not, all custom cameras and the first remaining number of path planning cameras among the plurality of path planning cameras are taken as the cameras to be pulled; the sum of the first remaining number and the number of custom cameras is equal to the target number;

[0116] f4, if the total number of path planning cameras is less than the first remaining number, all custom cameras, all path planning cameras, and the second remaining number of straight-line distance cameras are used as cameras to be pulled; the sum of the second remaining number and the total number of path planning cameras is the first remaining number.

[0117] The effect diagram of the camera monitoring circle determined by the above implementation can be seen in Figure 8 , Figure 8 An example diagram of an automatic robust camera monitoring circle provided by an embodiment of the present invention.

[0118] For easier understanding of the above steps e1 to e5 and steps f1 to f4, please refer to FIG. 9 . Fig. 9 A schematic diagram of a process of an automatic robust camera monitoring circle provided by an embodiment of the present invention, the process may include the following process:

[0119] Step 1: Determine if a custom camera collection exists.

[0120] If yes, go to step 2, if no, go to step 4.

[0121] Step 2: Determine whether the number of custom cameras in the custom camera set is greater than or equal to the target number.

[0122] If yes, go to step 3 and end; if no, go to step 4.

[0123] Step 3: Select the target number of custom cameras in the custom camera collection as the cameras to be pulled.

[0124] Step 4: Determine whether road network data exists.

[0125] If yes, go to step 5; if no, go to step 9 and end.

[0126] Step 5: Based on the road network data, determine a number of path planning cameras.

[0127] Step 6: Whether the total number of path planning cameras is greater than or equal to the first remaining number.

[0128] If yes, go to step 7, if no, go to step 8.

[0129] Step 7: All custom cameras and the first remaining number of path planning cameras among the plurality of path planning cameras are taken as cameras to be pulled, and the process ends.

[0130] Step 8: All custom cameras, all path planning cameras, and the second remaining number of straight-line distance cameras are taken as cameras to be pulled, and the process ends.

[0131] Step 9: Select the target number or the first remaining number of linear distance cameras from a plurality of linear distance cameras in the direction of linear distance from the central camera from small to large as the cameras to be pulled, and end.

[0132] It should be noted that when there is neither a custom camera set nor road network data, then execute step 9: select a target number of straight-line distance cameras from a number of straight-line distance cameras in the direction of the straight-line distance to the center camera from small to large, as the cameras to be pulled, and end; when there is a custom camera set but there is no road network data, then execute step 9: select a target number of straight-line distance cameras from the first remaining number of straight-line distance cameras in the direction of the straight-line distance to the center camera from small to large, as the cameras to be pulled, and end.

[0133] The cross-camera tracking method provided in the embodiment of the present application can be executed in a hardware device or implemented in the form of a software module. When the cross-camera tracking method is implemented in the form of a software module, the embodiment of the present application also provides a cross-camera tracking method device, see Fig.10 , Fig.10 This is a functional module diagram of a cross-camera tracking device provided in an embodiment of the present application. The cross-camera tracking device 400 may include:

[0134] The determination module 410 is used to expand the current camera monitoring circle if all cameras in the current camera monitoring circle have not captured an image of the target object within the theoretical occurrence time period and the preset buffer time period;

[0135] The search module 420 is used to start a video search mechanism based on all cameras in the expanded camera monitoring circle to determine the target video time of the target object that is first searched out;

[0136] An updating module 430, configured to update the current camera monitoring circle based on the camera when the target object appears at the target recording time, if the target recording time is greater than the last appearance time of the target object in the current camera monitoring circle;

[0137] The tracking module 440 is used to track the target object based on the updated current camera monitoring circle.

[0138] In an alternative embodiment, the determination module 410, the search module 420, the update module 430 and the tracking module 440 may be executed in coordination. Figure 3 Each step in the process is performed to achieve the corresponding technical effect.

[0139] In an optional implementation, the update module 430 is specifically configured to execute steps b1 and b2, steps d1 to d2, steps e1 to e5, steps f1 to f4, Figure 7 , Fig. 9 The various steps in the process are performed to achieve the effect of updating the camera monitoring circle.

[0140] In an optional implementation, the tracking module 440 is specifically used in steps c1 to c3 to achieve corresponding effects.

[0141] In an optional implementation, the determination module 410 is also used to execute step a1 to step a2 to achieve corresponding technical effects.

[0142] The present application also provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the cross-lens tracking method in any of the aforementioned embodiments is implemented. The computer readable storage medium can be, but is not limited to, a USB flash drive, a mobile hard disk, a ROM, a RAM, a PROM, an EPROM, an EEPROM, a magnetic disk, or an optical disk, and other media that can store program codes.

[0143] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A cross-camera tracking method, characterized in that: The method comprises: If all cameras in the current camera monitoring circle fail to capture an image of the target object within the theoretical occurrence time period and the preset buffer time period, the current camera monitoring circle is expanded; Based on all cameras within the expanded camera monitoring circle, a video search mechanism is started to determine a target video recording time of the searched target object; If the target recording time is greater than the last appearance time of the target object in the current camera monitoring circle, updating the current camera monitoring circle based on the camera when the target object appears at the target recording time; Tracking the target object based on the updated current camera monitoring circle; If the total number of cameras in the current camera monitoring circle is less than a preset number, a target number of cameras to be pulled is determined so that the total number of cameras is equal to the preset number, including: determining whether there is a custom camera set; if so, determining the target number of custom cameras in the custom camera set as the cameras to be pulled; if not, determining whether there is road network data; if so, determining a number of path planning cameras based on the road network data, and selecting the target number of path planning cameras as the cameras to be pulled; if not, determining a target number of straight-line distance cameras in the direction of the straight-line distance to the center camera from small to large in sequence as the cameras to be pulled; wherein the custom cameras represent cameras belonging to the custom camera set; the path planning cameras represent cameras determined based on the road network data; and the straight-line distance cameras represent cameras that are closest in straight-line distance to the center camera; The target number of cameras to be pulled is updated into the current camera monitoring circle.

2. The cross-camera tracking method according to claim 1, characterized in that: If the target recording time is greater than the last appearance time of the target object in the current camera monitoring circle, updating the current camera monitoring circle based on the camera when the target object appears at the target recording time includes: If the target object is located in an indoor environment, the current camera monitoring circle is updated based on a pre-generated custom camera set; wherein the custom camera set includes adjacent cameras corresponding to the camera; If the target object is located in an outdoor environment, a number of adjacent cameras corresponding to the camera are determined based on the position of the camera and road network data, and the current camera monitoring circle is updated based on the adjacent cameras and the camera.

3. The cross-camera tracking method according to claim 1, characterized in that: The method further comprises: Determine whether the number of custom cameras in the custom camera set is greater than or equal to the target number; If so, taking the target number of the custom cameras in the custom camera set as the cameras to be pulled; If not, all the custom cameras and the first remaining number of the path planning cameras among the plurality of the path planning cameras are used as the cameras to be pulled; the sum of the first remaining number and the number of the custom cameras is equal to the target number; If the total number of the path planning cameras is less than the first remaining number, all the custom cameras, all the path planning cameras, and the second remaining number of the straight-line distance cameras will be used as the cameras to be pulled; the sum of the second remaining number and the total number of path planning cameras is the first remaining number.

4. The cross-camera tracking method according to claim 1, characterized in that: Tracking the target object based on the updated current camera monitoring circle includes: Acquire a plurality of to-be-processed face images and a plurality of to-be-processed body images corresponding to the target object; the body angle of the target object in each of the to-be-processed body images is different; Based on preset facial organ weights, feature fusion is performed on the plurality of facial images to be processed to obtain fused facial features, and based on preset human body angle weights, feature fusion is performed on the plurality of human body images to be processed to obtain fused human body features; The target object is tracked based on the fused human body features and the fused facial features.

5. The cross-camera tracking method according to claim 1, characterized in that: The method further comprises: If a main camera switching instruction input by the user is received, determining the main camera to be switched within the current camera monitoring circle; The current camera monitoring circle is updated with the main camera as the center.

6. The cross-camera tracking method according to claim 1, characterized in that: If all cameras in the current camera monitoring circle fail to capture an image of the target object within the theoretical appearance time and the preset buffer time, before expanding the current camera monitoring circle, the method includes: Determine the maximum distance between a master camera and multiple slave cameras in the current camera monitoring circle; The theoretical occurrence time period is determined based on the maximum distance and a preset moving speed.

7. A cross-camera tracking device, characterized in that: include: A determination module, configured to expand the current camera monitoring circle if all cameras in the current camera monitoring circle have not captured an image of the target object within a theoretical occurrence time period and a preset buffer time period; A search module, used to start a video search mechanism based on all cameras within the expanded camera monitoring circle, and determine a target video recording time of the searched target object; An updating module, configured to update the current camera monitoring circle based on the camera at which the target object appears at the target recording time if the target recording time is greater than the last appearance time of the target object in the current camera monitoring circle; A tracking module, configured to track the target object based on the updated current camera monitoring circle; The update module is also used to determine a target number of cameras to be pulled if the total number of cameras in the current camera monitoring circle is less than a preset number, so that the total number of cameras is equal to the preset number, including: determining whether there is a custom camera set; if so, determining the target number of custom cameras in the custom camera set as the cameras to be pulled; if not, determining whether there is road network data; if so, determining a number of path planning cameras based on the road network data, and selecting the target number of path planning cameras as the cameras to be pulled; if not, determining a target number of straight-line distance cameras in the direction from the smallest to the largest straight-line distance to the center camera in sequence as the cameras to be pulled; wherein the custom cameras represent cameras belonging to the custom camera set; the path planning cameras represent cameras determined based on the road network data; the straight-line distance cameras represent cameras with the closest straight-line distance to the center camera; and updating the target number of cameras to be pulled into the current camera monitoring circle.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the method according to any one of claims 1 to 6.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images