Target tracking method and device, readable storage medium, camera equipment and server

Abstract

The invention discloses a target tracking method and device, a readable storage medium, camera equipment and a server, and the method comprises the steps: updating a first target tracking result of the camera equipment based on a second target tracking result of the server and a preset updating weight. Through the above method, the server can integrate the first target tracking results of the camera devices in the camera system to obtain the more accurate second target tracking result, and the camera devices can update the first target tracking result according to the second target tracking result. Therefore, the camera equipment can break through the limitation of a shooting view during target tracking, more comprehensive tracking information is obtained, and the accuracy of target tracking is improved.

Description

Technical Field

[0001] The present application relates to the technical field of target tracking, and in particular to a target tracking method, device, readable storage medium, camera equipment and server. Background Art

[0002] In live broadcast applications of sports events, entertainment performances, etc., multiple cameras are often used to track multiple targets from different angles to better present the real situation of the scene. However, when tracking targets, the camera is limited by the shooting field of view, resulting in low accuracy of successful tracking. SUMMARY OF THE INVENTION Technical issues

[0003] One of the purposes of the embodiments of the present application is to provide a target tracking method, device, and readable storage medium. , camera equipment and servers. Technical Solutions

[0004] The technical solution adopted in the embodiment of the present application is:

[0005] In a first aspect, a target tracking method applied to a camera device is provided, which may include:

[0006] Get the target video;

[0007] Perform target tracking based on the target video to obtain a first target tracking result;

[0008] Sending the target video to a server;

[0009] Receiving a second target tracking result sent by the server;

[0010] The first target tracking result is updated based on the second target tracking result and a preset update weight to obtain a third target tracking result.

[0011] In a second aspect, a target tracking method applied to a server is provided, which may include:

[0012] Receive target videos sent by various camera devices;

[0013] Perform target tracking based on each of the target videos to obtain a second target tracking result;

[0014] The second target tracking result is sent to each of the camera devices.

[0015] In a third aspect, a target tracking device applied to a camera device is provided, which may include:

[0016] A target video acquisition module is used to acquire a target video;

[0017] A target tracking module, used to perform target tracking based on the target video to obtain a first target tracking result;

[0018] A target video sending module, used for sending the target video to a server;

[0019] A tracking result receiving module, used for receiving the second target tracking result sent by the server;

[0020] The tracking result updating module is used to update the first target tracking result based on the second target tracking result and a preset update weight to obtain a third target tracking result.

[0021] In a fourth aspect, a target tracking device applied to a server is provided, which may include:

[0022] A target video receiving module is used to receive target videos sent by various camera devices;

[0023] A target tracking module, used to perform target tracking based on each of the target videos to obtain a second target tracking result;

[0024] The tracking result sending module is used to send the second target tracking result to each of the camera devices.

[0025] In a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned target tracking methods are implemented.

[0026] In a sixth aspect, a camera device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of any one of the above-mentioned target tracking methods applied to the camera device are implemented.

[0027] In a seventh aspect, a server is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of any one of the above-mentioned target tracking methods applied to the server are implemented.

[0028] In an eighth aspect, a computer program product is provided, which, when executed on a camera device, enables the camera device to execute the steps of any one of the above-mentioned target tracking methods. Beneficial Effects

[0029] The beneficial effect of the target tracking method provided by the embodiment of the present application is that the target tracking method provided by the embodiment of the present application updates the first target tracking result of the camera device based on the second target tracking result of the server and the preset update weight. Through the target tracking method provided by the embodiment of the present application, the server can integrate the first target tracking results of each camera device in the camera system to obtain a more accurate second target tracking result, and each camera device can update the first target tracking result according to the second target tracking result, so that the camera device can break through the limitations of the shooting field of view when tracking the target and obtain more comprehensive tracking information, thereby helping to improve the accuracy of target tracking.

[0030] The beneficial effect of the target tracking device provided by the embodiment of the present application is that the target tracking device provided by the embodiment of the present application updates the first target tracking result of the camera device based on the second target tracking result of the server and the preset update weight. Through the target tracking device provided by the embodiment of the present application, the server can integrate the first target tracking results of each camera device in the camera system to obtain a more accurate second target tracking result, and each camera device can update the first target tracking result according to the second target tracking result, so that the camera device can break through the limitations of the shooting field of view when tracking the target and obtain more comprehensive tracking information, thereby helping to improve the accuracy of target tracking.

[0031] The beneficial effect of the computer-readable storage medium provided in the embodiment of the present application is that the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned target tracking method are implemented.

[0032] The beneficial effect of the camera device provided by the embodiment of the present application is that the camera device provided by the embodiment of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the steps of any one of the above-mentioned target tracking methods applied to the camera device.

[0033] The beneficial effect of the server provided by the embodiment of the present application is that: the server provided by the embodiment of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the steps of any one of the above-mentioned target tracking methods applied to the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or exemplary technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A schematic diagram of a camera system in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of a tracking failure scenario;

[0037] Figure 3 A schematic diagram of another tracking failure scenario;

[0038] Figure 4 This is a flow chart of an embodiment of a target tracking method applied to a camera device in an embodiment of the present application;

[0039] Figure 5 This is a flow chart of an embodiment of a target tracking method applied to a server in an embodiment of the present application;

[0040] Figure 6 Schematic diagram of the target tracking process;

[0041] Figure 7 This is a structural diagram of an embodiment of a target tracking device applied to a camera device in an embodiment of the present application;

[0042] Figure 8 This is a structural diagram of an embodiment of a target tracking device applied to a server in an embodiment of the present application;

[0043] Fig. 9 This is a schematic block diagram of a camera device in an embodiment of the present application;

[0044] FIG10 is a schematic block diagram of a server in an embodiment of the present application. Embodiments of the present invention

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present application.

[0046] It should be noted that when a component is referred to as being "fixed on" or "disposed on" another component, it may be directly on the other component or indirectly on the other component. When a component is referred to as being "connected to" another component, it may be directly or indirectly connected to the other component. The orientation or positional relationship indicated by the terms "upper", "lower", "left", "right", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances. The terms "first" and "second" are only used for the purpose of convenience of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features. The meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0047] In order to illustrate the technical solution provided by the present application, a detailed description is given below in conjunction with specific drawings and embodiments.

[0048] In live broadcast applications of sports events, entertainment performances, etc., multiple cameras are often used to track multiple targets from different angles to better present the real situation of the scene. However, when tracking targets, the camera is limited by the shooting field of view, resulting in low accuracy of successful tracking.

[0049] In view of this, the embodiments of the present application provide a target tracking method, device, computer-readable storage medium, camera device and server to solve the problem that the camera device in the prior art is limited by the shooting field of view when performing target tracking, resulting in low accuracy of successful tracking.

[0050] In the embodiments of the present application, various camera devices can be combined into a camera system to shoot the target at different angles to present the scene from different perspectives, thereby enhancing the audience's viewing experience.

[0051] Specifically, the camera system may include a preset number of camera devices, and the various camera devices may be reasonably arranged so that the cross-coverage of the captured image by each camera device is kept at a low level; optionally, the camera system may include at least one camera device arranged at a higher position and a preset number of camera devices arranged at a lower position; wherein, the number of camera devices arranged at a lower position may be set according to actual needs, and the present application does not limit this; the camera device arranged at a higher position may be a fisheye camera or a wide-angle camera for shooting clear and comprehensive videos; the camera device arranged at a lower position may be a camera with an interchangeable lens for shooting a specific area at any focal length.

[0052] For example, see Figure 1 The camera device 1 can be set at a higher position so that the captured video can cover the scene as much as possible. The camera devices 2, 3 and 4 can also be set at a lower position to shoot a specific area.

[0053] When shooting, the camera device needs to track the target in the field of view to better focus and provide a clearer picture for the audience. However, a single camera device is limited by the field of view and is prone to lose the target or follow the wrong target during tracking.

[0054] For example, see Figure 2 , in picture 1, target 1 and target 2 can be identified; in picture 2, target 1 and target 2 meet, and the positions of the two targets in the video overlap and are blocked; in picture 3, target 1 and target 2 are separated, but because the positions of the two targets in picture 2 overlap and are blocked before, target 1 is mistakenly identified as target 2, and target 2 is mistakenly identified as target 1, resulting in tracking targets.

[0055] For another example, see Figure 3 In picture 1, target 1 can be identified; in picture 2, target 1 leaves the picture; in picture 3, target 1 re-enters the picture, but because target 1 has left the picture before, target 1 is identified as target 2 (new target), that is, target 1 is incorrectly identified, resulting in tracking failure.

[0056] Therefore, in an embodiment of the present application, each camera device can be connected to the server and send the video it has captured to the server. The server can perform more comprehensive and accurate tracking based on the received videos, and can send the tracking results to each camera device. The camera device can update its own tracking results based on the more accurate tracking results provided by the server, which helps to improve the accuracy of target tracking.

[0057] It should be noted that the camera device in the embodiment of the present application can be any wide-angle camera or pan-tilt camera, or other common camera devices; the server in the embodiment of the present application can be common computing devices such as desktop computers, notebooks, and handheld computers, or other computing devices.

[0058] For ease of description, the target tracking method of the embodiment of the present application will be described in detail from the perspective of any camera device. Figure 4 In the embodiment of the present application, an embodiment of a target tracking method applied to a camera device may include the following steps S401 to S405:

[0059] Step S401: Obtain target video.

[0060] In an embodiment of the present application, the camera device can shoot a specific area in real time and obtain a target video; thereafter, the target in the target video can be identified and continuously tracked to capture the object quickly and accurately.

[0061] Optionally, in practical applications, the target video may be preprocessed according to actual needs to obtain the preprocessed target video, and subsequent target tracking may be performed based on the preprocessed target video. In the embodiment of the present application, one or more preprocessing steps may be performed on the target video according to actual needs.

[0062] For example, the target video may be pre-processed for resolution adjustment. Specifically, any video editing software in the prior art may be used to adjust the resolution of the target video to a specific resolution to obtain the target video with adjusted resolution.

[0063] For another example, the target video may be preprocessed for format conversion. Specifically, any video conversion tool in the prior art may be used to convert the target video into a specific format to obtain the target video after format conversion.

[0064] For another example, the target video may be preprocessed for frame rate adjustment. Specifically, any video editing software in the prior art may be used to adjust the frame rate of the target video to a specific frame rate to obtain the target video after the frame rate is adjusted.

[0065] Step S402: performing target tracking based on the target video to obtain a first target tracking result.

[0066] In an embodiment of the present application, target detection may be first performed on the target video to obtain a target detection result; then, target tracking may be performed based on the target detection result to obtain a target tracking result; here, the tracking result of the camera device may be referred to as the first target tracking result.

[0067] Specifically, target detection may be performed on the target video to identify and locate each target included in the target video; here, target detection may be performed on the target video with reference to any target detection method in the prior art.

[0068] In a specific implementation method of an embodiment of the present application, a relevant deep learning model can be used to perform target detection on a target video to more accurately and efficiently identify each target; wherein the deep learning model used for target detection can be a deep learning model commonly used in the prior art; specifically, it can include but is not limited to YOLO series, Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Fast R-CNN, Faster R-CNN and other deep learning models.

[0069] After obtaining the target detection result, target tracking may be performed based on the target detection result to obtain a first target tracking result, wherein the first target tracking result may include a motion trajectory of the target.

[0070] In an embodiment of the present application, after each target is identified, features of each target can be extracted to obtain a target feature extraction result; the target feature extraction result may include but is not limited to feature information such as the shape, color, and texture of the target.

[0071] Afterwards, the target feature extraction results in different frames of the target video can be compared to determine whether the target in the current frame matches the target in the previous frame; if the target feature extraction result of a target in the current frame is the same or similar to the target feature extraction result of a target in the previous frame, it can be considered that the two targets match successfully; if the target feature extraction result of a target in the current frame is not similar to the target feature extraction results of each target in the previous frame, it can be determined that the target is a newly appeared target in the current frame; if the target feature extraction results of each target in the previous frame are not similar to the target feature extraction results of each target in the current frame, it can be determined that the target is lost in the current frame.

[0072] According to the successfully matched target, the position of the target in different pictures can be determined, so that the movement trajectory of the target can be determined to obtain the first target tracking result.

[0073] In a specific implementation of the embodiment of the present application, in some scenes (such as dance performances), information such as the order and position of target appearance can be determined in advance; in order to improve the efficiency and accuracy of target tracking, when the target is tracked for the first time, each identified target can be determined based on prior information, and the prior information here may include but is not limited to the number of targets, the order of appearance of targets, the position of appearance of targets, or the characteristics of targets (such as clothing color, facial information, etc.); specifically, a corresponding unique identifier can be set for each target according to the prior information to distinguish different targets. For example, combined with prior information, it can be extracted and determined that there are 3 targets, namely target 1, target 2 and target 3, and the 3 targets will appear in the picture from left to right in sequence. When performing actual target tracking, the target on the far left of the picture can be determined as target 1, the target in the middle of the picture can be determined as target 2, and the target on the far right of the picture can be determined as target 3, and different unique identifiers can be set for the 3 targets; through prior information, the search range of the target can be narrowed, computing resources can be saved, and the foundation for real-time target tracking is laid.

[0074] Step S403: Send the target video to the server.

[0075] In the embodiment of the present application, the camera device and the server can establish a connection through a preset communication link. Through the rich interaction between the camera device and the server, various types of information can be obtained to achieve accurate tracking.

[0076] Among them, the communication link can be a communication link established based on at least one wireless communication solution such as Wireless Local Area Networks (WLAN) (for example, Wireless Fidelity (WiFi)), Bluetooth, Bluetooth Low Energy, Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), Global System for Mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division-Synchronous Code Division Multiple Access (TD-SCDMA) and Long Term Evolution (LTE).

[0077] In an embodiment of the present application, each camera device can send the target video to the server; since the computing power of the server is usually higher than that of the camera device, and the server can receive videos taken by each camera device, the server is more likely to obtain a more accurate target tracking result. Here, the target tracking result of the server can be referred to as the second target tracking result.

[0078] In a specific implementation method of an embodiment of the present application, in order to improve the tracking efficiency of the server, the camera device can also send the target detection results and / or target feature extraction results to the server, so that the server can omit the process of performing target detection and / or feature extraction on the target video, thereby reducing tracking time and improving tracking speed.

[0079] Step S404: receiving the second target tracking result sent by the server.

[0080] After completing the target tracking, the server may send the obtained second target tracking result to each camera device, and the camera device may receive the second target tracking result sent by the server.

[0081] In a specific implementation of the embodiment of the present application, if the first target tracking result only includes the tracking result for a specific target, then after receiving each second target tracking result, the camera device can determine the second target tracking result of the specific target from each second target tracking result. For example, the first target tracking result of the camera device 1 only includes the first target tracking result for target 3, and the second target tracking result sent by the server specifically includes the second target tracking results of targets 1, 2, 3, and 4. Then, after receiving each second target tracking result sent by the server, the camera device 1 can filter out the second target tracking result of target 3 from each second target tracking result.

[0082] Step S405: Update the first target tracking result based on the second target tracking result and a preset update weight to obtain a third target tracking result.

[0083] In the embodiment of the present application, after receiving the second target tracking result sent by the server, the camera device can update the first target tracking result as needed.

[0084] Specifically, the first target tracking result can be updated according to a preset update weight, wherein the value of the update weight can be specific and situationally set according to actual needs, and the embodiment of the present application does not limit this.

[0085] If the credibility of the target tracking result of the server is high, then the value of the update weight can be set to a larger value; if the credibility of the target tracking result of the server is low, then the value of the update weight can be set to a smaller value.

[0086] In an embodiment of the present application, the first target tracking result can be first compared with the second target tracking result to determine whether the first target tracking result needs to be updated; if the similarity between the two target tracking results is greater than a preset similarity threshold, there is no need to update the first target tracking result; if the similarity between the two target tracking results is less than or equal to the similarity threshold, the first target tracking result can be updated; wherein the similarity threshold can be set according to actual conditions, and the present application does not limit this.

[0087] For the convenience of comparison, the first target tracking result and the second target tracking result may be unified into the same coordinate system.

[0088] Here, before the camera system starts shooting, the position information of each camera device may be determined; wherein the position information of the camera device may include relative position information of the camera device in the camera system.

[0089] In a specific implementation of the embodiment of the present application, the height and angle of each camera device in the camera system, as well as the distance between the camera devices, can be obtained, and based on this, the relative position information of the camera device in the camera system can be determined.

[0090] In another specific implementation of the embodiment of the present application, a camera device set at a higher position can be used to shoot other camera devices in the camera system to determine the relative positions of the other camera devices and obtain relative position information of the camera devices.

[0091] In another specific implementation method of the embodiment of the present application, each camera device in the camera system can capture images of the same object (such as a chessboard). Based on the images captured by the camera devices respectively, the relative positions of each camera device can be calculated to obtain the relative position information of each camera device.

[0092] In another specific implementation method of the embodiment of the present application, each camera device in the camera system can be provided with a wireless ranging signal transceiver to send or receive wireless signals; here, any three or more camera devices can be used as reference camera devices (anchor points), each camera device can send a wireless signal to the reference camera device, and can receive a wireless signal replied by the reference camera device, and the camera device can calculate the distance between the camera device and the reference camera device based on the time difference between the wireless signal sent by itself and the wireless signal received; thereafter, a multilateral measurement (Multilateration) algorithm can be used to calculate the relative position information of the camera device in the camera system based on the distance between the camera device and at least three reference camera devices.

[0093] In the embodiment of the present application, the position information of the camera device may also include the absolute position of the camera device in the world coordinate system.

[0094] In a specific implementation of the embodiment of the present application, the absolute position of each camera device in the camera system in the world coordinate system can be determined by manual measurement.

[0095] In another specific implementation method of the embodiment of the present application, a marked object with a known position in the world coordinate system can be photographed, and then the mark in the image can be identified, and the position of the mark can be associated with the corresponding position in the world coordinate system, so as to determine the absolute position of the camera device in the world coordinate system.

[0096] Afterwards, based on the position information of the camera device, the second target tracking result can be converted to the coordinate system corresponding to the first target tracking result to obtain a fourth target tracking result. If the first target tracking result is not the same as or similar to the fourth target tracking result, the first target tracking result can be updated based on the fourth target tracking result and the update weight to obtain a third target tracking result; specifically, the first target tracking result and the fourth target tracking result can be fused by weighted averaging according to the update weight to obtain the third target tracking result. If the first target tracking result is the same as or similar to the fourth target tracking result, there is no need to update the first target tracking result.

[0097] In a specific implementation of an embodiment of the present application, the server may also perform motion detection on the target during target tracking to obtain the target's motion information; wherein the target's motion information may include but is not limited to the target's speed, acceleration, motion direction and other information.

[0098] If the motion information sent by the server is received, the camera device can perform target prediction based on the motion information and the third target tracking result to obtain a target prediction result. Specifically, the subsequent position and state of the target can be predicted using methods such as Kalman filter, linear regression, neural network, etc. This application does not limit the method used in the target prediction process and can be set according to actual needs.

[0099] Afterwards, the camera device can calculate the posture adjustment information of the camera device based on the target prediction result. The posture adjustment information may include but is not limited to information such as rotation angle, translation distance, focal length adjustment, etc.; after calculating the posture adjustment information, the camera device can adjust the posture of the camera device based on the posture adjustment information to ensure that the camera device can maintain stable tracking of the target in a complex environment.

[0100] For example, if the position of target 1 at the next moment is predicted based on the motion information of target 1, the posture of the camera device can be adjusted so that target 1 can be in the middle area of ​​the picture at the next moment.

[0101] In another specific implementation method of the embodiment of the present application, the above-mentioned target prediction process can also be performed in the server. The server can send the obtained target prediction result directly to the camera device, and the camera device can adjust the posture based on the target prediction result.

[0102] In another specific implementation method of the embodiment of the present application, in order to improve tracking efficiency, the first target tracking result may also include intermediate result information of the tracking process, and the intermediate result information may include target detection results or target feature extraction results; the camera device can directly send the intermediate result information to the server so that the server can perform target tracking; based on the intermediate result information, the server skips the target detection and feature extraction steps, thereby completing the target tracking process more quickly, which helps to save computing power resources and ensure the timeliness of tracking.

[0103] In another specific implementation method of the embodiment of the present application, since the process of target detection and feature extraction in the target tracking process is time-consuming, and the computing power of the camera device is usually lower than that of the server, the camera device can send the target video to the server after acquiring the target video, and the server can perform target detection and feature extraction based on the target video to obtain the target detection result and the target feature extraction result, and can send the target feature extraction result to the camera device; the camera device can perform target tracking according to the target feature extraction result sent by the server; by splitting the tracking stage, the work of each stage in the target tracking process can be effectively and reasonably allocated to different devices, which helps to reduce the computing pressure of the camera device.

[0104] In another specific implementation method of the embodiment of the present application, the server can also determine the current scene, determine the composition type that matches the current scene, and send the composition type to the camera device, which can then compose the target video according to the composition type; for example, if the composition type is a vertical screen composition with a 4:3 ratio, the camera device can shoot the target video according to the composition type in subsequent shooting, thereby obtaining a video that is more in line with the composition habits of the current scene, which helps to ensure the user's viewing experience.

[0105] The target tracking method of the embodiment of the present application is described in detail from the perspective of the server. Figure 5 In an embodiment of the present application, an embodiment of a target tracking method applied to a server may include the following steps S501 to S503:

[0106] Step S501: Receive target videos sent by various camera devices.

[0107] In an embodiment of the present application, the server can receive target videos sent by various camera devices; since different camera devices have different positions in the camera system, the shooting angles of different camera devices will also be different; by integrating various target videos shot from different shooting angles, the server can obtain the accurate position of the target.

[0108] Step S502: performing target tracking based on each target video to obtain a second target tracking result.

[0109] The server may perform target detection on each target video to obtain a corresponding target detection result; or the server may splice each target video into one video, and perform target detection on the spliced ​​video to obtain each target detection result. The target detection process may refer to the detailed description above, which will not be repeated here.

[0110] Afterwards, the results of each target detection result can be associated to obtain a global association result. Specifically, feature extraction can be performed on each target detection result to obtain a corresponding target feature extraction result; afterward, the target feature extraction results of each target video can be matched, and the targets corresponding to the successfully matched target feature extraction results can be determined as the same target.

[0111] For example, if the target feature extraction result of target 1 in target video A successfully matches the target feature extraction result of target 3 in target video B, it can be determined that target 1 and target 3 are the same target.

[0112] According to the matching results, the results of each target detection can be associated to obtain a global association result; wherein the global association result can include the position of the target in the target video. For example, if the target feature extraction result of target 1 in target video A matches the target feature extraction result of target 3 in target video B and the target feature extraction result of target 8 in target video D, then target 1, target 3 and target 8 can be associated as the same target, and the target appears in the screens of target video A, target video B and target video D at the same time.

[0113] Based on the global association result, accurate target tracking can be performed to obtain the second target tracking result. For example, see Figure 6 , at the previous moment, target 1 appeared in target video A and target video B, and at the current moment, target 1 appears in target video B and target video C; based on this, it can be determined that target 1 moves from the shooting area corresponding to target video A to the shooting area corresponding to target video C, and the movement trajectory of target 1 can be obtained.

[0114] Step S503: Send the second target tracking result to each camera device.

[0115] In a specific implementation of the embodiment of the present application, the server may broadcast the second target tracking result in the camera system so that each camera device in the camera system can receive a unified second target tracking result.

[0116] In another specific implementation method of the embodiment of the present application, the server can send each second target tracking result to the corresponding camera device respectively; for example, target 1 and target 2 are detected in camera device 1, target 3 is detected in camera device 2, and target 4 is detected in camera device 3; the server can send the second target tracking results of target 1 and target 2 to camera device 1, send the second target tracking result of target 3 to camera device 2, and send the second target tracking result of target 4 to camera device 3.

[0117] In another specific implementation of the embodiment of the present application, since the device performance of the server is usually better than that of the camera device, the server can undertake part of the tracking process of the camera device; specifically, after the camera device obtains the target video, it can directly send the target video to the server; the server can receive each target video, and can perform target detection and feature extraction on each target video to obtain the corresponding target detection results and target feature extraction results; then, the server can send the target feature extraction results to the corresponding camera device, so that each camera device can track the target more quickly. For example, the camera device 1 sends the target video A it shoots to the server, and the server performs target detection and feature extraction on the target video A to obtain the target detection result and target feature extraction result of the target video A; then, the server can send the target feature extraction result of the target video A to the camera device 1, so that the camera device 1 can save the computing power resources required for target detection and feature extraction, and can use the computing power resources for subsequent tracking work.

[0118] In another specific implementation method of the embodiment of the present application, the first target tracking result may also include intermediate result information of the tracking process, and the intermediate result information may include target detection results or target feature extraction results; the camera device can send the intermediate result information to the server, and the server can directly perform subsequent target tracking based on the intermediate result information to obtain the second target tracking result without re-performing target detection or feature extraction, which helps to save computing resources and maintain stable real-time tracking.

[0119] In another specific implementation method of the embodiment of the present application, the server can also perform motion detection on the target during target tracking to obtain the target's motion information; for example, the server can calculate the target's speed, acceleration, and direction of movement based on the target's position at the previous moment and the current moment, and can send the target's motion information to the camera device. The camera device can then predict the target's position at the next moment based on the target's motion information, and adjust its own posture in time so that it can track the target at the next moment.

[0120] In another specific implementation of the embodiment of the present application, since different scenes may have different requirements for composition, in the embodiment of the present application, the video composition can be adjusted according to the requirements for composition of different scenes to ensure the user's viewing experience. Specifically, a scene composition correspondence can be established based on each scene and the corresponding recommended composition type; wherein the scene and the recommended composition type can be set according to actual needs, and the embodiments of the present application do not specifically limit this; when tracking a target, the server can determine the current scene corresponding to the target video based on the target detection result; for example, if it can be determined that a football is detected in the target video based on the target detection result, then the current scene can be determined to be a sports event scene; for another example, if it can be determined that a musical instrument is detected in the target video based on the target detection result, then the current scene can be determined to be a music performance scene; based on the scene composition correspondence, the target composition type corresponding to the current scene can be determined; thereafter, the server can send the target composition type to the camera device so that the camera device can shoot the target video according to the target composition type; alternatively, the server can directly compose each target video according to the target composition type, and can send the composed target video to the corresponding client, thereby ensuring that users can have a better viewing experience in various scenarios.

[0121] In summary, the embodiment of the present application updates the first target tracking result of the camera device based on the second target tracking result of the server and the preset update weight. Through the embodiment of the present application, the server can integrate the first target tracking results of each camera device in the camera system to obtain a more accurate second target tracking result, and each camera device can update the first target tracking result according to the second target tracking result, so that the camera device can break through the limitations of the shooting field of view when tracking the target and obtain more comprehensive tracking information, thereby helping to improve the accuracy of target tracking.

[0122] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0123] Corresponding to a target tracking method described in the above embodiment, Figure 7 A structural diagram of an embodiment of a target tracking device applied to a camera device provided in an embodiment of the present application is shown.

[0124] In an embodiment of the present application, a target tracking device applied to a camera device may include:

[0125] The target video acquisition module 701 is used to acquire the target video;

[0126] A target tracking module 702 is used to perform target tracking based on the target video to obtain a first target tracking result;

[0127] The target video sending module 703 is used to send the target video to the server;

[0128] The tracking result receiving module 704 is used to receive the second target tracking result sent by the server;

[0129] The tracking result updating module 705 is used to update the first target tracking result based on the second target tracking result and a preset update weight to obtain a third target tracking result.

[0130] In a specific implementation of the embodiment of the present application, the target tracking module includes:

[0131] The target detection submodule is used to perform target detection on the target video to obtain a target detection result;

[0132] The target tracking submodule is used to perform target tracking based on the target detection result to obtain the first target tracking result.

[0133] In a specific implementation of the embodiment of the present application, the device further includes:

[0134] A location information acquisition module, used to acquire the location information of the camera device;

[0135] The tracking result updating module comprises:

[0136] a target tracking result calculation submodule, configured to calculate a fourth target tracking result based on the position information and the second target tracking result;

[0137] The tracking result updating submodule is used to update the first target tracking result based on the fourth target tracking result and the update weight to obtain the third target tracking result if the first target tracking result is different from the fourth target tracking result.

[0138] In a specific implementation of the embodiment of the present application, the device further includes:

[0139] A motion information receiving module, used for receiving the motion information sent by the server;

[0140] A target prediction module, used to perform target prediction based on the motion information and the third target tracking result to obtain a target prediction result;

[0141] An adjustment information determination module, used to determine the posture adjustment information of the camera device based on the target prediction result;

[0142] A posture adjustment module is used to adjust the posture of the camera device based on the posture adjustment information.

[0143] Corresponding to a target tracking method described in the above embodiment, Figure 8 A structural diagram of an embodiment of a target tracking device applied to a server provided in an embodiment of the present application is shown.

[0144] In an embodiment of the present application, a target tracking device applied to a server may include:

[0145] The target video receiving module 801 is used to receive the target video sent by each camera device;

[0146] A target tracking module 802 is used to perform target tracking based on each of the target videos to obtain a second target tracking result;

[0147] The tracking result sending module 803 is used to send the second target tracking result to each of the camera devices.

[0148] In a specific implementation of the embodiment of the present application, the target tracking module includes:

[0149] The target detection submodule is used to perform target detection on each target video to obtain a corresponding target detection result;

[0150] A result association submodule, used to associate the results of each target detection result to obtain a global association result;

[0151] The target tracking submodule is used to perform target tracking based on the global association result to obtain the second target tracking result.

[0152] In a specific implementation of the embodiment of the present application, the device further includes:

[0153] A current scene determination module, used to determine a current scene corresponding to the target video based on each of the target detection results;

[0154] A target composition type determination module, used to determine a target composition type corresponding to the current scene based on a preset scene composition correspondence relationship; wherein the scene composition correspondence relationship is a correspondence relationship between a scene and a composition type;

[0155] A video composition module is used to compose each of the target videos based on the target composition type.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0157] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0158] Fig. 9 A schematic block diagram of an imaging device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0159] like Fig. 9 As shown, the camera device 9 of this embodiment includes: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and executable on the processor 90. When the processor 90 executes the computer program 92, the steps in the above-mentioned target tracking method embodiments are implemented, for example Figure 4 Alternatively, when the processor 90 executes the computer program 92, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 7 The functions of modules 701 to 705 are shown.

[0160] Exemplarily, the computer program 92 may be divided into one or more modules / units, which are stored in the memory 91 and executed by the processor 90 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 92 in the camera device 9.

[0161] Those skilled in the art will understand that Fig. 9 It is only an example of the camera device 9 and does not constitute a limitation of the camera device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the camera device 9 may also include input and output devices, network access devices, buses, etc.

[0162] The processor 90 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0163] The memory 91 may be an internal storage unit of the camera device 9, such as a hard disk or memory of the camera device 9. The memory 91 may also be an external storage device of the camera device 9, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the camera device 9. Further, the memory 91 may also include both an internal storage unit and an external storage device of the camera device 9. The memory 91 is used to store the computer program and other programs and data required by the camera device 9. The memory 91 may also be used to temporarily store data that has been output or is to be output.

[0164] FIG10 shows a schematic block diagram of a server provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0165] As shown in FIG10 , the server 10 of this embodiment includes: a processor 100, a memory 101, and a computer program 102 stored in the memory 101 and executable on the processor 100. When the processor 100 executes the computer program 102, the steps in the above-mentioned target tracking method embodiments are implemented, for example: Figure 5 Alternatively, when the processor 100 executes the computer program 102, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 8 The functions of modules 801 to 805 are shown.

[0166] Exemplarily, the computer program 102 may be divided into one or more modules / units, which are stored in the memory 101 and executed by the processor 100 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 102 in the server 10.

[0167] Those skilled in the art will appreciate that FIG. 10 is merely an example of a server 10 and does not constitute a limitation on the server 10 . The server 10 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the server 10 may also include input and output devices, network access devices, buses, etc.

[0168] The processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0169] The memory 101 may be an internal storage unit of the server 10, such as a hard disk or memory of the server 10. The memory 101 may also be an external storage device of the server 10, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server 10. Further, the memory 101 may also include both an internal storage unit of the server 10 and an external storage device. The memory 101 is used to store the computer program and other programs and data required by the server 10. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0170] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0171] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0172] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0173] In the embodiments provided in the present application, it should be understood that the disclosed devices / camera equipment / servers and methods can be implemented in other ways. For example, the device / camera equipment / server embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0174] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0176] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0177] The above are only optional embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A target tracking method, characterized in that: Applied to a camera device, the method comprises: Get the target video; Perform target tracking based on the target video to obtain a first target tracking result; Sending the target video to a server; Receiving a second target tracking result sent by the server; The first target tracking result is updated based on the second target tracking result and a preset update weight to obtain a third target tracking result.

2. The target tracking method according to claim 1, characterized in that: The performing target tracking based on the target video to obtain a first target tracking result includes: Performing target detection on the target video to obtain a target detection result; Target tracking is performed based on the target detection result to obtain the first target tracking result.

3. The target tracking method according to claim 1, characterized in that: Before obtaining the target video, the method further includes: Obtaining location information of the camera device; The updating of the first target tracking result based on the second target tracking result and a preset update weight to obtain a third target tracking result includes: Calculating a fourth target tracking result based on the position information and the second target tracking result; If the first target tracking result is different from the fourth target tracking result, the first target tracking result is updated based on the fourth target tracking result and the update weight to obtain the third target tracking result.

4. The target tracking method according to claim 1, characterized in that: Also includes: Receiving the motion information sent by the server; Based on the motion information and the third target tracking result, target prediction is performed to obtain a target prediction result; Based on the target prediction result, determining posture adjustment information of the camera device; Based on the posture adjustment information, the posture of the camera device is adjusted.

5. A target tracking method, characterized in that: Applied to a server, the method comprises: Receive target videos sent by various camera devices; Perform target tracking based on each of the target videos to obtain a second target tracking result; The second target tracking result is sent to each of the camera devices.

6. The target tracking method according to claim 5, characterized in that: The performing target tracking based on each of the target videos to obtain a second target tracking result includes: Performing target detection on each of the target videos to obtain corresponding target detection results; Associating the target detection results to obtain a global association result; Target tracking is performed based on the global association result to obtain the second target tracking result.

7. The target tracking method according to claim 6, characterized in that: After performing target detection on each of the target videos to obtain corresponding target detection results, the method further includes: Based on each of the target detection results, determining a current scene corresponding to the target video; Based on a preset scene-composition correspondence relationship, determining a target composition type corresponding to the current scene; wherein the scene-composition correspondence relationship is a correspondence relationship between a scene and a composition type; Composing each of the target videos based on the target composition type.

8. A target tracking device, characterized in that: Applied to a camera device, the device comprises: A target video acquisition module is used to acquire a target video; A target tracking module, used to perform target tracking based on the target video to obtain a first target tracking result; A target video sending module, used for sending the target video to a server; A tracking result receiving module, used for receiving the second target tracking result sent by the server; The tracking result updating module is used to update the first target tracking result based on the second target tracking result and a preset update weight to obtain a third target tracking result.

9. A target tracking device, characterized in that: Applied to a server, the device comprises: A target video receiving module is used to receive target videos sent by various camera devices; A target tracking module, used to perform target tracking based on each of the target videos to obtain a second target tracking result; The tracking result sending module is used to send the second target tracking result to each of the camera devices.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the target tracking method according to any one of claims 1 to 4 are implemented, or the steps of the target tracking method according to any one of claims 5 to 7 are implemented.

11. A camera device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the target tracking method according to any one of claims 1 to 4 are implemented.

12. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the target tracking method according to any one of claims 5 to 7 are implemented.

Images