Target tracking method, target tracking device, storage medium and electronic device

Abstract

The present disclosure provides a target tracking method, a target tracking device, a computer-readable storage medium and an electronic device, and relates to the field of computer vision technology. The target tracking method includes: obtaining a current frame in a target video; determining whether the current frame needs to be defogged based on the dark channel data of the current frame; if it is determined that the current frame needs to be defogged, determining the target object prediction area based on the target tracking result of the reference frame corresponding to the current frame; and defogging the target object prediction area in the current frame to obtain an image to be detected; if it is determined that the current frame does not need to be defogged, using the current frame as the image to be detected; and detecting the position of the target object in the image to be detected. The present disclosure improves the problem of low target tracking accuracy caused by fog occluding the image, and further improves the target tracking efficiency.

Description

Background Art

[0002] Target tracking is often used in video surveillance, human-computer interaction, robot navigation and other fields. By tracking the target, the target location information can be obtained in real time to provide related services.

[0003] However, when tracking a target in a high temperature, humid environment, etc., fog often blocks the target, making it difficult to accurately identify the target's position, resulting in reduced target tracking accuracy. Summary of the invention

[0004] The present disclosure provides a target tracking method, a target tracking device, a computer-readable storage medium and an electronic device, thereby improving the problem of low target tracking accuracy at least to a certain extent.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to a first aspect of the present disclosure, a target tracking method is provided, comprising: acquiring a current frame in a target video; determining whether the current frame requires defogging based on dark channel data of the current frame; if it is determined that the current frame requires defogging, determining a target object prediction area based on a target tracking result of a reference frame corresponding to the current frame, and defogging the target object prediction area in the current frame to obtain an image to be detected; if it is determined that the current frame does not require defogging, using the current frame as the image to be detected; and detecting the position of the target object in the image to be detected.

[0007] Optionally, the target tracking result of the reference frame includes a first target reference area and a second target reference area, the first target reference area is an area determined by detecting the position of the target object in the reference frame, and the second target reference area is an area determined by optimizing the first target reference area based on the historical position of the target object; determining the target object prediction area according to the target tracking result of the reference frame corresponding to the current frame includes: determining the target object prediction area based on the first target reference area and the second target reference area.

[0008] Optionally, the first target reference area and the second target reference area are both rectangular areas; determining the target object prediction area based on the first target reference area and the second target reference area includes: determining the target object prediction area based on the minimum value of the horizontal coordinate, the maximum value of the horizontal coordinate, the minimum value of the vertical coordinate, and the maximum value of the vertical coordinate in the first target reference area and the second target reference area.

[0009] Optionally, determining whether the current frame needs to be defogged according to the dark channel data of the current frame includes: if the dark channel data of the current frame is greater than a dark channel data threshold, determining that the current frame needs to be defogged.

[0010] Optionally, the dehazing process is performed on the target object prediction area to obtain the image to be detected, including: acquiring dark channel data of the target object prediction area; performing guided filtering optimization on the dark channel data to obtain first intermediate data; combining the dark channel data with the first intermediate data to obtain second intermediate data; and based on the first intermediate data and the second intermediate data, performing color correction on the target object prediction area in the current frame to obtain the image to be detected.

[0011] Optionally, the position of the target object in the current frame includes: a first target detection area of ​​the target object in the current frame; detecting the position of the target object in the image to be detected includes: inputting the image to be detected into a target detection network to obtain the first target detection area of ​​the image to be detected.

[0012] According to a second aspect of the present disclosure, a target tracking device is provided, comprising: a current frame acquisition module, configured to acquire a current frame in a target video; a defogging determination module, configured to determine whether the current frame requires defogging according to dark channel data of the current frame; a to-be-detected image acquisition module, configured to, if it is determined that the current frame requires defogging, determine a target object prediction area according to a target tracking result of a reference frame corresponding to the current frame; and perform defogging on the target object prediction area in the current frame to obtain an image to be detected; if it is determined that the current frame does not require defogging, use the current frame as the image to be detected; and a position detection module, configured to detect the position of the target object in the image to be detected.

[0013] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the target tracking method of the first aspect and possible implementation methods thereof are implemented.

[0014] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the target tracking method of the first aspect and possible implementation thereof by executing the executable instructions.

[0015] The technical solution disclosed in this disclosure has the following beneficial effects:

[0016] On the one hand, in the target tracking method, it is judged whether the current frame needs to be defogged according to the dark channel data of the current frame, instead of directly defogging all frames, which is conducive to accelerating the target tracking process and ensuring the real-time performance of the target tracking method; on the other hand, if it is determined that the current frame needs to be defogged, the target object prediction area is determined according to the target tracking result of the reference frame corresponding to the current frame; and the target object prediction area in the current frame is defogged. If the current frame needs to be defogged, only the target object prediction area is defogged, which reduces the defogging area and improves the defogging accuracy; on the other hand, when the current frame needs to be defogged, the current frame after defogging is used as the image to be detected; when the current frame does not need to be defogged, the current frame is directly used as the image to be detected, and the position of the target object is detected in the image to be detected. Defogging is added to the target tracking algorithm to improve the problem of low target tracking accuracy caused by fog blocking the image to be detected, and further improve the target tracking efficiency.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0019] Figure 1 A schematic diagram showing a process of a target tracking method;

[0020] Figure 2 The system architecture showing the operating environment of the exemplary embodiment;

[0021] Figure 3 A schematic flow chart of a target tracking method in this exemplary embodiment is shown;

[0022] Figure 4 A schematic diagram showing a first target reference area and a second target reference area in this exemplary embodiment;

[0023] Figure 5 A schematic diagram showing a target object prediction region in a current frame in this exemplary embodiment;

[0024] Figure 6 A schematic diagram showing another target object prediction region in the current frame in this exemplary embodiment;

[0025] Figure 7 A schematic diagram showing a process of acquiring an image to be detected in this exemplary embodiment;

[0026] Figure 8 A flow chart showing a target tracking method in this exemplary embodiment is shown;

[0027] Fig. 9 A schematic diagram showing the structure of a target tracking device in this exemplary embodiment is shown;

[0028] Fig.10 A schematic structural diagram of an electronic device in this exemplary embodiment is shown. DETAILED DESCRIPTION

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0030] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0031] In related technologies, such as Figure 1 As shown, target tracking is usually performed directly on the acquired original image to obtain the position of the target object in the current frame, thereby determining the target detection area of ​​the current frame. However, due to different application environments, the original image is often blocked by fog or water vapor, which increases the difficulty of target tracking in the original image and reduces the target tracking accuracy.

[0032] In view of one or more of the above problems, the exemplary embodiment of the present disclosure first provides a target tracking method. Figure 2The system architecture of the operating environment of this exemplary embodiment will be described.

[0033] refer to Figure 2 As shown, the system architecture 200 may include a terminal device 210 and a server 220. The terminal device 210 may be an electronic device such as a smart phone, a tablet computer, a camera, etc., and the terminal device 210 may be used to obtain the current frame in the target video. The server 220 generally refers to a background system that provides target tracking related services in this exemplary embodiment, such as a server that implements the target tracking method. The server 220 may be a cluster formed by one server or multiple servers, which is not limited in this disclosure. The terminal device 210 and the server 220 may be connected via a wired or wireless communication link to exchange data.

[0034] In one embodiment, the target tracking method in this exemplary embodiment may be executed by the terminal device 210. For example, in a steel coil production scenario, the terminal device 210 may be a computer on a production line, and the target video may be a monitoring or detection video of a steel coil product. The terminal device 210 determines the position of the steel coil tail in the current frame of the target video by executing the target tracking method to track the steel coil tail.

[0035] In one embodiment, the terminal device 210 can obtain the target video and send the target video to the server 220. After receiving the target video sent by the terminal device 210, the server 220 obtains the current frame in the target video, and then determines whether the current frame needs to be defogged based on the dark channel data of the current frame; if it is determined that the current frame needs to be defogged, the target object prediction area is determined based on the target tracking result of the reference frame corresponding to the current frame; and the target object prediction area in the current frame is defogged to obtain the image to be detected; if it is determined that the current frame does not need to be defogged, the current frame is used as the image to be detected; finally, the position of the target object is detected in the image to be detected.

[0036] As can be seen from the above, the target tracking method in this exemplary embodiment can be executed by the terminal device 210 or the server 220 mentioned above.

[0037] Combine the following Figure 3 The target tracking method is described. Figure 3 An exemplary process of the target tracking method is shown, including the following steps S310 to S350:

[0038] Step S310, obtaining the current frame in the target video;

[0039] Step S320, determining whether the current frame needs defogging processing according to the dark channel data of the current frame;

[0040] Step S330: if it is determined that the current frame needs to be defogged, a target object prediction region is determined according to the target tracking result of the reference frame corresponding to the current frame; and the target object prediction region in the current frame is defogged to obtain an image to be detected;

[0041] Step S340: if it is determined that the current frame does not need defogging, the current frame is used as the image to be detected;

[0042] Step S350: Detect the position of the target object in the image to be detected.

[0043] Based on the above method, on the one hand, in the target tracking method, it is determined whether the current frame needs to be defogged according to the dark channel data of the current frame, instead of directly defogging all frames, which is conducive to accelerating the target tracking process and ensuring the real-time performance of the target tracking method; on the other hand, if it is determined that the current frame needs to be defogged, the target object prediction area is determined according to the target tracking result of the reference frame corresponding to the current frame; and the target object prediction area in the current frame is defogged. If the current frame needs to be defogged, only the target object prediction area is defogged, which reduces the defogging area and improves the defogging accuracy; on the other hand, when the current frame needs to be defogged, the current frame after defogging is used as the image to be detected; when the current frame does not need to be defogged, the current frame is directly used as the image to be detected, and the position of the target object is detected in the image to be detected. Defogging is added to the target tracking algorithm to improve the problem of low target tracking accuracy caused by fog blocking the image to be detected, and further improve the target tracking efficiency.

[0044] Below Figure 3 Each step in the procedure is described in detail.

[0045] refer to Figure 3 , in step S310, the current frame in the target video is obtained.

[0046] Among them, the target video is a video that needs to be tracked. The present disclosure does not limit the method for obtaining the target video. For example, the target video can be captured by a camera and uploaded to the terminal device 110 for target tracking, or the target tracking method can be directly integrated into the camera; the target video can also be a local video, and the local video can be uploaded to the terminal device 110 or the server 120 for the target tracking method. The above-mentioned current frame can be a captured frame in the target video. The present disclosure does not limit the method for obtaining the current frame. For example, the current frame can be extracted from the target video through OpenCV code. OpenCV is an open source cross-platform computer vision and machine learning software library that can run on Linux, Windows, Android and Mac OS operating systems, and can implement many general algorithms in image processing and computer vision.

[0047] Continue to refer Figure 3 , in step S320, determining whether the current frame needs defogging processing according to the dark channel data of the current frame;

[0048] Among them, the dark channel data is the basis for judging whether the current frame is obscured by fog. The present disclosure does not limit the specific content of the dark channel data. For example, the dark channel data may include the dark channel mean of the current frame, or the dark channel mean of the area where the target object is located in the current frame; the target object may be an object to be tracked. The present disclosure does not make any special limitation on the target object. For example, the target object may be a steel coil tail, an animal, a person, etc.

[0049] In one implementation, the above-mentioned determining whether the current frame needs defogging processing according to the dark channel data of the current frame may include the following steps:

[0050] If the dark channel data of the current frame is greater than the dark channel data threshold, it is determined that the current frame needs to be defogged.

[0051] The dark channel data threshold is used to measure whether the value of the dark channel data is too large, and can be determined based on experience or specific circumstances. The dark channel data can be compared with the dark channel data threshold. If the dark channel data is greater than the dark channel data threshold, it means that there is a high probability of fog occlusion in the current frame, thereby determining that the current frame needs to be defogged. The dark channel data threshold can correspond to the dark channel data. For example, if the dark channel data is the dark channel mean, the dark channel data threshold can be the dark channel mean threshold.

[0052] In one embodiment, the dark channel mean of the current frame can be used as the dark channel data. Since the dark channel of an image not obscured by fog is similar to a completely black image, that is, the dark channel mean is equal to 0, the higher the dark channel mean, the greater the probability that the detected image is obscured by fog. Exemplarily, the current frame can be first read using the imread() function of OpenCV to obtain the pixel value and channel index of the current frame; then the dark channel mean of the current frame is obtained by combining the pixel value and channel index of the current frame and the np.mean() function; according to the comparison result of the dark channel mean and the dark channel threshold, it is determined whether the current frame needs to be defogged. If the dark channel mean of the current frame is greater than the dark channel threshold, it is determined that the current frame is obscured by fog and needs to be defogged; if the dark channel mean of the current frame is less than or equal to the dark channel threshold, it is determined that the current frame is not obscured by fog and does not need to be defogged.

[0053] The above-mentioned dark channel data-based judgment on whether to perform defogging on the current frame does not require defogging on each frame of video, which is beneficial to improving the overall speed of the target tracking method and ensuring the real-time performance of the target tracking method.

[0054] Continue to refer Figure 3 In step S330, if it is determined that the current frame needs to be defogged, a target object prediction region is determined according to a target tracking result of a reference frame corresponding to the current frame; and the target object prediction region in the current frame is defogged to obtain an image to be detected;

[0055] The reference frame is used to provide reference information for target tracking of the current frame, and may be a video frame other than the current frame in the target video. For example, the reference frame may be a frame before the current frame in the target video. The target object prediction region may include a local region or the entire region of the current frame. The present disclosure does not specifically limit the size and shape of the target object prediction region. For example, the target object prediction region may be a rectangular region predicted based on the target tracking result of the reference frame, and the target object may be included in the rectangular region.

[0056] In one embodiment, the target tracking result of the reference frame includes a first target reference area and a second target reference area. The first target reference area is an area determined by detecting the position of the target object in the reference frame. For example, the reference frame can be processed by a target detection network to detect the position of the target object. The first target reference area can be determined based on the contour of the target object, such as the first target reference area can be a circumscribed rectangular area of ​​the contour of the target object. The second target reference area is an area determined by optimizing the first target reference area based on the historical position of the target object. Among them, the historical position of the target object refers to the position of the target object in a video frame earlier than the reference frame. Based on the historical position of the target object, the position of the target object in the reference frame can be predicted. Since there may be an observation error in the first target reference area, the first target reference area can be optimized based on the predicted position of the target object to obtain the second target reference area. For example, a Kalman filter can be used to calculate the state transfer equation of the target object based on its historical position, calculate the predicted position of the target object in the reference frame according to the state transfer equation, and optimize the position of the target object in combination with the predicted position and the first target reference area to obtain the second target reference area.

[0057] The above-mentioned determining the target object prediction area according to the target tracking result of the reference frame corresponding to the current frame may include the following steps:

[0058] A target object prediction region is determined based on the first target reference region and the second target reference region.

[0059] Since the larger the defogging image is, the less obvious the defogging effect may be. Therefore, in one embodiment, the reference frame of the current frame may include the previous frame of the current frame, and the first target reference area may be an area determined according to the observed position of the target object in the reference frame; the second target reference area may be an area determined by optimizing the first target reference area using Kalman filtering, and the target object prediction area may be determined based on the coordinates of the first target reference area and the second target reference area to ensure that the target object is in the target object prediction area; and the previous frame of the current frame in the target video may be less different from the position of the target object in the current frame. Therefore, the previous video frame of the current frame is used as a reference frame, and the area where the target may appear in the current frame is predicted according to the target tracking result of the reference frame. The interference of fog flow can be eliminated, thereby reducing the positioning error of the target object prediction area and improving the positioning accuracy of the target object prediction area.

[0060] In one implementation, the first target reference area and the second target reference area may both be rectangular areas; and determining the target object prediction area based on the first target reference area and the second target reference area may include the following steps:

[0061] A target object prediction region is determined based on a minimum value of the abscissa, a maximum value of the abscissa, a minimum value of the ordinate, and a maximum value of the ordinate in the first target reference region and the second target reference region.

[0062] like Figure 4 As shown, it can be based on Figure 4 The coordinates of the first target reference area and the second target reference area in the horizontal coordinate minimum value x min , the maximum value of the horizontal axis x max , minimum value of y coordinate min , maximum value of ordinate y max ; Then according to x min 、x max ,y min ,y max The enclosed area determines the target object prediction area.

[0063] In this exemplary embodiment, a method for determining a target object prediction area selects the minimum value of the horizontal coordinate, the maximum value of the horizontal coordinate, the minimum value of the vertical coordinate, and the maximum value of the vertical coordinate in the first target reference area and the second target reference area to determine the target object prediction area. The calculation method is relatively simple, which reduces the positioning complexity of the target object prediction area, reduces the image area that needs to be defogged, improves the defogging effect, and improves the defogging processing efficiency.

[0064] In one embodiment, reference Figure 5As shown, the upper left corner coordinates and the lower right corner coordinates of the first target reference area in the reference frame can be selected, which are recorded as (x1, y1) and (x2, y2), and the upper left corner coordinates and the lower right corner coordinates of the second target reference area can be selected, which are recorded as (x3, y3) and (x4, y4); the minimum value x of the horizontal coordinates in the first target reference area and the second target reference area is obtained. min , the maximum value of the horizontal axis x max , minimum value of y coordinate min , maximum value of ordinate y max :

[0065] x min =min(x1,x2,x3,x4);

[0066] x max =max(x1,x2,x3,x4);

[0067] y min =min(y1,y2,y3,y4);

[0068] y max =max(y1,y2,y3,y4);

[0069] Finally, combined with OpenCV code, according to x in the current frame img min 、x max ,y min ,y max Cut the target object prediction area im_cut, that is Figure 5 The shaded area:

[0070] im_cut=img[x min :x max ,y min :y max ,:].

[0071] In one embodiment, Figure 6 As shown, the target object prediction area can also be determined according to the intersection of the first target reference area and the second target reference area; illustratively, the upper left corner coordinates and the lower right corner coordinates of the first target reference area in the reference frame are recorded as (x1, y1) and (x2, y2), and the upper left corner coordinates and the lower right corner coordinates of the second target reference area are selected, which are recorded as (x3, y3) and (x4, y4); then the minimum value of x1 and x3 can be selected as x min , select the minimum value of x2 and x4 as x max , select the minimum value of y1 and y3 as y min , select the minimum value of y2 and y4 as y max , and then according to x min 、xmax ,y min ,y max The area enclosed is Figure 6 The shaded area in determines the target object prediction area.

[0072] After determining the target object prediction area, the target object prediction area can be defogged to obtain the image to be detected. Figure 7 As shown, the above-mentioned defogging process is performed on the target object prediction area in the current frame to obtain the image to be detected, which may include the following steps S710 to S740:

[0073] Step S710, obtaining dark channel data of the target object prediction area;

[0074] Step S720, performing guided filtering optimization on the dark channel data to obtain first intermediate data;

[0075] Step S730, combining the dark channel data with the first intermediate data to obtain second intermediate data;

[0076] Step S740 , based on the first intermediate data and the second intermediate data, color correction is performed on the target object prediction area in the current frame to obtain an image to be detected.

[0077] The first intermediate data may be an atmospheric mask image obtained by performing guided filtering optimization on dark channel data of the target object prediction area; the second intermediate data may include an illumination value obtained by calculating the dark channel data and the first intermediate data.

[0078] In this exemplary embodiment, in step S710, dark channel data of the target object prediction area is first obtained; in step S720, guided filtering optimization is performed on the dark channel data to obtain first intermediate data; in step S730, the dark channel data and the first intermediate data are operated on to obtain second intermediate data; in step S740, based on the first intermediate data and the second intermediate data, color correction is performed on the target object prediction area in the current frame, and the target object prediction area in the current frame is replaced with the color-corrected target object prediction area to obtain an image to be detected.

[0079] In one implementation, the OpenCV code can be used to perform defogging on the target object prediction area. First, the dark channel data D1 of the target object prediction area can be obtained:

[0080] D1=np.min(m,2), m=img[x_min:x_max,y_min:y_max,:];

[0081] Then, the dark channel data D1 is optimized by guided filtering to obtain the atmospheric mask image V1 corresponding to the target object prediction area:

[0082] V1=guidedfilter(D1,zmMinFilterGray(D1,7));

[0083] The dark channel data is then combined with the atmospheric mask image V1 corresponding to the target object prediction area to obtain the illumination value A of the target object prediction area:

[0084] The number of bins in the histogram can be set to 2000;

[0085] Then perform histogram statistics on the atmospheric mask image V1: ht = np.histogram (V1, bins);

[0086] The atmospheric mask image V1 is calculated to obtain the illumination value A:

[0087] A=np.mean(m,2)[V1>=ht[1][lmax]].max();

[0088] Finally, based on the atmospheric mask image V1 corresponding to the target object prediction region and the illumination value A of the target object prediction region, the target object prediction region in the current frame can be color corrected to obtain the color-corrected target object prediction region m:

[0089] m[:,:,:]=(m[:,:,:]-V1) / (1-V1 / A)

[0090] The target object prediction area in the current frame can be replaced by m to obtain the image to be detected.

[0091] In one embodiment, the previous frame of the current frame can be used as a reference frame of the current frame, and the current frame can be any frame in the target video except the first frame. For the first frame, it is impossible to obtain a reference frame, and it is impossible to determine the target object prediction area in the first frame. If it is determined that the first frame needs to be defogged, the entire image of the first frame can be defogged, and the defogged image is used as the image to be detected to detect the position of the target object. Then, the first frame can be used as a reference frame for the second frame, and the target object prediction area in the second frame is determined according to the position of the target object in the first frame. Then, when it is determined that the second frame needs to be defogged, the target object prediction area in the second frame can be defogged.

[0092] Compared with the method of directly defogging the entire current frame, the above method only defogs the target object prediction area in the current frame, and retains the original pixel values ​​in the remaining areas to obtain the image to be detected. The calculation amount is smaller, and the local defogging effect is better than the whole image defogging, which improves the efficiency of the defogging process.

[0093] Continue to refer Figure 3 In step S340, if it is determined that the current frame does not require defogging, the current frame can be used as the image to be detected.

[0094] After obtaining the image to be detected, continue to refer to Figure 3 In step S350, the position of the target object can be detected in the image to be detected.

[0095] Among them, the image to be detected is the current frame obtained after defogging processing, or the current frame that does not need to be defogged, which can ensure that the image to be detected is the current frame that is not obscured by fog. Therefore, when detecting the position of the target object in the image to be detected, the interference of fog obstruction on the detection of the target object can be eliminated, thereby improving the accuracy of target object position detection.

[0096] In one implementation, the position of the target object in the current frame may include: a first target detection area of ​​the target object in the current frame; and detecting the position of the target object in the image to be detected may include the following steps:

[0097] The image to be detected is input into the target detection network to obtain the first target detection area of ​​the image to be detected.

[0098] Among them, the target detection network may include a neural network model for target detection. The present disclosure does not specifically limit the specific composition of the target detection network; for example, the target detection network may include YOLO (You Only LookOnce, a target detection algorithm model), SSD (Single Shot MultiBox Detector), and other neural network models that can perform target detection.

[0099] The first target detection area may be an area determined based on the outline of the target object in the current frame. For example, the first target detection area may include a rectangular area circumscribed by the outline of the target object. In this exemplary embodiment, the image to be detected may be input into a target detection network, and the target detection network may be used to process the image to be detected to detect the position of the target object in the current frame, thereby determining the first target detection area in the image to be detected.

[0100] The above method of inputting the image to be detected into the target detection network to determine the first target detection area can obtain the position information of the target object in the current frame through the target detection network. The higher the accuracy of the target detection network, the more accurate the position information of the target object in the current frame obtained, and the more accurate the positioning of the first target detection area.

[0101] In one implementation, the position of the target object in the current frame may further include: a second target detection area of ​​the target object in the current frame; and the detecting the position of the target object in the image to be detected may further include the following steps:

[0102] The position of the first target detection area is optimized by Kalman filtering to obtain a second target detection area of ​​the image to be detected.

[0103] Among them, Kalman filtering is an algorithm that uses the linear system state equation to filter out some of the noise and interference of the input observation data on the system to obtain output observation data. Kalman filtering can achieve the optimal estimation of the system state.

[0104] The second target detection area can be an area in the image to be detected that is determined by optimizing the position of the first target detection area based on the Kalman filter in combination with the historical position of the target object. Since the detection accuracy of the target detection network varies, there may be observation errors in the first target detection area. In this exemplary embodiment, the state transfer equation is calculated based on the historical position of the target object based on the Kalman filter, and the predicted position of the target object in the reference frame is calculated based on the state transfer equation. The position of the target object is then optimized in combination with the predicted position and the first target detection area to obtain the second target detection area; the problem of different positioning accuracy of the first target detection area caused by the accuracy difference of the target detection network is improved, the accuracy of locating the target object in the image to be detected is improved, the reliability of the target tracking method is further improved, and the user experience is improved.

[0105] Based on the above target tracking method, defogging judgment is added in the target tracking process, which improves the problem of low accuracy of the target tracking method caused by fog blocking the target video frame, improves the target tracking accuracy and speed, and further improves the target tracking efficiency.

[0106] In one embodiment, an exemplary process of the target tracking method disclosed in the present invention is as follows: Figure 5 As shown, target tracking can be performed according to steps S801 to S811.

[0107] Step S801, obtaining the current frame in the target video;

[0108] Step S802, calculating dark channel data of the current frame;

[0109] Step S803, determine whether the dark channel data is greater than the dark channel data threshold; if so, jump to step S804, otherwise, jump to step S809;

[0110] Step S804, determining a target object prediction region based on a first target reference region and a second target reference region of a reference frame corresponding to the current frame;

[0111] Step S805, obtaining dark channel data of the target object prediction area;

[0112] Step S806, performing guided filtering optimization on the dark channel data to obtain first intermediate data;

[0113] Step S807, combining the dark channel data with the first image data to obtain second intermediate data;

[0114] Step S808, performing color correction on the target object prediction area in the current frame based on the first intermediate data and the second intermediate data to obtain an image to be detected;

[0115] Step S809, taking the current frame as the image to be detected;

[0116] Step S810, inputting the image to be detected into the target detection network to obtain a first target detection area of ​​the image to be detected;

[0117] Step S811, optimizing the position of the first target detection area by Kalman filtering to obtain a second target detection area of ​​the image to be detected.

[0118] In one embodiment, the target tracking method of the present disclosure may use the tail of a steel coil as a target object, and may use a monitoring or detection video of the steel coil product as a target video to perform target tracking.

[0119] In this exemplary embodiment, first, a current frame in the target video may be obtained, and then a dark channel mean value of the current frame may be obtained; wherein the current frame may be any frame in the target video except the first frame;

[0120] (1) If the dark channel data threshold is 0.8 and the dark channel mean of the current frame is 0.4, the dark channel mean of the current frame is less than the dark channel data threshold. At this time, there is no need to perform defogging on the current frame, and the current frame is determined as the image to be detected;

[0121] (2) If the dark channel data threshold is 0.8 and the dark channel mean of the current frame is 0.88, the dark channel mean of the current frame is greater than the dark channel data threshold, and the current frame needs to be defogged. The previous frame of the current frame in the target video can be used as a reference frame, and the target object prediction area is determined according to the maximum and minimum values ​​of the horizontal and vertical coordinates of the first target reference area and the second target reference area of ​​the reference frame:

[0122] First, the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2) of the first target reference area in the reference frame, and the upper left corner coordinates (x_kalman1, y_kalman1) and the lower right corner coordinates (x_kalman2, y_kalman2) of the second target reference area in the reference frame can be obtained;

[0123] Select the minimum and maximum values ​​of the horizontal coordinates, and the minimum and maximum values ​​of the vertical coordinates in (x1, y1), (x2, y2), (x_kalman1, y_kalman1), and (x_kalman2, y_kalman2). For example,

[0124] x_min=min(x1,x2,x_kalman1,x_kalman2)=540;

[0125] x_min=max(x1,x2,x_kalman1,x_kalman2)=597;

[0126] y_min=min(y1,y2,y_kalman1,y_kalman2)=165;

[0127] y_max=max(y1,y2,y_kalman1,y_kalman2)=261;

[0128] According to x_min, x_max, y_min, y_max, intercept the target object prediction area im_cut in the current frame:

[0129] im_cut=img2[540:597,165:261,:];

[0130] After obtaining the target object prediction area im_cut, the im_cut can be defogged. First, the dark channel mean of im_cut can be calculated, and then the atmospheric mask image corresponding to im_cut and the illumination value of im_cut can be calculated by using guided filtering optimization; the im_cut can be color corrected according to the atmospheric mask image and the illumination value; finally, the im_cut in the current frame can be replaced with the color-corrected im_cut to obtain the image to be detected;

[0131] After obtaining the image to be detected, the image to be detected can be input into the target detection network to obtain the first target detection area of ​​the image to be detected; then the position of the first target detection area is optimized through Kalman filtering to obtain the second target detection area of ​​the image to be detected; after obtaining the first target detection area and the second target detection area, the first target detection area and the second target detection area can be used as the target tracking results of the image to be detected.

[0132] In addition, in this exemplary embodiment, if the reference frame is determined to be the previous frame of the current frame, and the current frame is the first frame of the target video, the reference frame cannot be obtained, and thus the target object prediction area cannot be determined in the first frame; at this time, if the dark channel data threshold is 0.8 and the dark channel mean of the first frame is 0.9, it can be determined that the first frame needs to be defogged, so that the entire image of the first frame can be defogged, and the defogged image is used as the image to be detected to detect the position of the target object; when tracking the target in the second frame of the target video, the first frame can be used as a reference frame of the second frame, and the target object prediction area in the second frame is determined according to the position of the target object in the first frame. If the second frame needs to be defogged, the target object prediction area in the second frame can be defogged to obtain the image to be detected corresponding to the second frame, thereby detecting the position of the target object in the image to be detected.

[0133] The exemplary embodiment of the present disclosure also provides a target tracking device. Fig. 9 As shown, the target tracking device 900 may include:

[0134] The current frame acquisition module 910 is configured to acquire the current frame in the target video;

[0135] A defogging determination module 920 is configured to determine whether the current frame needs defogging according to the dark channel data of the current frame;

[0136] The image acquisition module 930 to be detected is configured to determine the target object prediction area according to the target tracking result of the reference frame corresponding to the current frame if it is determined that the current frame needs to be defogged; and perform defogging on the target object prediction area in the current frame to obtain the image to be detected; if it is determined that the current frame does not need to be defogged, the current frame is used as the image to be detected;

[0137] The position detection module 940 is configured to detect the position of the target object in the image to be detected.

[0138] In one embodiment, the target tracking result of the reference frame includes a first target reference area and a second target reference area, the first target reference area is an area determined by detecting the position of the target object in the reference frame, and the second target reference area is an area determined by optimizing the first target reference area based on the historical position of the target object; the target object prediction area determined according to the target tracking result of the reference frame corresponding to the current frame may include:

[0139] A target object prediction region is determined based on the first target reference region and the second target reference region.

[0140] In one implementation, the first target reference area and the second target reference area are both rectangular areas; and determining the target object prediction area based on the first target reference area and the second target reference area may include:

[0141] A target object prediction region is determined based on a minimum value of the abscissa, a maximum value of the abscissa, a minimum value of the ordinate, and a maximum value of the ordinate in the first target reference region and the second target reference region.

[0142] In one implementation, determining whether the current frame needs defogging according to the dark channel data of the current frame may include:

[0143] If the dark channel data of the current frame is greater than the dark channel data threshold, it is determined that the current frame needs to be defogged.

[0144] In one implementation, the above-mentioned defogging of the target object prediction area to obtain the image to be detected may include:

[0145] Obtain dark channel data of the target object prediction area;

[0146] Performing guided filtering optimization on the dark channel data to obtain first intermediate data;

[0147] combining the dark channel data with the first intermediate data to obtain second intermediate data;

[0148] Based on the first intermediate data and the second intermediate data, color correction is performed on the target object prediction area in the current frame to obtain an image to be detected.

[0149] In one embodiment, the position of the target object in the current frame includes: a first target detection area of ​​the target object in the current frame; the position of the target object detected in the image to be detected may include:

[0150] The image to be detected is input into the target detection network to obtain the first target detection area of ​​the image to be detected.

[0151] In one implementation, the position of the target object in the current frame may further include: a second target detection area of ​​the target object in the current frame; and the position of the target object detected in the image to be detected may further include:

[0152] The position of the first target detection area is optimized by Kalman filtering to obtain a second target detection area of ​​the image to be detected.

[0153] The specific details of each part of the above device have been described in detail in the implementation method part, so they will not be repeated here.

[0154] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium, which can be implemented in the form of a program product, which includes a program code, and when the program product is run on an electronic device, the program code is used to cause the electronic device to perform the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present disclosure. In an optional embodiment, the program product can be implemented as a portable compact disk read-only memory (CD-ROM) and includes program code, and can be run on an electronic device, such as a personal computer. However, the program product of the present disclosure is not limited to this, and in this document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or device.

[0155] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0156] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0157] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0158] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0159] The exemplary embodiment of the present disclosure also provides an electronic device. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as program codes. The processor executes the method in the exemplary embodiment by executing the executable instructions.

[0160] Reference below Fig.10 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Fig.10 The electronic device 1000 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0161] like Fig.10 As shown, the electronic device 1000 may include: a processor 1010 , a memory 1020 , a bus 1030 , an I / O (input / output) interface 1040 , and a network adapter 1050 .

[0162] The processor 1010 may include one or more processing units, for example: the processor 1010 may include a central processing unit (CPU), an AP (Application Processor), a modem processor, a display processor (DPU), a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor, an artificial intelligence processor, etc. In one embodiment, the artificial intelligence processor may obtain a current frame in a target video; determine whether the current frame needs to be defogged according to the dark channel data of the current frame; if it is determined that the current frame needs to be defogged, determine the target object prediction area according to the target tracking result of the reference frame corresponding to the current frame; and perform defogging on the target object prediction area in the current frame to obtain an image to be detected; if it is determined that the current frame does not need to be defogged, use the current frame as the image to be detected; and detect the position of the target object in the image to be detected.

[0163] The memory 1020 may include a volatile memory, such as a RAM 1021, a cache unit 1022, and may also include a non-volatile memory, such as a ROM 1023. The memory 1020 may also include one or more program modules 1024, such program modules 1024 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof may include the implementation of a network environment. For example, the program module 1024 may include each module in the above-mentioned apparatus 1000.

[0164] The bus 1030 is used to realize the connection between different components of the electronic device 1000, and may include a data bus, an address bus, and a control bus.

[0165] The electronic device 1000 can communicate with one or more external devices 1100 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 1040 .

[0166] The electronic device 1000 can communicate with one or more networks through the network adapter 1050. For example, the network adapter 1050 can provide mobile communication solutions such as 3G / 4G / 5G, or provide wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 1050 can communicate with other modules of the electronic device 1000 through the bus 1030.

[0167] although Fig.10 Not shown, other hardware and / or software modules may also be provided in the electronic device 1000, including but not limited to: a display, a microcode, a device driver, a redundant processor, an external disk drive array, a RAID system, a tape drive, and a data backup storage system.

[0168] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0169] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or implementations combining hardware and software aspects, which may be collectively referred to herein as "circuit", "module" or "system". Those skilled in the art will readily think of other implementations of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and implementation are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0170] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A target tracking method, characterized in that: include: Get the current frame in the target video; Determining whether the current frame needs defogging processing according to the dark channel data of the current frame; If it is determined that the current frame needs to be defogged, a target object prediction region is determined according to a target tracking result of a reference frame corresponding to the current frame, and defogging is performed on the target object prediction region in the current frame to obtain an image to be detected; If it is determined that the current frame does not need defogging, the current frame is used as the image to be detected; Detecting the position of a target object in the image to be detected; The target tracking result of the reference frame includes a first target reference area and a second target reference area, the first target reference area is an area determined by detecting the position of the target object in the reference frame, and the second target reference area is an area determined by optimizing the first target reference area based on the historical position of the target object; The step of determining the target object prediction area according to the target tracking result of the reference frame corresponding to the current frame includes: The target object prediction region is determined based on the first target reference region and the second target reference region.

2. The method according to claim 1, characterized in that: The first target reference area and the second target reference area are both rectangular areas; and determining the target object prediction area based on the first target reference area and the second target reference area includes: The target object prediction region is determined based on the minimum value of the abscissa, the maximum value of the abscissa, the minimum value of the ordinate, and the maximum value of the ordinate in the first target reference region and the second target reference region.

3. The method according to claim 1, characterized in that The determining, according to the dark channel data of the current frame, whether the current frame needs defogging processing comprises: If the dark channel data of the current frame is greater than the dark channel data threshold, it is determined that the current frame needs to be defogged.

4. The method according to claim 1, characterized in that: The defogging process is performed on the target object prediction area to obtain the image to be detected, including: Acquire dark channel data of the target object prediction area; Performing guided filtering optimization on the dark channel data to obtain first intermediate data; combining the dark channel data with the first intermediate data to obtain second intermediate data; Based on the first intermediate data and the second intermediate data, color correction is performed on the target object prediction area in the current frame to obtain the image to be detected.

5. The method according to claim 1, characterized in that The position of the target object in the current frame includes: a first target detection area of ​​the target object in the current frame; the position of the target object to be detected in the image to be detected includes: The image to be detected is input into an object detection network to obtain a first object detection area of ​​the image to be detected.

6. The method according to claim 5, characterized in that The position of the target object in the current frame further includes: a second target detection area of ​​the target object in the current frame; the position of the target object detected in the image to be detected further includes: The position of the first target detection area is optimized by Kalman filtering to obtain a second target detection area of ​​the image to be detected.

7. A target tracking device, characterized in that: include: A current frame acquisition module is configured to acquire a current frame in a target video; a defogging determination module, configured to determine whether the current frame needs defogging according to the dark channel data of the current frame; The image acquisition module to be detected is configured to determine the target object prediction area according to the target tracking result of the reference frame corresponding to the current frame if it is determined that the current frame needs to be defogged, and perform defogging on the target object prediction area in the current frame to obtain the image to be detected; if it is determined that the current frame does not need to be defogged, use the current frame as the image to be detected; A position detection module, configured to detect the position of a target object in the image to be detected; The target tracking result of the reference frame includes a first target reference area and a second target reference area, the first target reference area is an area determined by detecting the position of the target object in the reference frame, and the second target reference area is an area determined by optimizing the first target reference area based on the historical position of the target object; The step of determining the target object prediction area according to the target tracking result of the reference frame corresponding to the current frame includes: The target object prediction region is determined based on the first target reference region and the second target reference region.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 6 by executing the executable instructions.

Images