Target detection method and device, terminal equipment and storage medium
Through the background differential and density clustering method, the problem of the reduction in object detection accuracy when the camera lens is contaminated by dust is solved, and a higher detection accuracy and success rate are achieved.
Patent Information
- Application Number
- CN202311747410.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-19
- Publication Date
- 2025-06-27
AI Technical Summary
The existing object detection method is difficult to accurately detect targets in the image when the camera lens is contaminated by dust, resulting in a decrease in detection accuracy and success rate.
Through a method based on background difference processing, the image to be detected is subjected to background difference to obtain the differential image, and then density clustering algorithm is used to perform density clustering processing, and the enclosure box is constructed and its area is calculated. If the area is within the target range and the cluster center position meets the conditions, the enclosure box is determined to be the target detection result.
The detection accuracy and success rate of target detection are improved, especially when the camera lens is heavily contaminated, the target can be detected more accurately.
Smart Images
Figure CN120219698A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the technical field of automobiles, and particularly relates to a target detection method, device, terminal device and storage medium. Background Art
[0002] Currently, due to dust pollution, the surface of the camera lens will be blurred and soiled, which will affect the quality of the video image captured by the camera, thereby reducing the accuracy of target detection based on the video image.
[0003] Existing target detection methods usually perform denoising processing on the video image after obtaining it, or use data augmentation algorithms to perform image enhancement processing on the video image, and then perform target detection on the processed image. However, the above methods can only handle the impact on the image quality caused by some relatively minor pollution, such as raindrops, snowflakes or dust particles. However, in some scenarios such as closed tunnels, vehicle exhaust contains a large amount of oil and dust, which accumulates on the camera lens. Even after denoising processing or image enhancement processing on the captured video image, it is difficult to detect whether there is a vehicle in the processed image, thereby reducing the detection accuracy and success rate of target detection. Summary of the Invention
[0004] The embodiments of this application provide a target detection method, device, terminal device and storage medium, which improve the detection accuracy and success rate of target detection.
[0005] In a first aspect, the embodiments of this application provide a target detection method, including:
[0006] Performing background difference processing on the image to be detected based on the first background image to obtain a difference image; wherein, the first background image and the image to be detected are images of the same scene collected at different time points;
[0007] Performing density clustering processing on the difference image according to the density clustering algorithm to obtain multiple clusters included in the difference image and the number of objects included in each cluster;
[0008] If among the multiple clusters, the first number of clusters with the number of objects being the specified number is less than or equal to the first threshold, then bounding boxes are respectively constructed for each cluster according to the clustering center and boundary points of each cluster in the multiple clusters, and the area of each bounding box is calculated;
[0009] For any one of the bounding boxes, if the area of the bounding box is within the target range and the position information of the clustering center of the cluster corresponding to the bounding box meets the set conditions, then the bounding box is determined as the detection result of the specified category target.
[0010] Optionally, before performing background difference processing on the image to be detected based on the first background image to obtain a difference image, the method further includes:
[0011] Obtaining a first video image for the same scene;
[0012] Performing object detection on the first video image to obtain an object detection result;
[0013] If the object detection result indicates that the first video image contains the specified category of objects, determining the first video image as the image to be detected.
[0014] Optionally, after performing density clustering processing on the difference image according to the density clustering algorithm to obtain a plurality of clusters included in the difference image and the number of objects included in each cluster, the method further includes:
[0015] If the first number is greater than the first threshold, updating the first background image according to the image to be detected to obtain a second background image, and using the second background image to update the first background image, and then returning to execute the step of performing background difference processing on the image to be detected based on the first background image to obtain a difference image and subsequent steps.
[0016] Optionally, the updating the first background image according to the image to be detected to obtain a second background image includes:
[0017] Determining a second number of moving objects included in the image to be detected;
[0018] If the second number is greater than or equal to a second threshold, determining first pixel positions of each moving object included in the image to be detected;
[0019] Obtaining a plurality of frames of second video images adjacent to the image to be detected, and respectively determining second pixel positions of each moving object included in each second video image;
[0020] Processing the image to be detected according to the first pixel positions of each moving object included in the image to be detected and the second pixel positions of each moving object included in each second video image to obtain the second background image.
[0021] Optionally, the processing the image to be detected according to the first pixel positions of each moving object included in the image to be detected and the second pixel positions of each moving object included in each second video image to obtain the second background image includes:
[0022] For each of the first pixel positions, if the first pixel position does not coincide with any of the second pixel positions, then set the pixel at the first pixel position in the image to be detected to a specified pixel value to obtain the processed image to be detected.
[0023] Determine the processed image to be detected as the second background image.
[0024] Optionally, after determining the second quantity of moving objects included in the image to be detected, further include:
[0025] If the second quantity is less than the second threshold, obtain multiple frames of third video images adjacent to the image to be detected.
[0026] Process the image to be detected and the multiple frames of third video images using the Gaussian mixture background modeling algorithm to obtain the second background image.
[0027] Optionally, the target range is determined in the following manner:
[0028] Determine the scaling ratio corresponding to the bounding box corresponding to the position information in the imaging device used to capture the image to be detected.
[0029] Calculate the target range based on the scaling ratio and the area of the target of the specified category.
[0030] Optionally, the target of the specified category is a vehicle; after, for any one of the bounding boxes, if the area of the bounding box is within the target range and the position information of the clustering center of the cluster corresponding to the bounding box meets the set conditions, and the bounding box is determined as the detection result of the target of the specified category, further include:
[0031] Obtain the historical driving trajectory of the target vehicle.
[0032] Predict the position of the target vehicle within the shooting range based on the historical driving trajectory to obtain a predicted position; the shooting range refers to the shooting range of the imaging device used to capture the image to be detected.
[0033] If the difference between the predicted position and the target position of the bounding box determined as the detection result of the target of the specified category within the shooting range is less than or equal to the third threshold, then generate the current driving trajectory of the target vehicle based on the target position.
[0034] In a second aspect, an embodiment of the present application provides a target detection device, including:
[0035] A first processing unit for performing background difference processing on an image to be detected based on a first background image to obtain a difference image; wherein, the first background image and the image to be detected are images of the same scene collected at different time points;
[0036] A second processing unit for performing density clustering processing on the difference image according to a density clustering algorithm to obtain a plurality of clusters included in the difference image and the number of objects included in each cluster;
[0037] A construction unit for, if, among the plurality of clusters, the first number of clusters containing a specified number of objects is less than or equal to a first threshold, respectively constructing a bounding box for each cluster according to the clustering center and boundary points of each cluster in the plurality of clusters, and calculating the area of each of the bounding boxes;
[0038] A first determination unit for, for any one of the bounding boxes, if the area of the bounding box is within a target range and the position information of the clustering center of the cluster corresponding to the bounding box meets a set condition, determining the bounding box as the detection result of a specified category target.
[0039] In a third aspect, an embodiment of the present application provides a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, where when the processor executes the computer program, the target detection method described in any one of the first aspects above is implemented.
[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the target detection method described in any one of the first aspects above is implemented.
[0041] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a terminal device, enables the terminal device to execute the target detection method described in any one of the first aspects above.
[0042] The beneficial effects of the embodiments of the present application compared with the prior art are:
[0043] A target detection method provided by an embodiment of the present application performs background difference processing on a to-be-detected image based on a first background image to obtain a difference image; that is, a difference image containing light changes different from the first background image is extracted from the to-be-detected image. Then, density clustering processing is performed on the difference image according to the density clustering algorithm to obtain multiple clusters included in the difference image and the number of objects included in each cluster; and when it is detected that among the multiple clusters, the first number of clusters with the number of objects being a specified number is less than or equal to a first threshold, it indicates that the difference image meets the requirements, that is, there is no need to update the first background image. Therefore, the bounding box of each cluster can be directly constructed according to the clustering center and boundary points of each cluster among the multiple clusters, and the area of each bounding box is calculated; finally, for any one bounding box, when it is detected that the area of the bounding box is within the target range and the position information of the clustering center of the cluster corresponding to the bounding box meets the set conditions, it indicates that the bounding box conforms to the true specified category target. Therefore, the bounding box can be determined as the detection result of the specified category target, improving the detection accuracy and success rate of target detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following-described drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0045] Figure 1 is the implementation flowchart of the target detection method provided by an embodiment of the present application;
[0046] Figure 2 is the implementation flowchart of the target detection method provided by another embodiment of the present application;
[0047] Figure 3 is the implementation flowchart of the target detection method provided by still another embodiment of the present application;
[0048] Figure 4 is the implementation flowchart of the target detection method provided by yet another embodiment of the present application;
[0049] Figure 5 is the structural schematic diagram of the target detection device provided by an embodiment of the present application;
[0050] Figure 6 is the structural schematic diagram of the terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0051] In the following description, for purposes of illustration and not limitation, specific details such as particular system architectures, technologies, etc. are set forth in order to provide a thorough understanding of embodiments of the present application. However, those skilled in the art should understand that the present application may be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary details.
[0052] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or their groups.
[0053] It should also be understood that the term "and / or" as used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0054] As used in the specification of the present application and the appended claims, the term "if" may be construed, depending on the context, as "when" or "once" or "in response to determining" or "in response to detecting". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, as meaning "once determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]".
[0055] In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.
[0056] Reference to "one embodiment" or "some embodiments" or the like described in the specification of the present application means that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.
[0057] In practical applications, existing object detection methods usually denoise a video image after obtaining it, or use a data augmentation algorithm to enhance the video image, and then perform object detection on the processed image. However, the above methods can only handle the impact on picture quality caused by some relatively minor contaminants, such as raindrops, snowflakes, or dust. However, in some scenarios such as closed tunnels, vehicle exhaust contains a large amount of oil and dust, which accumulates on the camera lens. Even after denoising or image enhancement processing of the captured video image, it is difficult to detect whether there is a vehicle in the processed image, thereby reducing the detection accuracy and success rate of object detection. Therefore, when a dirty camera captures a low-quality image, in all embodiments of the present application, the terminal device can perform background difference on the image, perform density clustering on the result of the difference, and then determine the target pixel area threshold for the result of the density clustering in combination with the pixel coordinate position to improve the object detection accuracy and success rate.
[0058] It should be noted that the specific process of how the present application performs object detection can be seen in the detailed description of the following various embodiments, and will not be elaborated here.
[0059] Please refer to Figure 1 , Figure 1 which is a flowchart of the implementation of an object detection method provided by an embodiment of the present application. In the embodiment of the present application, the execution subject of the object detection method is a terminal device. Among them, the terminal device includes, but is not limited to, electronic devices such as notebooks, desktop computers, and computers.
[0060] As Figure 1 shown, the object detection method provided by an embodiment of the present application may include S101 to S104, which are described in detail as follows:
[0061] In S101, background difference processing is performed on the image to be detected based on the first background image to obtain a difference image; wherein, the first background image and the image to be detected are images of the same scene collected at different time points.
[0062] In practical applications, in order to avoid the problem of differential misdetection caused by light, which may lead to the situation where the obtained dirty image cannot be used for object detection using deep learning, the user can send an object detection request to the terminal device.
[0063] In the embodiments of the present application, the terminal device detecting a target detection request may include: detecting a preset operation on the terminal device. The preset operation may be set according to actual needs and is not limited herein. Exemplarily, the preset operation may be clicking a preset control on the terminal device. Based on this, when the terminal device detects that its preset control is clicked, it indicates that a preset operation on the terminal device is detected, that is, a target detection request is detected.
[0064] After the terminal device detects a target detection request, it may obtain an image to be detected.
[0065] In an embodiment of the present application, in order to obtain an accurate image to be detected, the terminal device may specifically obtain the image to be detected through the following steps, which are described in detail below:
[0066] Obtain a first video image of the same scene;
[0067] Perform target detection on the first video image to obtain a target detection result;
[0068] If the target detection result is that the first video image contains the specified category target, determine the first video image as the image to be detected.
[0069] In an implementation manner of this embodiment, the terminal device may obtain a first video image of the same scene in real time through an outdoor camera that is wirelessly communicatively connected to it and is in a scene corresponding to the first background image.
[0070] It should be noted that the first video image may be one frame or multiple frames at the beginning.
[0071] In this embodiment, after the terminal device obtains the first video image, it may perform target detection on the first video image to obtain a target detection result.
[0072] In some possible embodiments, the terminal device may input the first video image into a trained target detection model for target recognition to obtain a target detection result corresponding to the first video image.
[0073] It should be noted that the target detection model is used to detect target objects in an image and identify the types of the target objects. Among them, the target detection model may be an existing target detection model based on a convolutional neural network.
[0074] In this embodiment, the target detection result is used to describe the category of the target object included in the first video image.
[0075] The category of the target object may be a specified category target. Exemplarily, the specified category target refers to a vehicle.
[0076] In this embodiment, when the terminal device detects that the target detection result is that the first video image contains a specified category of target, the first video image can be determined as the image to be detected.
[0077] In an embodiment of the present application, after obtaining the image to be detected, the terminal device can perform background difference processing on the image to be detected based on the first background image to obtain a difference image. Wherein, the first background image and the image to be detected are images of the same scene collected at different time points.
[0078] In practical applications, background difference processing specifically refers to extracting the changing area from the sequence image (such as the image to be detected) from the background image (such as the first background image).
[0079] In S102, perform density clustering processing on the difference image according to the density clustering algorithm to obtain multiple clusters included in the difference image and the number of objects included in each cluster.
[0080] In an embodiment of the present application, after obtaining the difference image, the terminal device can perform density clustering processing on the difference image according to the density clustering algorithm to obtain multiple clusters included in the difference image and the number of objects included in each cluster.
[0081] It should be noted that the density clustering algorithm can be Density-Based Spatial Clustering of Application with Noise (DBSCAN).
[0082] After the terminal device obtains multiple clusters included in the difference image and the number of objects included in each cluster, it can detect whether there is a cluster with the specified number of objects among the multiple clusters, and detect the first number of clusters with the specified number of objects. Wherein, the specified number can be determined according to actual needs and is not limited here. Exemplarily, the specified number can be 1.
[0083] After that, the terminal device can compare the above first number with a first threshold. Wherein, the first threshold can be determined according to actual needs and is not limited here.
[0084] In an embodiment of the present application, when the terminal device detects that the first number is less than or equal to the first threshold, it can execute steps S103 to S104.
[0085] In another embodiment of the present application, when the terminal device detects that the first number is greater than the first threshold, it can execute step S201 as Figure 2 shown.
[0086] In S103, if, among the multiple clusters, the first quantity of clusters each containing a specified number of objects is less than or equal to the first threshold, then bounding boxes for each cluster are respectively constructed based on the cluster center and boundary points of each cluster among the multiple clusters, and the areas of the respective bounding boxes are calculated.
[0087] In the embodiments of the present application, when the terminal device detects that the first quantity is less than or equal to the first threshold, it indicates that the first background image is the latest background image. That is to say, the difference image obtained by the terminal device performing background difference processing on the image to be detected based on the first background image is accurate. Therefore, the terminal device can respectively construct bounding boxes for each cluster among the multiple clusters determined based on the difference image, and calculate the areas of the respective bounding boxes.
[0088] After the terminal device obtains the areas of the respective bounding boxes, for any one of the bounding boxes, the terminal device can detect whether the area of the bounding box is within the target range, and detect whether the position information of the cluster center corresponding to the bounding box meets the set conditions.
[0089] Among them, the set conditions can be determined according to actual needs and are not limited herein. Exemplarily, the set condition can be that the position information of the cluster center is the position of the center of the vehicle.
[0090] The target range can be determined according to the actual size of the vehicle and the imaging ratio of the outdoor camera that captures the image to be detected.
[0091] In some possible embodiments, the terminal device can specifically determine the target range according to the following steps, which are described in detail as follows:
[0092] Determine the scaling ratio corresponding to the bounding box of the position information in the imaging device used to capture the image to be detected;
[0093] Calculate the target range based on the scaling ratio and the area of the specified category target.
[0094] In this embodiment, the terminal device can determine the scaling ratio corresponding to the bounding box of the position information of the cluster center corresponding to the bounding box in the imaging device used to capture the image to be detected.
[0095] After that, the terminal device can calculate the size information of the specified category target at the above position information in the image to be detected based on the scaling ratio and the actual area of the specified category target, and determine the size information as the target range.
[0096] In an embodiment of the present application, when the terminal device detects that the area of the bounding box is within the target range and the position information of the clustering center of the cluster corresponding to the bounding box meets the set conditions, step S104 can be executed.
[0097] In another embodiment of the present application, when the terminal device detects that the area of the bounding box is not within the target range and / or the position information of the clustering center of the cluster corresponding to the bounding box does not meet the set conditions, it indicates that the object in the bounding box is not a real vehicle. Therefore, the terminal device can discard the bounding box and detect the next bounding box.
[0098] In S104, for any one of the bounding boxes, if the area of the bounding box is within the target range and the position information of the clustering center of the cluster corresponding to the bounding box meets the set conditions, then the bounding box is determined as the detection result of the specified category target.
[0099] In the embodiment of the present application, for any one bounding box, when the terminal device detects that the area of the bounding box is within the target range and the position information of the clustering center of the cluster corresponding to the bounding box meets the set conditions, it indicates that the object in the bounding box is a real vehicle. Therefore, the terminal device can determine the bounding box as the detection result of the specified category target.
[0100] As can be seen from the above, a target detection method provided by the embodiment of the present application performs background difference processing on the image to be detected based on the first background image to obtain a difference image; that is, a difference image containing light changes different from the first background image is extracted from the image to be detected. Then, density clustering processing is performed on the difference image according to the density clustering algorithm to obtain multiple clusters contained in the difference image and the number of objects contained in each cluster; and when it is detected that among the multiple clusters, the first number of clusters containing the specified number of objects is less than or equal to the first threshold, it indicates that the difference image meets the requirements, that is, there is no need to update the first background image. Therefore, the bounding box of each cluster can be directly constructed according to the clustering center and boundary points of each cluster in the multiple clusters, and the area of each bounding box is calculated; finally, for any one bounding box, when it is detected that the area of the bounding box is within the target range and the position information of the clustering center of the cluster corresponding to the bounding box meets the set conditions, it indicates that the bounding box meets the real specified category target. Therefore, the bounding box can be determined as the detection result of the specified category target, improving the detection accuracy and success rate of target detection.
[0101] Please refer to Figure 2 , Figure 2 which is the target detection method provided by another embodiment of the present application. Compared with Figure 1 the corresponding embodiment, after S104, this embodiment may further include S401 to S403, which are described in detail as follows:
[0102] In S201, if the first quantity is greater than the first threshold, the first background image is updated according to the image to be detected to obtain a second background image, and the first background image is updated with the second background image, and then the steps of performing background difference processing on the image to be detected based on the first background image to obtain a difference image and subsequent steps are returned for execution.
[0103] In this embodiment, when the terminal device detects that the first quantity is greater than the first threshold, it indicates that the first background image is not the latest background image. That is to say, the difference image obtained by the terminal device performing background difference processing on the image to be detected based on the first background image is inaccurate. That is, the terminal device needs to update the first background image. Therefore, the terminal device can update the first background image according to the image to be detected to obtain a second background image, and update the first background image with the second background image, and then return to execute steps S101 - S104.
[0104] In an embodiment of the present application, the terminal device can specifically obtain the second background image through steps S301 - S304 as shown in Figure 3 as follows:
[0105] In S301, determine the second quantity of moving objects included in the image to be detected.
[0106] It should be noted that the moving object can be a vehicle.
[0107] In this embodiment, the terminal device can compare the second quantity with a second threshold. The second threshold can be determined according to actual needs and is not limited here.
[0108] In an embodiment of the present application, when the terminal device detects that the second quantity is greater than or equal to the second threshold, the terminal device can execute steps S302 - S304.
[0109] In another embodiment of the present application, when the terminal device detects that the second quantity is less than the second threshold, the terminal device can specifically obtain the second background image according to the following steps, which are described in detail as follows:
[0110] If the second quantity is less than the second threshold, obtain multiple frames of third video images adjacent to the image to be detected;
[0111] Process the image to be detected and the multiple frames of third video images using the mixture Gaussian background modeling algorithm to obtain the second background image.
[0112] In this embodiment, when the terminal device detects that the second quantity is less than the second threshold, it indicates that there are fewer interfering objects in the image to be detected. Therefore, in order to improve the processing efficiency, the terminal device can obtain multiple frames of third video images adjacent to the image to be detected, and use the Gaussian mixture background modeling algorithm to process the image to be detected and the multiple frames of third video images, so as to obtain the second background image.
[0113] It should be noted that the number of multiple frames of third video images can be 5 frames.
[0114] The Gaussian mixture background modeling algorithm can be an existing background modeling based on the Gaussian mixture model.
[0115] In S302, if the second quantity is greater than or equal to the second threshold, then determine the first pixel positions of the respective moving objects included in the image to be detected.
[0116] In S303, obtain multiple frames of second video images adjacent to the image to be detected, and respectively determine the second pixel positions of the respective moving objects included in each of the second video images.
[0117] In S304, process the image to be detected according to the first pixel positions of the respective moving objects included in the image to be detected and the second pixel positions of the respective moving objects included in each of the second video images, so as to obtain the second background image.
[0118] In this embodiment, when the terminal device detects that the second quantity is greater than or equal to the second threshold, it indicates that there are more interfering objects in the image to be detected. Therefore, in order to improve the accuracy of the second background image, the terminal device can determine the first pixel positions of the respective moving objects included in the image to be detected.
[0119] After that, the terminal device can obtain multiple frames of second video images adjacent to the image to be detected, and respectively determine the second pixel positions of the respective moving objects included in each of the second video images. Among them, the number of multiple frames of second video images can be 10 frames.
[0120] In this embodiment, after the terminal device determines the second pixel positions of the respective moving objects included in each of the second video images, it can compare the first pixel positions with the respective second pixel positions, and determine the second background image according to the comparison result.
[0121] Based on this, in an embodiment of the present application, the terminal device can specifically obtain the second background image according to the following steps, which are described in detail as follows:
[0122] For each of the first pixel positions, if the first pixel position does not coincide with each of the second pixel positions, then set the pixel at the first pixel position in the image to be detected to a specified pixel value to obtain the processed image to be detected.
[0123] Determine the processed image to be detected as the second background image.
[0124] In this embodiment, for the first pixel position of any moving object, when the terminal device detects that the first pixel position does not coincide with each second pixel position, it indicates that the moving object is in the process of movement, that is, the movement is not the background. Therefore, the terminal device can set the pixel at the first pixel position in the image to be detected to a specified pixel value, thereby obtaining the processed image to be detected.
[0125] Among them, the specified pixel value can be determined according to actual needs and is not limited here. Exemplarily, the specified pixel value can be 255 or 0.
[0126] Based on this, the terminal device can determine the processed image to be detected as the second background image.
[0127] As can be seen from the above, in the object detection method provided in this embodiment, when it is detected that the first quantity is greater than the first threshold, the first background image can be updated according to the image to be detected to obtain the second background image, and the first background image is updated with the second background image, and then return to execute the step of performing background difference processing on the image to be detected based on the first background image to obtain the difference image and subsequent steps, so that the terminal device can process the image to be detected according to the updated first background image, thereby further improving the object detection accuracy.
[0128] Please refer to Figure 4 , Figure 4 which is the object detection method provided in another embodiment of the present application. Compared with Figure 1 the corresponding embodiment, when the specified category object is a vehicle, after S104, this embodiment may further include S401 to S403, which are described in detail as follows:
[0129] In S401, obtain the historical driving trajectory of the target vehicle.
[0130] In an implementation manner of this embodiment, the terminal device can determine the historical driving trajectory of the target vehicle according to the positioning device provided on the target vehicle, such as the Global Positioning System (GPS).
[0131] In S402, the position of the target vehicle within the shooting range is predicted according to the historical driving trajectory to obtain a predicted position; the shooting range refers to the shooting range of the imaging device for shooting the image to be detected.
[0132] In this embodiment, after the terminal device obtains the historical driving trajectory of the target vehicle, it can input the historical driving trajectory into a trained position prediction model for prediction to obtain the predicted position of the target vehicle within the shooting range. The shooting range refers to the shooting range of the imaging device for shooting the image to be detected.
[0133] The position prediction model can be obtained by training a pre-constructed deep learning model based on a preset sample set. Each sample data in the preset sample set includes a sample driving trajectory and a predicted position corresponding to the sample driving trajectory. When training the pre-constructed deep learning model, the sample driving trajectory in each sample is used as the input of the deep learning model, and the predicted position corresponding to the sample driving trajectory in each sample is used as the output of the deep learning model. Through training, the deep learning model can learn the corresponding relationship between all possible driving trajectories and predicted positions, and the trained deep learning model is used as the position prediction model.
[0134] In this embodiment, after the terminal device obtains the predicted position of the target vehicle within the shooting range, it can calculate the difference between the target position of the bounding box of the detection result of the specified category target within the shooting range and the predicted position, and compare the difference with a third threshold. The third threshold can be determined according to actual needs and is not limited here.
[0135] In an embodiment of the present application, when the terminal device detects that the above difference is less than or equal to the third threshold, it can execute step S403.
[0136] In S403, if the difference between the predicted position and the target position of the bounding box of the detection result determined as the specified category target within the shooting range is less than or equal to the third threshold, the current driving trajectory of the target vehicle is generated according to the target position.
[0137] In this embodiment, when the terminal device detects that the above difference is less than or equal to the third threshold, it indicates that the target position is the actual position of the target vehicle within the shooting range. Therefore, the terminal device can generate the current driving trajectory of the target vehicle according to the target position, thereby improving the driving trajectory of the target vehicle, that is, realizing the trajectory tracking of the target vehicle.
[0138] As can be seen above, the object detection method provided in this embodiment obtains the historical driving trajectory of the target vehicle; predicts the position of the target vehicle within the shooting range according to the historical driving trajectory to obtain the predicted position; the shooting range refers to the shooting range of the imaging device used to shoot the image to be detected; if the difference between the predicted position and the target position within the shooting range of the bounding box of the detection result determined as the target of the specified category is less than or equal to the third threshold, then the current driving trajectory of the target vehicle is generated according to the target position. The method provided in this embodiment can perform tracking and matching on the vehicle by combining the predicted position and the target position determined according to the bounding box, so as to accurately generate the driving trajectory of the vehicle and improve the accuracy of determining the driving trajectory of the vehicle.
[0139] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
[0140] Corresponding to the object detection method described in the above embodiment, Figure 5 The structural block diagram of an object detection device provided by an embodiment of the present application is shown. For the sake of convenience of description, only the parts related to the embodiment of the present application are shown. Referring to Figure 5 The object detection device 500 includes: a first processing unit 51, a second processing unit 52, a construction unit 53, and a first determination unit 54. Among them:
[0141] The first processing unit 51 is used to perform background difference processing on the image to be detected based on the first background image to obtain a difference image; wherein, the first background image and the image to be detected are images of the same scene collected at different time points.
[0142] The second processing unit 52 is used to perform density clustering processing on the difference image according to the density clustering algorithm to obtain a plurality of clusters included in the difference image and the number of objects included in each cluster.
[0143] The construction unit 53 is used to, if the first number of clusters with the specified number of objects included in the plurality of clusters is less than or equal to the first threshold, construct a bounding box for each cluster according to the clustering center and boundary points of each cluster in the plurality of clusters, and calculate the area of each bounding box.
[0144] The first determination unit 54 is used to, for any one of the bounding boxes, if the area of the bounding box is within the target range and the position information of the clustering center of the cluster corresponding to the bounding box meets the set conditions, determine the bounding box as the detection result of the target of the specified category.
[0145] In an embodiment of the present application, the target detection device 500 further includes: a first acquisition unit, a first detection unit, and a second determination unit. Among them:
[0146] The first acquisition unit is configured to acquire a first video image for the same scene.
[0147] The first detection unit is configured to perform target detection on the first video image to obtain a target detection result.
[0148] The second determination unit is configured to, if the target detection result indicates that the first video image includes the specified category target, determine the first video image as the image to be detected.
[0149] In an embodiment of the present application, the target detection device 500 further includes: an update unit.
[0150] The update unit is configured to, if the first quantity is greater than the first threshold, update the first background image according to the image to be detected to obtain a second background image, and use the second background image to update the first background image, and then return to execute the step of performing background difference processing on the image to be detected based on the first background image to obtain a difference image and subsequent steps.
[0151] In an embodiment of the present application, the update unit specifically includes: a third determination unit, a fourth determination unit, a second acquisition unit, and a third processing unit. Among them:
[0152] The third determination unit is configured to determine a second quantity of moving objects included in the image to be detected.
[0153] The fourth determination unit is configured to, if the second quantity is greater than or equal to a second threshold, determine a first pixel position of each moving object included in the image to be detected.
[0154] The second acquisition unit is configured to acquire multiple frames of second video images adjacent to the image to be detected, and respectively determine a second pixel position of each moving object included in each second video image.
[0155] The third processing unit is configured to process the image to be detected according to the first pixel position of each moving object included in the image to be detected and the second pixel position of each moving object included in each second video image to obtain the second background image.
[0156] In an embodiment of the present application, the third processing unit specifically includes: a fourth processing unit and a fifth determination unit. Among them:
[0157] The fourth processing unit is configured to, for each of the first pixel positions, if the first pixel position does not coincide with any of the second pixel positions, set the pixel at the first pixel position in the image to be detected to a specified pixel value, so as to obtain the processed image to be detected.
[0158] The fifth determination unit is configured to determine the processed image to be detected as the second background image.
[0159] In an embodiment of the present application, the target detection device 500 further includes: a third acquisition unit and a fifth processing unit. Among them:
[0160] The third acquisition unit is configured to, if the second quantity is less than the second threshold, acquire multiple frames of third video images adjacent to the image to be detected.
[0161] The fifth processing unit is configured to process the image to be detected and the multiple frames of third video images by using a mixture of Gaussian background modeling algorithm to obtain the second background image.
[0162] In an embodiment of the present application, the first determination unit 54 specifically includes: a sixth determination unit and a calculation unit. Among them:
[0163] The sixth determination unit is configured to determine the scaling ratio corresponding to the bounding box corresponding to the position information in the imaging device used to capture the image to be detected.
[0164] The calculation unit is configured to calculate the target range according to the scaling ratio and the area of the specified category target.
[0165] In an embodiment of the present application, the specified category target is a vehicle; the target detection device 500 further includes: a fourth acquisition unit, a prediction unit, and a generation unit. Among them:
[0166] The fourth acquisition unit is configured to acquire the historical driving trajectory of the target vehicle.
[0167] The prediction unit is configured to predict the position of the target vehicle within the shooting range according to the historical driving trajectory to obtain a predicted position; the shooting range refers to the shooting range of the imaging device used to capture the image to be detected.
[0168] The generation unit is configured to, if the difference between the predicted position and the target position within the shooting range of the bounding box of the detection result determined as the specified category target is less than or equal to a third threshold, generate the current driving trajectory of the target vehicle according to the target position.
[0169] It should be noted that for the content such as information interaction and execution process between the above-mentioned devices / units, since it is based on the same concept as the method embodiment of the present application, for its specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details will not be repeated here.
[0170] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above-mentioned division of each functional unit and module is used as an example. In practical applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in a processing unit, or each unit exists physically alone, or two or more units are integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment, and details will not be repeated here.
[0171] Figure 6 It is a schematic structural diagram of a terminal device provided by an embodiment of the present application. As Figure 6 shown, the terminal device 6 of this embodiment includes: at least one processor 60 ( Figure 6 only one is shown in the figure), a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60. When the processor 60 executes the computer program 62, it implements the steps in any of the above-mentioned target detection method embodiments.
[0172] The terminal device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art can understand that Figure 6 this is only an example of the terminal device 6 and does not constitute a limitation on the terminal device 6. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may also include input and output devices, network access devices, etc.
[0173] The so-called processor 60 may be a Central Processing Unit (CPU), and this processor 60 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.
[0174] In some embodiments, the memory 61 may be an internal storage unit of the terminal device 6, such as the memory of the terminal device 6. In some other embodiments, the memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk equipped on the terminal device 1, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 61 may also include both the internal storage unit and the external storage device of the terminal device 6. The memory 61 is used to store an operating system, application programs, a BootLoader, data, and other programs, such as the program code of the computer program, etc. The memory 61 may also be used to temporarily store data that has been output or is to be output.
[0175] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above various method embodiments can be implemented.
[0176] The embodiments of the present application provide a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above various method embodiments when executed.
[0177] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned embodiment methods of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code to a terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.
[0178] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0179] The above-described embodiments are only used to illustrate the technical solutions of this application, rather than to limit it; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of this application, and should all be included in the protection scope of this application.
Claims
1. A target detection method, characterized in that, Including: Performing background difference processing on the image to be detected based on the first background image to obtain a difference image; wherein, the first background image and the image to be detected are images of the same scene collected at different time points; Performing density clustering processing on the difference image according to the density clustering algorithm to obtain a plurality of clusters included in the difference image and the number of objects included in each cluster; If, among the plurality of clusters, the first number of clusters with the number of objects being a specified number is less than or equal to a first threshold, then bounding boxes for each cluster are respectively constructed based on the clustering center and boundary points of each cluster in the plurality of clusters, and the area of each bounding box is calculated; For any one of the bounding boxes, if the area of the bounding box is within a target range and the position information of the clustering center of the cluster corresponding to the bounding box meets a set condition, then the bounding box is determined as the detection result of a specified category target.
2. The object detection method according to claim 1, characterized in that Before performing background difference processing on the image to be detected based on the first background image to obtain a difference image, it further includes: Obtaining a first video image of the same scene; Performing target detection on the first video image to obtain a target detection result; If the target detection result is that the first video image includes the specified category target, then the first video image is determined as the image to be detected.
3. The object detection method according to claim 1, characterized in that After performing density clustering processing on the difference image according to the density clustering algorithm to obtain a plurality of clusters included in the difference image and the number of objects included in each cluster, it further includes: If the first number is greater than the first threshold, then the first background image is updated according to the image to be detected to obtain a second background image, and the first background image is updated with the second background image, and then the step of performing background difference processing on the image to be detected based on the first background image to obtain a difference image and subsequent steps are returned to for execution.
4. The object detection method according to claim 3, wherein The updating the first background image according to the image to be detected to obtain a second background image includes: Determining a second number of moving objects included in the image to be detected; If the second number is greater than or equal to a second threshold, then determining the first pixel positions of the respective moving objects included in the image to be detected; Obtaining a plurality of frames of second video images adjacent to the image to be detected, and respectively determining the second pixel positions of the respective moving objects included in each second video image; Processing the image to be detected according to the first pixel positions of the respective moving objects included in the image to be detected and the second pixel positions of the respective moving objects included in each second video image to obtain the second background image.
5. The object detection method according to claim 4, wherein The processing the image to be detected according to the first pixel positions of the respective moving objects included in the image to be detected and the second pixel positions of the respective moving objects included in each second video image to obtain the second background image includes: For each of the first pixel positions, if the first pixel position does not coincide with each of the second pixel positions, then setting the pixel point at the first pixel position to a specified pixel value in the image to be detected to obtain the processed image to be detected; Determine the processed image to be detected as the second background image.
6. The object detection method according to claim 4, characterized in that, After determining the second quantity of moving objects included in the image to be detected, it further includes: If the second quantity is less than the second threshold, obtain multiple frames of third video images adjacent to the image to be detected; Process the image to be detected and the multiple frames of third video images using the Gaussian mixture background modeling algorithm to obtain the second background image.
7. The object detection method according to claim 1, wherein The target range is determined according to the following method: Determine the scaling ratio corresponding to the bounding box corresponding to the position information in the imaging device used to capture the image to be detected; Calculate the target range based on the scaling ratio and the area of the specified category target.
8. The object detection method according to any one of claims 1-7, characterized in that The specified category target is a vehicle; after, for any one of the bounding boxes, if the area of the bounding box is within the target range and the position information of the clustering center of the cluster corresponding to the bounding box meets the set conditions, determining the bounding box as the detection result of the specified category target, it further includes: Obtain the historical driving trajectory of the target vehicle; Predict the position of the target vehicle within the shooting range based on the historical driving trajectory to obtain a predicted position; the shooting range refers to the shooting range of the imaging device used to capture the image to be detected; If the difference between the predicted position and the target position of the bounding box determined as the detection result of the specified category target within the shooting range is less than or equal to the third threshold, generate the current driving trajectory of the target vehicle based on the target position.
9. A target detection device, characterized in that, It includes: A first processing unit for performing background difference processing on the image to be detected based on the first background image to obtain a difference image; wherein, the first background image and the image to be detected are images of the same scene captured at different time points; A second processing unit for performing density clustering processing on the difference image according to the density clustering algorithm to obtain multiple clusters included in the difference image and the number of objects included in each cluster; A construction unit for, if among the multiple clusters, the first quantity of clusters containing a specified number of objects is less than or equal to the first threshold, respectively construct a bounding box for each cluster based on the clustering center and boundary points of each cluster in the multiple clusters, and calculate the area of each bounding box; A first determination unit for, for any one of the bounding boxes, if the area of the bounding box is within the target range and the position information of the clustering center of the cluster corresponding to the bounding box meets the set conditions, determining the bounding box as the detection result of the specified category target.
10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the target detection method according to any one of claims 1 to 8.
11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the target detection method according to any one of claims 1 to 8.