Target detection method and device, electronic equipment and storage medium
By identifying the first region of the target in the target detection image and utilizing the cross-union ratio variation law and similarity recognition network, the problem of low accuracy in moving target recognition is solved, and efficient and accurate target management is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE QUANTONG SYST INTEGRATION CO LTD
- Filing Date
- 2021-10-12
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies have low accuracy in identifying and managing moving targets, cannot identify abnormal behavior, are prone to false identification and false warnings, and are inefficient and labor-intensive to manage manually.
By determining the sequence of consecutive frames of the target detection image, inputting it into the target detection model, identifying the first region of the target, and judging whether there is anomaly in the target based on the change law of cross-union ratio, the model is trained by combining similarity recognition network and dwell analysis, using one-dimensional linear interpolation and cosine annealing algorithms.
It improves the accuracy of target identification, reduces labor costs, lowers the rate of false identification and false alarms, and enables effective monitoring and management of moving targets.
Smart Images
Figure CN115984760B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target recognition technology, and in particular to a target detection method, apparatus, electronic device, and storage medium. Background Technology
[0002] In current monitoring and management work, one approach is on-site supervision primarily based on human intervention, while the other is fixed-point monitoring and snapshot capture, combined with emerging technologies such as big data and artificial intelligence to identify and manage the targets to be monitored.
[0003] However, for highly mobile targets, the governance and management of on-site supervision methods dominated by manual labor are inefficient, labor-intensive, and lack effective data collection and early warning. While existing methods for identifying and managing mobile targets using technologies such as artificial intelligence reduce labor costs and lower the difficulty of evidence collection, their identification accuracy is low, they cannot identify whether there are any anomalies in the mobile targets, and they are prone to misidentification and false warnings. Summary of the Invention
[0004] To address the problems existing in the prior art, the present invention provides a target detection method, apparatus, electronic device, and storage medium.
[0005] In a first aspect, the present invention provides a target detection method, comprising: determining a target detection image, wherein the target detection image comprises a sequence of images of a consecutive preset number of frames;
[0006] The target detection image is input into the target detection model to identify the target and obtain the first region where the target is located. Each frame of the target detection image corresponds to one first region.
[0007] Based on the variation pattern of the cross-union ratio in the first region, it is determined whether the detected target is abnormal.
[0008] In one embodiment, determining whether the detected target is abnormal based on the variation pattern of the cross-union ratio in the first region includes:
[0009] Determine the second region where the detection target is located, wherein the second region is the first region corresponding to the last frame image in the target detection image;
[0010] Calculate the intersection-union ratio (IUR) between the second region and all the first regions except the first region corresponding to the second region, to obtain an IUR sequence.
[0011] If all values in the cross-union ratio sequence are greater than or equal to the first threshold, it is determined that the detected target is in a stationary state.
[0012] If the values in the cross-union ratio sequence are not all greater than the first threshold and are not all equal, the detected target is determined to be in motion.
[0013] In one embodiment, after determining that the detected target is in a stationary state when all values in the cross-union ratio sequence are greater than or equal to a first threshold, the method further includes:
[0014] Determine the first duration for which the detected target remains in the dwell state;
[0015] If the first duration is longer than the first preset duration, the detection target is determined to be in the dwell state.
[0016] In one embodiment, the method further includes:
[0017] Input any frame of the target detection image into a similarity recognition network to determine whether the detected target has been registered;
[0018] If the target being detected has not registered the information, it is determined that the target being detected is abnormal.
[0019] In one embodiment, the method further includes:
[0020] If the detection target has already registered the information, determine whether the location of the detection target is within the first designated area;
[0021] If the location of the detected target is not in the first designated area, determine a second duration for which the detected target is not in the first designated area;
[0022] If the second duration is longer than the second preset duration, it is determined that the detection target is abnormal.
[0023] In one embodiment, the object detection model is trained based on a loss function;
[0024] The loss function is used to characterize the location loss of the detected target, the confidence loss of the target box, and the category loss of the detected target.
[0025] In one embodiment, the method further includes:
[0026] During the training of the target detection model, a combination of one-dimensional linear interpolation and cosine annealing algorithms is used to update the learning rate of each layer in the target detection model.
[0027] In a second aspect, the present invention provides a target detection device, comprising:
[0028] A determination module is used to determine a target detection image, wherein the target detection image includes a sequence of images for a consecutive preset number of frames;
[0029] The acquisition module is used to input the target detection image into the target detection model, identify the detection target, and acquire the first region where the detection target is located, wherein each frame of the target detection image corresponds to one first region;
[0030] The judgment module is used to determine whether the detected target has any abnormalities based on the change pattern of the cross-union ratio in the first region.
[0031] Thirdly, the present invention provides an electronic device, including a memory and a memory storing a computer program, wherein the processor executes the program to implement the steps of the target detection method described in the first aspect.
[0032] Fourthly, the present invention provides a processor-readable storage medium storing a computer program for causing the processor to perform the steps of the target detection method described in the first aspect.
[0033] The target detection method, apparatus, electronic device, and storage medium provided by this invention determine a target detection image, input the target detection image into a target detection model, identify the target, obtain a first region where the target is located, and then determine whether the target is abnormal based on the change law of the cross-union ratio of the first region. This invention combines the target detection model and the change law of the cross-union ratio of the target's region to determine whether the target is abnormal, which not only reduces labor costs but also effectively improves the accuracy of target recognition. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0035] Figure 1 This is one of the flowcharts of the target detection method provided by the present invention;
[0036] Figure 2 This is a schematic diagram of the similarity recognition network structure provided by the present invention;
[0037] Figure 3 This is the second flowchart of the target detection method provided by the present invention;
[0038] Figure 4 This is a schematic diagram of the regional distribution provided by the present invention;
[0039] Figure 5 This is the third flowchart of the target detection method provided by the present invention;
[0040] Figure 6 This is a schematic diagram of the target detection device provided by the present invention;
[0041] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0043] The following is combined Figures 1-6 This invention describes the target detection method and apparatus provided by the present invention.
[0044] Figure 1 This is one of the flowcharts of the target detection method provided by the present invention, such as... Figure 1 As shown, the method includes the following steps:
[0045] Step 100: Determine the target detection image, which includes a sequence of images with a preset number of consecutive frames;
[0046] Step 110: Input the target detection image into the target detection model, identify the target, and obtain the first region where the target is located. Each frame of the target detection image corresponds to one first region.
[0047] Step 120: Based on the variation pattern of the cross-union ratio in the first region, determine whether the detected target is abnormal.
[0048] Optionally, the target detection image can be determined.
[0049] Optionally, the target detection images may include a sequence of images for a predetermined number of consecutive frames.
[0050] Optionally, the target detection image may include 20 frames, 40 frames, 65 frames or 100 frames, and the present invention does not specifically limit this.
[0051] Alternatively, the target detection image can be an image captured by a device with video recording or camera capabilities.
[0052] Alternatively, the target detection image can be input into the target detection model to identify the target.
[0053] Optionally, the object detection model can be any deep neural network structure, including but not limited to YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), and other structures.
[0054] Optionally, the first region where the target is located can be obtained.
[0055] Optionally, each frame of the object detection image can correspond to a first region.
[0056] Optionally, the presence of anomalies in the detected target can be determined based on the variation pattern of the cross-union ratio in the first region.
[0057] Alternatively, it can be done through a formula Calculate the cross-union ratio (CUP) of the first region corresponding to two different frame images.
[0058] Where p represents the Euclidean distance between the center points of the detection boxes of the first region corresponding to the image frame being detected and the detection boxes of the first region corresponding to the image frame in another frame, c represents the diagonal distance of the smallest closure region that can simultaneously contain two detection boxes, iou represents the intersection-union ratio (IU) between the predicted box and the actual box, and α is expressed by the formula... Calculations show that v is obtained through the formula. The calculations show that w and h are the width and height of the detection box of the first region corresponding to the image frame being detected, respectively, and w′ and h′ are the width and height of the detection box of the first region corresponding to any other image frame, respectively.
[0059] To address the issues of low accuracy in moving target recognition in existing technologies, the inability to identify anomalies in moving targets, and the susceptibility to false identification and false warnings, this invention first inputs a target detection image of a preset number of consecutive frames into a target detection model for recognition, thereby obtaining a first region where the target is located. Then, based on the variation pattern of the intersection-union ratio (IU) of the first region, it determines whether the target is abnormal. By combining the target detection model with the variation pattern of the IU of the target's region, the accuracy of target recognition is effectively improved.
[0060] The target detection method provided by this invention determines a target detection image, inputs the target detection image into a target detection model, identifies the target, obtains the first region where the target is located, and then judges whether the target has any abnormalities based on the change law of the cross-union ratio of the first region. This invention combines the target detection model and the change law of the cross-union ratio of the target's region to judge whether the target has any abnormalities, which not only reduces labor costs but also effectively improves the accuracy of target recognition.
[0061] Optionally, determining whether the detected target is abnormal based on the variation pattern of the cross-union ratio in the first region includes:
[0062] Determine the second region where the detection target is located, wherein the second region is the first region corresponding to the last frame image in the target detection image;
[0063] Calculate the intersection-union ratio (IUR) between the second region and all the first regions except the first region corresponding to the second region, to obtain an IUR sequence.
[0064] If all values in the cross-union ratio sequence are greater than or equal to the first threshold, it is determined that the detected target is in a stationary state.
[0065] If the values in the cross-union ratio sequence are not all greater than the first threshold and are not all equal, the detected target is determined to be in motion.
[0066] Optionally, a second region may be identified where the target to be detected is located.
[0067] Optionally, the second region can be the first region corresponding to the last frame of the object detection image.
[0068] Optionally, the cross-union ratios (CUNRs) between the second region and all first regions other than the first region corresponding to the second region can be calculated to obtain a CUNR sequence.
[0069] For example, if the target detection image consists of 10 frames, the cross-union ratio (CUP) between the first region corresponding to the 10th frame and the first regions corresponding to the previous 9 frames can be calculated to obtain a CUP sequence, which includes 9 CUP values.
[0070] Optionally, if all values in the cross-union ratio sequence are greater than or equal to the first threshold, it can be determined that the detected target is in a stationary state.
[0071] For example, if the values in the cross-union ratio sequence are 0.958, 0.962, 0.956, 0.963, 0.971, 0.989, 0.995, and 0.998, and the first threshold is 0.95, it can be determined that the detected target is in a stationary state.
[0072] Optionally, if the values in the cross-union ratio sequence are not all greater than the first threshold and are not all equal, it is determined that the detected target is in motion.
[0073] For example, if the values in the cross-union ratio sequence are 0.081, 0.262, 0.356, 0.463, 0.471, 0.489, 0.595, and 0.698, and the first threshold is 0.90, it can be determined that the detected target is in motion.
[0074] Optionally, the size of the first threshold can be 0.8, 0.9, 0.95, or 0.98, and the present invention does not specifically limit it.
[0075] This invention calculates the cross-union ratio (CUP) between the first region corresponding to the last frame of the target detection image and the first region corresponding to other frames, obtains an CUP sequence, and judges the values in the CUP sequence against a first threshold, thereby determining whether the detected target is in a stationary or moving state, and realizing effective monitoring of the detected target.
[0076] Optionally, after determining that the detected target is in a stationary state when all values in the intersection-union ratio sequence are greater than or equal to a first threshold, the method further includes:
[0077] Determine the first duration for which the detected target remains in the dwell state;
[0078] If the first duration is longer than the first preset duration, the detection target is determined to be in the dwell state.
[0079] Optionally, the first duration for which the target remains in the detection state can be determined.
[0080] Optionally, the detection time of each frame in the target detection image can be recorded by the target detection model to determine the first duration of the target's dwell state.
[0081] Optionally, if the first duration is longer than the first preset duration, it can be determined that the target is in a stationary state.
[0082] Optionally, the first preset duration can be 10 minutes, 20 minutes, or 25 minutes, etc., and the present invention does not specifically limit it.
[0083] Optionally, if the first duration is not greater than the first preset duration, it can be determined that the target is in a temporary dwelling state.
[0084] The target detection method provided by this invention can effectively reduce the misidentification of targets that are temporarily stationary by statistically analyzing the duration of the detected targets.
[0085] Optionally, the method further includes:
[0086] Input any frame of the target detection image into a similarity recognition network to determine whether the detected target has been registered;
[0087] If the target being detected has not registered the information, it is determined that the target being detected is abnormal.
[0088] Optionally, any frame of the target detection image can be input into the similarity recognition network to determine whether the detected target has been registered.
[0089] Optionally, a similarity recognition network can be used to determine whether the target information has been registered and stored in a database.
[0090] Optionally, the similarity recognition network can calculate the similarity between any frame of the target detection image and the images stored in the information database to determine whether the detected target has been registered.
[0091] Figure 2 This is a schematic diagram of the similarity recognition network structure provided by the present invention, as shown below. Figure 2 As shown, the similarity matching network consists of two identical deep convolutional neural networks (Network1 and Network2), where X1 is an arbitrary frame of object detection image and X2 is an image stored in the database. Each network takes an image as input and feeds it sequentially into the backbone network, passing through convolutional layers, pooling layers, and fully connected layers, finally outputting a feature vector (e.g., G in the figure). W (X1) and G W (X2) is added to the contrast loss function to calculate the similarity between the two images.
[0092] Optionally, the structures of the two deep convolutional neural networks in the similarity matching network include, but are not limited to, network structures such as Visual Geometry Group (VGG) and Residual Neural Network (ResNet).
[0093] Alternatively, the similarity matching network can be implemented using a formula. Calculate the loss function.
[0094] Among them, E w Y is the Euclidean distance between the two images output by the similarity matching network; Y is 1 or 0 if the two images predicted by the similarity matching network are similar, and 1 otherwise; m is the threshold.
[0095] It is understandable that when the input images are similar, the larger the Euclidean distance, the greater the loss value; when the input images are different, the smaller the Euclidean distance, the greater the loss value; and when the Euclidean distance is greater than the threshold m, no further loss will occur.
[0096] Optionally, the training dataset used by the similarity matching network can be the detection data from different angles used by the object detection part.
[0097] Optionally, if the target has not registered its information, it can be determined that the target is abnormal.
[0098] Optionally, if the detected target is abnormal, an evidence collection alarm can be issued for the detected target.
[0099] The target detection method provided by this invention achieves multi-level detection of abnormal behavior of the target by combining deep convolutional neural network target detection technology, dwell analysis and similarity matching for multiple joint judgment.
[0100] Optionally, the method further includes:
[0101] If the detection target has already registered the information, determine whether the location of the detection target is within the first designated area;
[0102] If the location of the detected target is not in the first designated area, determine a second duration for which the detected target is not in the first designated area;
[0103] If the second duration is longer than the second preset duration, it is determined that the detection target is abnormal.
[0104] Optionally, if the target has been registered, it can be determined whether the target is located in the first designated area.
[0105] Optionally, if the location of the detected target is not in the first designated area, a second duration for which the detected target is not in the first designated area can be determined.
[0106] Optionally, if the second duration is longer than the second preset duration, it can be determined that the detected target is abnormal.
[0107] Optionally, the second preset duration can be 10 minutes, 20 minutes, or 25 minutes, etc., and the present invention does not specifically limit it.
[0108] For example, if the center point of the target being detected is outside the designated first area for 20 minutes or more, it can be considered that the target is illegally crossing the boundary or engaging in illegal street vending, and an evidence collection and alarm can be issued for the target. The first designated area is the area that meets the regulations for the target being detected.
[0109] Optionally, the target detection model is trained based on a loss function;
[0110] The loss function is used to characterize the location loss of the detected target, the confidence loss of the target box, and the category loss of the detected target.
[0111] Alternatively, the object detection model can be trained based on a loss function.
[0112] Alternatively, sample images can be collected using a device with video recording or camera capabilities as a training dataset for the object detection model.
[0113] Optionally, information such as the location and type of the detected target in the sample image can be labeled to obtain corresponding labels.
[0114] Optionally, data augmentation can be performed on the sample images by randomly flipping, scaling, color jittering, and cropping them.
[0115] Alternatively, the enhanced sample images can be used as training data.
[0116] Optionally, the network structure of the object detection model can be trained first using the large ImageNet dataset to obtain a pre-trained model. Then, specific training parameters can be set based on the pre-trained model using the object training dataset.
[0117] Alternatively, the pre-trained model can be iteratively trained using an object detection data training set until the loss function converges to obtain the object detection model.
[0118] Optionally, the loss function can be used to characterize the location loss of the detected target, the confidence loss of the bounding box, and the category loss of the detected target.
[0119] Alternatively, the loss function can be expressed by the formula Loss = L loc +L conf +L cls Perform the calculation.
[0120] Among them, L loc L represents the position loss of the detected target. conf L represents the confidence loss of the target bounding box. cls This represents the category loss of the detected target.
[0121] Optionally, the position loss of the detected target can be calculated using the formula... Perform the calculation.
[0122] Where p represents the Euclidean distance between the center points of the predicted bounding box and the actual bounding box, c represents the diagonal distance of the smallest closure region that can simultaneously contain both the predicted and actual bounding boxes, iou represents the intersection-union ratio (IU) between the predicted and actual bounding boxes, and α is expressed by the formula... Calculations show that v is obtained through the formula. The calculations show that w and h are the width and height of the actual bounding box, respectively, and w′ and h′ are the width and height of the predicted bounding box, respectively.
[0123] Optionally, the confidence loss of the bounding box and the category loss of the detected target can be expressed by the formula... Perform the calculation.
[0124] Where y represents the actual label value, y′ represents the predicted label value, and n represents the number of categories of the detected target.
[0125] Optionally, the method further includes:
[0126] During the training of the target detection model, a combination of one-dimensional linear interpolation and cosine annealing algorithms is used to update the learning rate of each layer in the target detection model.
[0127] Alternatively, during the training of the object detection model, a combination of one-dimensional linear interpolation and cosine annealing algorithms can be used to update the learning rate of each layer in the object detection model.
[0128] Alternatively, the object detection model can be optimized using the stochastic gradient descent (SGD) method.
[0129] Optionally, the object detection model may include a weight layer, a bias layer, and a batch normalization (BN) layer.
[0130] Optionally, different learning rate adjustment methods can be used for different layers of the object detection model, and the learning rate of different layers can be adjusted separately.
[0131] Alternatively, a warm-up method can be used to warm up the learning rate.
[0132] Optionally, during the warm-up phase, one-dimensional linear interpolation can be used to update the learning rate for each iteration.
[0133] Optionally, after the warm-up phase, the learning rate can be updated using a cosine annealing algorithm.
[0134] Alternatively, the cosine annealing algorithm can be expressed as:
[0135]
[0136] Wherein, new_lr represents the updated learning rate; ini_lr represents the initial learning rate, which is a fixed value (e.g., 0.01); min_lr represents the minimum learning rate, which is a fixed value (e.g., 0.0001); epoch′ represents the value corresponding to the epoch being trained; epoch represents the total number of training epochs, which is a fixed value, for example, the total number of epochs is 200. The number of epochs can be set according to the accuracy requirements.
[0137] The target detection method provided by this invention updates the learning rate of each layer in the target detection model by combining one-dimensional linear interpolation and cosine annealing algorithm, thereby obtaining a higher quality learning rate and a more reliable target detection model, thus improving the accuracy of target detection.
[0138] The invention will now be described through a specific application scenario.
[0139] In social management, street vendors have become one of the main targets of urban management departments. The target detection method provided by this invention can be used to effectively supervise street vendors.
[0140] Figure 3 This is the second flowchart of the target detection method provided by the present invention, as shown below. Figure 3 As shown, it mainly consists of an entrance scanning module, a vendor information management database, an intelligent analysis platform, a resource management module, a storage module, and an alarm module.
[0141] Figure 4 This is a schematic diagram of the regional distribution provided by the present invention, such as... Figure 4 As shown, a series of legal fixed stalls for mobile vendors can be set up in smart communities or smart shopping malls and streets, and numbered, for example, stall 1, stall 2, stall 3, ..., stall N. At the same time, a QR code scanning area is set up, and QR codes with vendor information filling pages are placed at the entrance of smart streets, communities or other designated locations. Vendors can scan the code to register their relevant information using mobile devices such as mobile phones.
[0142] Optionally, vendor information may include a variety of information that can characterize the vendor, such as the vendor's personal information, current address, vendor type, type of goods sold, and photos of the vendor operating on-site.
[0143] Optionally, the vendor information management database can store the information submitted by mobile vendors.
[0144] This invention establishes a vendor information management database based on information uploaded by vendors, enabling the sharing of vendor information at different monitoring locations.
[0145] Optionally, after receiving the information submitted by the mobile vendor, the system can receive the vendor's input of a selected stall number and lock the selected stall number. The remaining number of stalls is calculated based on the locked stall number and the specified number of stalls. The system then informs the next vendor of the remaining number of stalls in the current location and the available stall numbers. If there are no available stalls, the vendor is prompted to sell in other areas.
[0146] Optionally, for registered vendors, a personal information confirmation can be displayed. If there are any changes, the vendor can be prompted to correct the information or reselect the stall number.
[0147] Optionally, unregistered vendors can be prompted to complete the registration process.
[0148] Optionally, a payment system can be set up for stall management fees, sanitation and cleaning fees, etc. Multiple payment methods can be adopted, such as one-time payment or time-based payment, depending on the actual situation. Vendors can only enter or leave after they have paid. If the time-based payment method is adopted, vendors can be required to scan the code to pay again when leaving. For vendors who have not paid, corresponding law enforcement measures can be taken and recorded.
[0149] Optionally, for vendors who have not registered by scanning the code, the intelligent analysis platform can be used to detect them and compare them with the information stored in the vendor information management database.
[0150] Optionally, an alert can be issued to vendors who have not registered by scanning the QR code.
[0151] Optionally, the vendor information management database can be used to store vendor registration information, payment details, monitoring locations, and stall numbers, and can aggregate vendor information from different locations to build an information sharing mechanism.
[0152] Optionally, the vendor information management database can also be used to perform matching analysis on vendors detected by the intelligent analysis platform.
[0153] Optionally, after receiving the personal information uploaded by the vendor, the information can be compared with the existing information in the vendor information management database. If it is detected that the vendor has already registered at another location, the vendor can be prompted that they have already registered and do not need to register again.
[0154] Optionally, the intelligent analysis platform can perform intelligent analysis and evidence collection based on the target detection model provided by this invention.
[0155] Optionally, the intelligent analytics platform can perform dwell time analysis on vendors.
[0156] Optionally, the location and time of the first detected stall can be recorded for continuous multi-frame stall location detection.
[0157] Optionally, the cross-union ratio (CUP) of the detection results of the current frame and the previous N frames can be calculated.
[0158] Optionally, if the values of consecutive crossover / union ratios are all greater than or equal to a certain threshold, the stall can be considered to be in a stagnant state.
[0159] Optionally, if the value of the crossover ratio is continuously less than the threshold and changes continuously, the stall can be considered to be in motion.
[0160] Optionally, for stalls in a stationary state, the stationary duration can be calculated. If it does not exceed the preset duration, the stall can be considered to be temporarily stationary; otherwise, the next step of analysis can be performed.
[0161] Optionally, the image of the detected target vendor can be cropped, and the cropped image can be input into a similarity recognition network to determine whether it is the same vendor registered in the vendor information management database.
[0162] Optionally, for unmatched vendors, they can be identified as unregistered vendors, and evidence can be collected and alerts can be issued against them.
[0163] Optionally, a first designated area can be defined in the monitoring interface of the intelligent analysis platform, and the determination can be made based on the detected target location of the stall. The first designated area is an area that meets the regulations for the target stall vendor.
[0164] Optionally, if the stall is located in the first designated area, further determination can be made according to the rules.
[0165] Optionally, if a stall is not located in the designated first area or is detected in a prohibited area, an alert can be issued to the vendor at that stall.
[0166] Optionally, the rules for determining relevant violations may include one or more of the following:
[0167] (1) Illegal crossing of boundaries or occupation of the road: The center point of the stall being inspected is determined. If the target center point is outside the first designated area, it can be considered that the vendor has illegally crossed the boundary or occupied the road, and evidence can be collected and an alarm can be issued against the vendor.
[0168] (2) Unregistered vendors: The similarity recognition network is used to judge the image of the detected target vendor. If it does not match the vendor information management database and the time exceeds a certain period of time, the vendor can be recorded as an unregistered vendor.
[0169] (3) Illegal operation: If a vendor is detected in a prohibited area and the time exceeds a certain period, the vendor can be considered to be operating illegally, and an alarm can be issued and evidence collected.
[0170] In the analysis of mobile vendors, relying solely on stall detection cannot effectively distinguish between idle roadside vendor vehicles, temporarily parked vendor vehicles, and vendors operating normally. Therefore, this invention reduces false identification and false alarm rates by combining target detection, dwell time analysis, and multiple joint judgment methods based on similarity matching.
[0171] Figure 5 This is the third flowchart of the target detection method provided by the present invention, as shown below. Figure 5 As shown, by combining deep convolutional neural network target detection technology, dwell analysis, and similarity matching for multi-level joint judgment, it is possible to effectively detect vendor violations at multiple levels and conduct intelligent supervision of vendors based on different violation judgment rules, thereby effectively reducing the difficulty of law enforcement and evidence collection and improving management efficiency.
[0172] Existing methods for managing street vendors, while employing deep learning for vendor identification, have limited capabilities. For example, they are prone to misidentification and false alarms for vendors who are only temporarily stopping or passing by. Moreover, in the same scenario, they can only identify vendors, not whether they are legal or illegal, resulting in low accuracy in identifying illegal vendors. The target detection method proposed in this invention, on the one hand, can utilize the personal information uploaded by vendors to establish a vendor information management database, effectively enabling the sharing of vendor information across different monitoring points. On the other hand, by combining deep convolutional neural network target detection technology, dwell time analysis, and similarity matching for multi-level joint judgment, it can achieve multi-layered detection of vendor violations and intelligent supervision of vendors based on different violation judgment rules.
[0173] The target detection method provided by this invention determines a target detection image, inputs the target detection image into a target detection model, identifies the target, obtains the first region where the target is located, and then judges whether the target has any abnormalities based on the change law of the cross-union ratio of the first region. This invention combines the target detection model and the change law of the cross-union ratio of the target's region to judge whether the target has any abnormalities, which not only reduces labor costs but also effectively improves the accuracy of target recognition.
[0174] The target detection device provided by the present invention is described below. The target detection device described below and the target detection method described above can be referred to in correspondence.
[0175] Figure 6 This is a schematic diagram of the target detection device provided by the present invention, as shown below. Figure 6 As shown, the device includes: a determining module 610, an acquiring module 620, and a judging module 630; wherein:
[0176] The determining module 610 is used to determine the target detection image, which includes a sequence of images for a consecutive preset number of frames;
[0177] The acquisition module 620 is used to input the target detection image into the target detection model, identify the detection target, and acquire the first region where the detection target is located. Each frame of the target detection image corresponds to one first region.
[0178] The judgment module 630 is used to determine whether the detected target is abnormal based on the change pattern of the cross-union ratio in the first region.
[0179] The target detection device provided by the present invention determines a target detection image, inputs the target detection image into a target detection model, identifies the target, obtains the first region where the target is located, and then determines whether the target is abnormal based on the change law of the cross-union ratio of the first region. The present invention combines the target detection model and the change law of the cross-union ratio of the target region to determine whether the target is abnormal, which not only reduces labor costs, but also effectively improves the accuracy of target recognition.
[0180] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call a computer program stored in the memory 730 to execute the steps of the target detection method, such as:
[0181] Determine the target detection image, which includes a sequence of images for a consecutive preset number of frames;
[0182] The target detection image is input into the target detection model to identify the target and obtain the first region where the target is located. Each frame of the target detection image corresponds to one first region.
[0183] Based on the variation pattern of the cross-union ratio in the first region, it is determined whether the detected target is abnormal.
[0184] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0185] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the target detection method provided by the above methods, the method comprising:
[0186] Determine the target detection image, which includes a sequence of images for a consecutive preset number of frames;
[0187] The target detection image is input into the target detection model to identify the target and obtain the first region where the target is located. Each frame of the target detection image corresponds to one first region.
[0188] Based on the variation pattern of the cross-union ratio in the first region, it is determined whether the detected target is abnormal.
[0189] In another aspect, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing the processor to execute the target detection methods provided in the above embodiments, such as including:
[0190] Determine the target detection image, which includes a sequence of images for a consecutive preset number of frames;
[0191] The target detection image is input into the target detection model to identify the target and obtain the first region where the target is located. Each frame of the target detection image corresponds to one first region.
[0192] Based on the variation pattern of the cross-union ratio in the first region, it is determined whether the detected target is abnormal.
[0193] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).
[0194] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0195] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A target detection method, characterized in that, include: Determine the target detection image, which includes a sequence of images for a consecutive preset number of frames; The target detection image is input into the target detection model to identify the target and obtain the first region where the target is located. Each frame of the target detection image corresponds to one first region. Based on the variation pattern of the cross-union ratio in the first region, it is determined whether the detected target is in a stationary state or a moving state. For a target in a stationary state, the stationary duration is calculated. If it does not exceed the preset duration, the target is considered to be temporarily stationary; otherwise, the next step of analysis is performed. Input any frame of the target detection image into a similarity recognition network to determine whether the detected target has been registered; If the target being detected has not registered the information, it is determined that the target being detected is abnormal. If the detection target has already registered the information, determine whether the location of the detection target is within the first designated area; If the location of the detected target is not in the first designated area, determine a second duration for which the detected target is not in the first designated area; If the second duration is longer than the second preset duration, it is determined that the detection target is abnormal.
2. The target detection method according to claim 1, characterized in that, The step of determining whether the detected target is in a stationary or moving state based on the variation pattern of the cross-union ratio in the first region includes: Determine the second region where the detection target is located, wherein the second region is the first region corresponding to the last frame image in the target detection image; Calculate the intersection-union ratio (IUR) between the second region and all the first regions except the first region corresponding to the second region, to obtain an IUR sequence. If all values in the cross-union ratio sequence are greater than or equal to the first threshold, it is determined that the detected target is in a stationary state. If the values in the cross-union ratio sequence are not all greater than the first threshold and are not all equal, the detected target is determined to be in motion.
3. The target detection method according to claim 2, characterized in that, After determining that the detected target is in a stationary state when all values in the cross-union ratio sequence are greater than or equal to the first threshold, the method further includes: Determine the first duration for which the detected target remains in the dwell state; If the first duration is longer than the first preset duration, the detection target is determined to be in the dwell state.
4. The target detection method according to claim 1, characterized in that, The target detection model is trained based on a loss function; The loss function is used to characterize the location loss of the detected target, the confidence loss of the target box, and the category loss of the detected target.
5. The target detection method according to claim 4, characterized in that, The method further includes: During the training of the target detection model, a combination of one-dimensional linear interpolation and cosine annealing algorithms is used to update the learning rate of each layer in the target detection model.
6. A target detection device, characterized in that, include: A determination module is used to determine a target detection image, wherein the target detection image includes a sequence of images for a consecutive preset number of frames; The acquisition module is used to input the target detection image into the target detection model, identify the detection target, and acquire the first region where the detection target is located, wherein each frame of the target detection image corresponds to one first region; The judgment module is used to determine whether the detected target is in a stationary state or a moving state based on the change law of the cross-union ratio in the first region. For a target in a stationary state, the stationary duration is calculated. If it does not exceed the preset duration, the target is considered to be temporarily stationary; otherwise, the next step of analysis is performed. Input any frame of the target detection image into a similarity recognition network to determine whether the detected target has been registered; If the target being detected has not registered the information, it is determined that the target being detected is abnormal. If the detection target has already registered the information, determine whether the location of the detection target is within the first designated area; If the location of the detected target is not in the first designated area, determine a second duration for which the detected target is not in the first designated area; If the second duration is longer than the second preset duration, it is determined that the detection target is abnormal.
7. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.
8. A processor-readable storage medium, characterized in that, The processor-readable storage medium stores a computer program for causing the processor to perform the steps of the target detection method according to any one of claims 1 to 5.