An object determination method and device based on anomaly detection

Abstract

The application provides a target determination method and device based on anomaly detection, which are used to accurately detect targets that are difficult to classify. A first image is obtained; at least one target image sub-region in the first image is determined based on a target detection method; a first feature vector corresponding to the first image is determined; at least one anomaly image sub-region in the first image is determined based on the first feature vector and a plurality of sample feature vectors; for target image sub-regions and anomaly image sub-regions that have overlapping regions, whether to delete or retain the anomaly image sub-regions in the first image is determined based on the areas of the target image sub-regions and the areas of the anomaly image sub-regions; and the positions of the remaining anomaly image sub-regions in the first image are determined as the positions of targets to be detected in the first image. Targets that are difficult to classify can be accurately detected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a target determination method and device based on abnormality detection. BACKGROUND

[0002] In detecting specific information in an image, a target detection method is usually adopted. The target detection can detect specific targets in the image, such as people, vehicles, bags or other common categories. The target detection task in a related scene can be implemented by training a neural network. The target detection technology based on the neural network usually needs a high-precision sample data set as support. The sample data set in many scenes can be obtained by manual annotation, or the same category can be obtained by generalization training on an existing large public data set.

[0003] However, for some special scenes (for example, a scene of garbage accumulation in a corridor), specific information (for example, accumulated garbage) in the scene is difficult to classify, and enough sample data cannot be obtained. The method based on target detection cannot detect the specific information in the image. SUMMARY

[0004] The present application provides a target determination method and device based on abnormality detection, which can accurately detect a target that is difficult to classify.

[0005] To achieve the above object, the present application discloses a target determination method based on abnormality detection, comprising:

[0006] obtaining a first image of a scene region to be detected;

[0007] determining at least one target image sub-region in the first image based on a target detection method;

[0008] determining a first feature vector corresponding to the first image; determining at least one abnormal image sub-region in the first image based on the first feature vector and a plurality of pre-stored sample feature vectors; wherein the first image and the sample feature vector correspond to the same scene region, and the sample image does not include an abnormal image sub-region;

[0009] For the target image sub-region and the abnormal image sub-region with an overlapping region, based on the area of the target image sub-region and the area of the abnormal image sub-region, it is determined whether to delete or retain the abnormal image sub-region in the first image;

[0010] determining the position of the remaining abnormal image sub-region in the first image as the position of the target to be detected in the first image.

[0011] In an example, the determining the first feature vector corresponding to the first image comprises:

[0012] inputting the first image into a pre-trained first neural network model to obtain a plurality of first features; wherein the first neural network model is configured to extract features in an image;

[0013] determining the first feature vector corresponding to the first image based on the plurality of first features.

[0014] In an example, the determining the at least one abnormal image sub-region in the first image based on the first feature vector and a plurality of pre-stored sample feature vectors comprises:

[0015] determining a mean value and a covariance matrix corresponding to the plurality of sample feature vectors based on the plurality of sample feature vectors;

[0016] determining a difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors based on the first feature vector, the mean value and the covariance matrix; wherein each element in the difference value matrix represents a difference value of each first sub-region in the first image and each sample sub-region in a sample image;

[0017] comparing each element in the difference value matrix with a first threshold value, and determining an abnormal image sub-region based on a first sub-region corresponding to an element greater than or equal to the first threshold value.

[0018] In an example, the determining the difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors based on the first feature vector, the mean value and the covariance matrix comprises:

[0019] determining the difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors based on a Mahalanobis distance formula, the first feature vector, the mean value and the covariance matrix.

[0020] In an example, the determining whether to delete or retain the abnormal image sub-region in the first image based on the area of the target image sub-region and the area of the abnormal image sub-region comprises:

[0021] determining an intersection-over-union ratio of the target image sub-region and the abnormal image sub-region;

[0022] determining whether the intersection-over-union ratio is greater than or equal to a second threshold value;

[0023] if yes, deleting the abnormal image sub-region in the first image;

[0024] If no, the abnormal image sub-region is kept in the first image.

[0025] In one example, before determining the position of the remaining abnormal image sub-region in the first image as the position of the target to be detected in the first image, further comprising:

[0026] For each abnormal image sub-region in the first image, based on the first position of the abnormal image sub-region in the first image, determine whether there is an abnormal image sub-region at the first position of the second image; wherein the first image and the second image are acquired for the same scene region, and the acquisition time of the second image is earlier than that of the first image;

[0027] If no, the abnormal image sub-region is deleted in the first image;

[0028] If yes, the abnormal image sub-region is kept in the first image.

[0029] Embodiments of the present application disclose a target determination device based on abnormality detection, comprising:

[0030] An acquisition module is configured to acquire a first image of a scene region to be detected;

[0031] A target detection module is configured to determine at least one target image sub-region in the first image based on a target detection method;

[0032] An abnormality detection module is configured to determine a first feature vector corresponding to the first image, and determine at least one abnormal image sub-region in the first image based on the first feature vector and a plurality of sample feature vectors saved in advance; wherein the first image and a sample image corresponding to the sample feature vector are acquired for the same scene region, and the sample image does not include an abnormal image sub-region;

[0033] A target screening module is configured to, for a target image sub-region and an abnormal image sub-region with an overlapping region, determine whether to delete or keep the abnormal image sub-region in the first image based on the area of the target image sub-region and the area of the abnormal image sub-region, and determine the position of the remaining abnormal image sub-region in the first image as the position of the target to be detected in the first image.

[0034] In one example, the abnormality detection module is specifically configured to input the first image into a pre-trained first neural network model to obtain a plurality of first features; wherein the first neural network model is used to extract features in an image; and determine the first feature vector corresponding to the first image based on the plurality of first features.

[0035] In an example, the anomaly detection module is specifically configured to determine a mean value and a covariance matrix corresponding to the plurality of sample feature vectors based on the plurality of sample feature vectors; determine a difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors based on the first feature vector, the mean value, and the covariance matrix; wherein each element in the difference value matrix represents a difference value of each first sub-region in the first image and each sample sub-region in a sample image; compare each element in the difference value matrix with a first threshold value, and determine an abnormal image sub-region based on a first sub-region corresponding to an element greater than or equal to the first threshold value.

[0036] In an example, the anomaly detection module is specifically configured to determine a difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors based on a Mahalanobis distance formula, the first feature vector, the mean value, and the covariance matrix.

[0037] In an example, the anomaly detection module is specifically configured to determine an intersection-over-union ratio of the target image region and the abnormal image sub-region; determine whether the intersection-over-union ratio is greater than or equal to a second threshold value; if yes, delete the abnormal image sub-region in the first image; and if no, retain the abnormal image sub-region in the first image.

[0038] In an example, the anomaly detection module is specifically configured to determine, for each abnormal image sub-region in a first image, whether there is an abnormal image sub-region at a first position of the abnormal image sub-region in the first image based on the first position of the abnormal image sub-region in the first image; wherein the first image and a second image are acquired for a same scene region, and the second image is acquired earlier than the first image; if no, delete the abnormal image sub-region in the first image; and if yes, retain the abnormal image sub-region in the first image.

[0039] Embodiments of the present application provide an anomaly detection-based target determination apparatus, comprising a processor and a memory;

[0040] The memory is configured to store computer programs or instructions.

[0041] The processor is configured to execute part or all of the computer programs or instructions in the memory, and when the part or all of the computer programs or instructions are executed, is configured to implement the anomaly detection-based target determination method.

[0042] Embodiments of the present application provide a computer readable storage medium for storing a computer program, wherein the computer program comprises instructions for implementing the anomaly detection-based target determination method.

[0043] For the scene that the target category is not easy to distinguish (for example, the scene of the accumulation of garbage in the corridor), a large number of non-abnormal images are directly obtained as sample images, and the first feature vector of the first image actually collected is compared with the sample feature vector of the sample image, so as to filter out the abnormal image sub-region in the first image, and the abnormal image sub-region is taken as the target to be detected. Since the process of obtaining non-abnormal images is relatively simple compared to determining the target category in the abnormal image sub-region, the target to be detected is detected, and the accuracy of detection is improved. Through abnormal detection, the difficulty of defining complex categories is avoided, and the dependence on labeled supervised data sets is eliminated.

[0044] Further, in the abnormal detection, the target detection method is combined to eliminate the influence of the appearance of the conventional target (for example, the pedestrian and the animal) in the scene area (for example, the corridor) to be detected on the difficult-to-detect target (for example, the accumulated garbage), so that the detection accuracy of the difficult-to-detect target (for example, the accumulated garbage) can be improved. The false positive rate of the detection of the difficult-to-detect target (for example, the accumulated garbage) by only using abnormal detection can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0046] Figure 1 A target determination flowchart based on abnormal detection is provided for the present application;

[0047] Figure 2 A target determination system architecture diagram based on abnormal detection is provided for the present application;

[0048] Figure 3 A target determination device structure diagram based on abnormal detection is provided for the present application;

[0049] Figure 4 A target determination device structure diagram based on abnormal detection is provided for the present application. DETAILED DESCRIPTION

[0050] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0051] As shown in Figure 1 , a target determination method based on anomaly detection is introduced, which can be applied to electronic devices and includes the following steps:

[0052] Step 101: Obtain a first image of a scene area to be detected.

[0053] The first image is an image of the scene area to be detected, such as a corridor. A monitor is installed in the scene to be detected, which collects images or video streams of the scene area to be detected. The first image is an image collected by the monitor, or a frame image in a video stream, or an image obtained by scaling the image collected by the monitor, or an image obtained by scaling a frame image in the video stream collected by the monitor.

[0054] The present application can detect whether there is an abnormal image sub-area in the first image, which can be an image area of accumulated debris.

[0055] The first image can be an RGB three-channel image, and the size of the first image is, for example, 224*224.

[0056] Step 102: Determine a first feature vector corresponding to the first image.

[0057] In an example, the first image is input into a pre-trained first neural network model to obtain a plurality of first features; wherein the first neural network model is used to extract features in the image. Based on the plurality of first features, a first feature vector corresponding to the first image is determined.

[0058] The first neural network model is a ResNet (e.g. ResNet-50) neural network model, which can be pre-trained on ImageNet. The main structure framework of PaDiM can be used to integrate the plurality of first features to obtain the first feature vector corresponding to the first image.

[0059] Step 103: Based on the first feature vector and a plurality of sample feature vectors pre-stored, at least one abnormal image sub-area in the first image is determined.

[0060] The sample feature vector is a feature vector corresponding to a sample image. The first image and the sample image are captured for the same scene region, and the sample image does not include an abnormal image sub-region. The sample image has the same size as the first image, for example, 224*224. The sample feature vector is obtained in the same way as the first feature vector.

[0061] In an example, the mean and the covariance matrix corresponding to the plurality of sample feature vectors can be determined based on the plurality of sample feature vectors; then, the difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors can be determined based on the first feature vector, the mean, and the covariance matrix; next, each element in the difference value matrix is compared with a first threshold value; the abnormal image sub-region is determined based on the first sub-region corresponding to the abnormal element, wherein the abnormal element is an element greater than or equal to the first threshold value; if each element in the difference value matrix is less than the first threshold value, it is determined that there is no abnormal image sub-region in the first image. The first image may or may not include one or more abnormal image sub-regions.

[0062] The mean and the covariance matrix can be referred to as Gaussian distribution parameters, and the main structure framework of PaDiM can be used to determine the mean and the covariance matrix corresponding to the plurality of sample feature vectors based on the plurality of sample feature vectors. Each element in the difference value matrix represents the difference value between each first sub-region in the first image and each sample sub-region in the sample image. The position of the first sub-region corresponding to any element in the first image is the same as the position of the sample sub-region corresponding to the element in the sample image.

[0063] In an example, the difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors can be determined based on the Mahalanobis distance formula, the first feature vector, the mean, and the covariance matrix. For example, 1000 sample images are obtained in advance, and each sample image corresponds to a sample feature vector. Based on the 1000 sample feature vectors, the mean of the 1000 sample feature vectors and the covariance matrix between the 1000 sample feature vectors are determined. The difference value matrix is determined by the Mahalanobis distance formula D M (X) represents the difference value matrix, x represents the first feature vector, μ represents the mean, T represents the transpose, Σ represents the covariance matrix, and the superscript (-1) represents the inverse.

[0064] In an example, the abnormal image sub-region can be determined based on the first sub-region corresponding to the abnormal element. In this way, each first sub-region corresponding to the abnormal element can be determined as the abnormal image sub-region. If each first sub-region in the first image is small, the abnormal image sub-region determined in this way can be irregular and scattered. In another example, a circumscribed rectangle can be set on the first sub-region corresponding to the continuous abnormal element (it should be noted that the continuous here refers to the continuous first sub-region, not the continuous abnormal element). The circumscribed rectangle can include multiple first sub-regions. The region where the circumscribed rectangle is located can be determined as the abnormal image sub-region, and the abnormal image sub-region is a regular rectangular region.

[0065] Optionally, in order to reduce the complexity of determining the circumscribed rectangle, the circumscribed rectangle can be determined based on a mask. For example, before setting the circumscribed rectangle on the first sub-region corresponding to the continuous abnormal element, the pixel value of the pixel point of the first sub-region corresponding to the abnormal element can be set to 0, and the pixel value of the pixel point of the first sub-region corresponding to the non-abnormal element can be set to 1 or the original pixel value of the first image. Then, the circumscribed rectangle can be set on the region where the continuous pixel points are 0.

[0066] Step 104: determining at least one target image sub-region in the first image based on a target detection method.

[0067] The target corresponding to the target image sub-region can be a human body, an animal, a vehicle, or other targets that can be easily classified. The target image sub-region can exist in the first image, or can not exist in the first image.

[0068] For example, whether a human image sub-region exists in the first image can be detected by using an existing target detection algorithm Yolov5.

[0069] The target image sub-region can be a regular region, such as a rectangular region.

[0070] The sequence of steps 104 and 102 is not limited.

[0071] Step 105: for the target image sub-region and the abnormal image sub-region with an overlapping region, determining whether to delete or retain the abnormal image sub-region in the first image based on the area of the target image sub-region and the area of the abnormal image sub-region.

[0072] The general target image sub-region and the abnormal image sub-region can overlap, which can be complete overlap or partial overlap. For example, an area ratio can be determined based on an area of the target image sub-region and an area of the abnormal image sub-region, and the area ratio is used to indicate an influence of the target image sub-region on the abnormal image sub-region. The area ratio is compared with a second threshold value; when the area ratio is greater than or equal to the second threshold value, it is determined that the abnormal image sub-region is deleted in the first image; and when the area ratio is less than or equal to the second threshold value, it is determined that the abnormal image sub-region is retained in the first image.

[0073] In an example, the area ratio can be a ratio of an overlapping area of the target image sub-region and the abnormal image sub-region to an area of the abnormal image sub-region.

[0074] In an example, the area ratio can be a ratio of an overlapping area of the target image sub-region and the abnormal image sub-region to an area of the abnormal image sub-region excluding the area of the target image sub-region.

[0075] In an example, the area ratio can be an intersection over union of the target image sub-region and the abnormal image sub-region. The intersection over union is a ratio of an intersection and a union of two regions.

[0076] In a scene where a target class is not easy to distinguish (for example, a scene of a corridor with accumulated garbage), an image without an abnormality (for example, without accumulated garbage, pedestrians, animals, etc.) is used as a reference, and the abnormal image sub-region can be caused by accumulated garbage, pedestrians, animals, etc. When the area ratio is greater than or equal to the second threshold value, it is indicated to some extent that the abnormal image sub-region is caused by the presence of pedestrians, animals, etc., which is a false judgment for the accumulated garbage, and the abnormal image sub-region can be deleted in the first image. In the abnormality detection, a target detection method is combined to eliminate the influence of the presence of a conventional target (for example, a pedestrian, an animal) in a scene region to be detected (for example, a corridor) on a difficult-to-detect target (for example, accumulated garbage), which can improve the detection accuracy of the difficult-to-detect target (for example, accumulated garbage). The false positive rate of the detection of the difficult-to-detect target (for example, accumulated garbage) can be reduced by only using abnormality detection.

[0077] Step 106: determining a position of the remaining abnormal image sub-region in the first image as a position of a target to be detected in the first image.

[0078] For example, when there is an abnormal image sub-region (a target to be detected) in the first image, an alarm can be given.

[0079] For example, a position of the target to be detected (that is, the remaining abnormal image sub-region) in the first image can be output or marked.

[0080] If there is no target image sub-region in step 104, the remaining abnormal image sub-region in the first image is the abnormal image sub-region determined in step 103 based on the first feature vector and the plurality of sample feature vectors pre-stored.

[0081] For a scene in which the target category is not easy to distinguish (for example, a scene in which garbage is accumulated in a corridor), a large number of images without abnormalities are directly obtained as sample images in the present application, and the first feature vector of the first image actually collected is compared with the sample feature vector of the sample image, so as to screen out an abnormal image sub-region in the first image, and the abnormal image sub-region is taken as a target to be detected. Since the process of obtaining images without abnormalities is relatively simple compared with the determination of the target category in the abnormal image sub-region, the present application not only detects the target to be detected, but also improves the accuracy of detection. By means of abnormality detection, the difficulty of defining a complex category is avoided, and the dependence on a labeled supervised data set is eliminated.

[0082] Further, in the abnormality detection, a target detection method is combined to eliminate the influence of the appearance of a regular target (for example, a pedestrian or an animal) in a scene region (for example, a corridor) to be detected on a difficult-to-detect target (for example, accumulated garbage), so as to improve the detection accuracy of the difficult-to-detect target (for example, accumulated garbage). The false positive rate of the detection of the difficult-to-detect target (for example, accumulated garbage) by only using abnormality detection can be reduced.

[0083] In an optional example, before the position of the remaining abnormal image sub-region in the first image is determined as the position of the target to be detected in the first image, for each abnormal image sub-region in the first image, whether there is an abnormal image sub-region at the first position of the second image based on the first position of the abnormal image sub-region in the first image can be determined; wherein the first image and the second image are collected for the same scene region, and the acquisition time of the second image is earlier than that of the first image; if not, the abnormal image sub-region is deleted in the first image; if yes, the abnormal image sub-region is retained in the first image. This process can be performed before step 104 or after step 105.

[0084] The first image is an image acquired at the t+n moment, and the second image is an image acquired at the t moment; or the first image is an image acquired at the t+2n moment, and the second image is an image acquired at the t+n moment. The n is a hyperparameter, which can be modified according to requirements. The n can represent how long an abnormal image sub-region existing within a period of time is taken as a target to be detected (for example, accumulated garbage), and the n can be 8 seconds, 10 seconds, 15 seconds, etc.

[0085] With t time as the reference, if there is the abnormal image sub-area at t time, and the abnormal image sub-area still exists at t+n time, it is determined that the abnormal image sub-area is indeed the target to be detected (for example, accumulated sundries). The t time can be removed from the queue, and the t+n time is regarded as the reference of t+2n time, and the iterative detection result is updated as the time sequence advances. When the abnormality exists for a period of time, it is determined to be the target to be detected, which can eliminate the influence of pedestrians or some objects only passing through the video screen, and can reduce false alarms and high-frequency warnings.

[0086] As shown in Figure 2 , a target determination system architecture diagram based on abnormality detection is introduced. Video stream data under normal conditions is collected as a sample, and the real-time video stream of the corridor monitoring is compared with the sample to determine the abnormal image sub-area, so as to realize abnormality detection. The image sub-area where the conventional target is located is obtained through target detection. The influence of pedestrians, animals and other conventional targets is eliminated through target screening. The final result is output through time sequence tracking.

[0087] The following introduces a specific example:

[0088] Step 1: Obtain the video data collected by the monitoring installed in the corridor under the condition that there is no sundry accumulation, no pedestrians, animals and the like passing through the corridor. The video data is split into images and the detection area is delimited to produce a sample image set.

[0089] Step 2: Input the sample image set into the ResNet neural network model pre-trained on ImageNet to perform feature extraction. The sample feature vector Gaussian distribution parameters (such as mean and covariance matrix) are obtained by integrating the sample features through PaDiM and saved to the local for subsequent operations.

[0090] Step 3: Load the neural network model parameters and Gaussian distribution parameters, and access real-time video stream data. Get an image every n seconds (for example, 15s).

[0091] Step 4: For each image obtained, determine the abnormal image sub-area in the image through abnormality detection; determine the image sub-area where the conventional target such as pedestrians and animals is located through target detection; eliminate the influence of the conventional target such as pedestrians and animals in the image in combination with the results of abnormality detection and target detection.

[0092] Step 5: Based on the difference between the previous and subsequent two frames, finally output whether the target to be detected (for example, accumulated sundries) appears.

[0093] Repeat steps 3, 4 and 5 above to monitor and update the warning result in real time.

[0094] Based on the same technical concept as the above method, see Figure 3 , an object determination device based on anomaly detection is provided, comprising:

[0095] The acquisition module 31 is configured to acquire a first image of a scene area to be detected.

[0096] The target detection module 32 is configured to determine at least one target image sub-region in the first image based on a target detection method.

[0097] The anomaly detection module 33 is configured to determine a first feature vector corresponding to the first image, and determine at least one abnormal image sub-region in the first image based on the first feature vector and a plurality of pre-stored sample feature vectors. The sample image corresponding to the first image and the sample feature vector is acquired for the same scene area, and the sample image does not include an abnormal image sub-region.

[0098] The target screening module 34 is configured to determine whether to delete or retain the abnormal image sub-region in the first image based on the area of the target image sub-region and the area of the abnormal image sub-region for the target image sub-region and the abnormal image sub-region with overlapping regions. The position of the remaining abnormal image sub-region in the first image is determined as the position of the target to be detected in the first image.

[0099] In an example, the anomaly detection module 33 is specifically configured to input the first image into a pre-trained first neural network model to obtain a plurality of first features, wherein the first neural network model is used to extract features in the image; and determine a first feature vector corresponding to the first image based on the plurality of first features.

[0100] In an example, the anomaly detection module 33 is specifically configured to determine a mean value and a covariance matrix corresponding to a plurality of sample feature vectors based on the plurality of sample feature vectors; determine a difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors based on the first feature vector, the mean value, and the covariance matrix, wherein each element in the difference value matrix represents a difference value between each first sub-region in the first image and each sample sub-region in a sample image; compare each element in the difference value matrix with a first threshold value, and determine an abnormal image sub-region based on a first sub-region corresponding to an element greater than or equal to the first threshold value.

[0101] In an example, the anomaly detection module 33 is specifically configured to determine a difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors based on a Mahalanobis distance formula, the first feature vector, the mean value, and the covariance matrix.

[0102] In an example, the anomaly detection module 33 is specifically configured to determine an intersection-over-union of the target image region and the anomaly image sub-region; determine whether the intersection-over-union is greater than or equal to a second threshold value; if yes, delete the anomaly image sub-region in the first image; if no, retain the anomaly image sub-region in the first image.

[0103] In an example, the anomaly detection module 33 is specifically configured to determine an intersection-over-union of the target image region and the anomaly image sub-region; determine whether the intersection-over-union is greater than or equal to a second threshold value; if yes, delete the anomaly image sub-region in the first image; if no, retain the anomaly image sub-region in the first image.

[0104] Based on the same technical concept as the above method, see Figure 4 , an anomaly detection-based target determination apparatus is provided, which comprises a processor 401 and a memory 402, and optionally, a transceiver 403.

[0105] The memory 402 is configured to store computer programs or instructions.

[0106] The processor 401 is configured to execute part or all of the computer programs or instructions in the memory, and when the part or all of the computer programs or instructions are executed, to implement any of the anomaly detection-based target determination methods described above. For example, the transceiver 403 performs receiving and sending actions, and the processor 401 performs other actions in addition to the receiving and sending actions.

[0107] The embodiments of the present application provide a computer readable storage medium for storing a computer program, wherein the computer program comprises instructions for implementing any of the anomaly detection-based target determination methods.

[0108] The embodiments of the present application further provide a computer program product, which comprises computer program codes, and when the computer program codes are run on a computer, the computer can execute the anomaly detection-based target determination method provided above.

[0109] The embodiments of the present application further provide an anomaly detection-based target determination system, which comprises a monitoring device installed in a scene region to be detected and an electronic device for executing the anomaly detection-based target determination method.

[0110] In addition, the processor mentioned in the embodiments of the present application can be a central processing unit (CPU), a baseband processor, the baseband processor and the CPU can be integrated together or separated, and can also be a network processor (NP) or a combination of the CPU and the NP. The processor can further include a hardware chip or other general-purpose processors. The hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) and other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components or any combination thereof. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0111] The memory mentioned in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM) or a flash memory. The volatile memory can be a random access memory (Random Access Memory, RAM) used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synchlink DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM). It should be noted that the memory described in the present application is intended to include but not limited to these and any other suitable types of memory.

[0112] The transceiver mentioned in the embodiments of the present application can include a separate transmitter and / or a separate receiver, or can be a transmitter and a receiver integrated together. The transceiver can work under the indication of the corresponding processor. Alternatively, the transmitter can correspond to the transmitter in the physical device, and the receiver can correspond to the receiver in the physical device.

[0113] Those skilled in the art can realize that, in combination with the method steps and units described in the embodiments disclosed herein, the methods and units can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the steps and components of the embodiments have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0114] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely illustrative. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between the units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms of connection.

[0115] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0116] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0117] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0118] The "and / or" in the present application describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. The plurality of the present application refers to two or more. In addition, it should be understood that in the description of the present application, "first", "second", and the like are used only to distinguish the description purpose, and cannot be understood as indicating or implying relative importance, nor can it be understood as indicating or implying order.

[0119] Although the preferred embodiments of the present application have been described, those skilled in the art who are familiar with the basic inventive concept can make additional changes and modifications to the embodiments. Therefore, the appended claims are intended to include the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0120] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. An anomaly detection based target determination method, characterized in that, The method comprises: acquiring a first image of a scene region to be detected; determining at least one target image sub-region in the first image based on a target detection method; determining a first feature vector corresponding to the first image; determining at least one abnormal image sub-region in the first image based on the first feature vector and a plurality of pre-stored sample feature vectors; wherein the first image and the sample feature vectors correspond to the same scene region, and the sample image does not include an abnormal image sub-region; for the target image sub-region and the abnormal image sub-region with an overlapping region, determining whether to delete or retain the abnormal image sub-region in the first image based on the area of the target image sub-region and the area of the abnormal image sub-region; determining the position of the remaining abnormal image sub-region in the first image as the position of the target to be detected in the first image.

2. The method of claim 1, wherein, The method further comprises: determining a mean value and a covariance matrix corresponding to the plurality of sample feature vectors based on the plurality of sample feature vectors; determining a difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors based on the first feature vector, the mean value, and the covariance matrix; wherein each element in the difference value matrix represents a difference value between each first sub-region in the first image and each sample sub-region in the sample image; comparing each element in the difference value matrix with a first threshold value, and determining an abnormal image sub-region based on a first sub-region corresponding to an element greater than or equal to the first threshold value.

3. The method of claim 2, wherein, The method further comprises: determining the difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors based on a Mahalanobis distance formula, the first feature vector, the mean value, and the covariance matrix.

4. The method of claim 1, wherein, The method further comprises: determining an intersection-over-union ratio of the target image sub-region and the abnormal image sub-region; determining whether the intersection-over-union ratio is greater than or equal to a second threshold value; if yes, deleting the abnormal image sub-region in the first image; if no, retaining the abnormal image sub-region in the first image.

5. The method of claim 1, wherein, The method further comprises: for each abnormal image sub-region in the first image, determining whether there is an abnormal image sub-region at a first position of the second image based on the first position of the abnormal image sub-region in the first image; wherein the first image and the second image are acquired for the same scene region, and the second image is acquired earlier than the first image. If no, the abnormal image sub-region is deleted in the first image; If yes, the abnormal image sub-region is kept in the first image.

6. An anomaly detection based target determination apparatus, characterized by, Comprise: An acquisition module is configured to acquire a first image of a scene region to be detected; A target detection module is configured to determine at least one target image sub-region in the first image based on a target detection method; An abnormality detection module is configured to determine a first feature vector corresponding to the first image, and determine at least one abnormal image sub-region in the first image based on the first feature vector and a plurality of pre-stored sample feature vectors; the first image and the sample feature vectors correspond to the same scene region, and the sample image does not include an abnormal image sub-region; A target screening module is configured to, for the target image sub-region and the abnormal image sub-region with an overlapping region, determine whether to delete or keep the abnormal image sub-region in the first image based on an area of the target image sub-region and an area of the abnormal image sub-region, and determine a position of a remaining abnormal image sub-region in the first image as a position of a target to be detected in the first image.

7. The apparatus of claim 6, wherein, The abnormality detection module is specifically configured to determine a mean value and a covariance matrix corresponding to the plurality of sample feature vectors based on the plurality of sample feature vectors, determine a difference value matrix of the first feature vector and the plurality of pre-stored sample feature vectors based on the first feature vector, the mean value, and the covariance matrix, wherein each element in the difference value matrix represents a difference value between each first sub-region in the first image and each sample sub-region in a sample image, and compare each element in the difference value matrix with a first threshold value to determine an abnormal image sub-region based on a first sub-region corresponding to an element greater than or equal to the first threshold value.

8. The apparatus of claim 6, wherein, Further comprise: A time sequence tracking module is configured to, for each abnormal image sub-region in the first image, determine whether there is an abnormal image sub-region at a first position of the second image based on a first position of the abnormal image sub-region in the first image; if no, the abnormal image sub-region is deleted in the first image; If yes, the abnormal image sub-region is kept in the first image; wherein the first image and the second image are acquired for the same scene region, and the second image is acquired earlier than the first image.

9. An anomaly detection based target determination apparatus, characterized by, Comprise a processor and a memory; The memory is configured to store computer programs or instructions; The processor is configured to execute part or all of the computer programs or instructions in the memory, and when the part or all of the computer programs or instructions are executed, the processor is configured to implement the method in any one of claims 1-5.

10. A computer-readable storage medium, characterized in that, A computer program is stored, and the computer program comprises instructions for implementing the method in any one of claims 1-5. A computer program is stored, and the computer program comprises instructions for implementing the method in any one of claims 1-5.

Images