A method and device for abnormal identification

Abstract

The present invention is applicable to the technical field of image recognition, and provides a method and device for abnormal recognition, wherein the method for abnormal recognition includes: extracting real-time video frames through a preset sampling frequency, and performing grayscale comparison with standard comparison video frames, so as to efficiently identify and distinguish abnormal image areas. First, the first abnormal image area is extracted by grayscale comparison, and then the area is expanded and compared pixel by pixel to form a second abnormal image area. This process can accurately capture abnormal situations and significantly improve the adaptability of deep learning models for low-computing power surveillance cameras. When the displacement of the abnormal area exceeds the first threshold, it is sent to a cloud server for object classification. The cloud server processes the image area to be identified based on the classification algorithm, and finally transmits the recognition results (including animals and non-animals) to the user terminal, realizing efficient and accurate real-time monitoring and abnormal situation classification.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and in particular relates to a method and device for abnormality recognition. Background Art

[0002] Anomaly recognition of surveillance cameras is a key function in video surveillance systems, which can help identify and alert potential security threats or abnormal events. Anomaly recognition of surveillance cameras relies on machine learning algorithms. Machine learning algorithms can learn and improve prediction or classification capabilities from data. They include supervised learning and unsupervised learning. Deep learning models (such as convolutional neural networks, LSTM) are used to identify complex abnormal behaviors. The model is trained with a large amount of normal and abnormal data to learn the features that distinguish between normal and abnormal.

[0003] However, deep learning models require surveillance cameras to provide a large amount of computing power and cannot distinguish abnormal situations such as people, pets, and non-animals. This is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and device for abnormality identification to solve the technical problem of being unable to well distinguish abnormal situations such as people, pets and non-animals.

[0005] A first aspect of an embodiment of the present invention provides a method for abnormality identification, the method for abnormality identification comprising:

[0006] Extracting real-time video frames according to a preset sampling frequency, and obtaining standard comparison video frames; the standard comparison video frames include historical video frames that are determined to be in a normal state;

[0007] Performing grayscale comparison between the current real-time video frame and the standard comparison video frame to obtain a first abnormal image region;

[0008] Expanding the first abnormal image region to obtain an image region to be identified;

[0009] Based on the image area to be identified, a pixel-by-pixel comparison is performed between the subsequent real-time video frame and the standard comparison video frame to obtain a second abnormal image area;

[0010] If the displacement between the first abnormal image area and the second abnormal image area is greater than a first threshold, sending the image area to be identified to a cloud server;

[0011] The cloud server performs object classification on the image area to be identified to obtain an identification result, and sends the identification result to the user terminal; the identification result includes animals and non-animals.

[0012] Furthermore, the step of performing grayscale comparison between the current real-time video frame and the standard comparison video frame to obtain the first abnormal image area includes:

[0013] The current real-time video frame is divided into a plurality of first sub-images, and the standard comparison video frame is divided into a plurality of second sub-images;

[0014] Calculate a first grayscale mean and a second grayscale mean respectively corresponding to a first sub-image and a second sub-image in the same area;

[0015] If a first difference between the first grayscale mean and the second grayscale mean is greater than a second threshold, matching other sub-images adjacent to the first sub-image to form a first image region;

[0016] In the first image area, calculating a second grayscale mean and a third grayscale mean respectively corresponding to the first sub-image and the second sub-image in the same area;

[0017] Calculate a second difference between the second grayscale mean and the third grayscale mean of the same area;

[0018] If the number of the second difference values ​​greater than the second threshold is greater than a preset number, match other sub-images adjacent to the first image area to form a second image area, and use the second image area as the first image area, and repeatedly execute the steps of calculating the second grayscale mean and the third grayscale mean corresponding to the first sub-image and the second sub-image of the same area in the first image area and subsequent steps until the number of the second difference values ​​greater than the second threshold is no greater than a preset number, so as to obtain the first abnormal image area; the preset number includes 0.

[0019] Furthermore, the step of classifying the objects in the image area to be identified by the cloud server to obtain the identification result and sending the identification result to the user terminal includes:

[0020] Extracting the color, number of pixels and aspect ratio of the current object in the first abnormal image area;

[0021] Calculating the moving speed of the object in the image area to be identified;

[0022] The color, number of pixels, aspect ratio and moving speed of the current object are constructed as a real-time feature vector;

[0023] Obtain the historical feature vector corresponding to the historical object;

[0024] Calculating the similarity between the real-time feature vector and the historical feature vector;

[0025] If the similarity is greater than a third threshold, confirming that the recognition result is a historical object;

[0026] If the similarity is not greater than a third threshold, classifying the object according to the change in contour pixels to obtain a recognition result;

[0027] The recognition result is sent to the user terminal.

[0028] Furthermore, the step of calculating the moving speed of the object in the image area to be identified includes:

[0029] Extracting a target real-time video frame at a preset interval from a plurality of real-time video frames;

[0030] Obtaining position parameters corresponding to each object in multiple target real-time video frames;

[0031] Calculate the difference between the position parameters of objects in adjacent sorted real-time video frames;

[0032] The instantaneous speed is obtained by dividing the difference value by the time interval; the time interval refers to the interval between two adjacent target real-time video frames;

[0033] The median of the multiple instantaneous speeds is obtained, and the average value of the multiple instantaneous speeds greater than the median is used as the moving speed.

[0034] Furthermore, if the similarity is not greater than a third threshold, the step of classifying the object according to the contour pixel change to obtain the recognition result includes:

[0035] If the similarity is not greater than a third threshold, calculating the contour change amount;

[0036] If the contour change is greater than a fourth threshold, confirming that the recognition result is a pet or a person;

[0037] If the contour change is not greater than the fourth threshold, it is confirmed that the identification result is not an animal.

[0038] Furthermore, if the similarity is not greater than a third threshold, the step of calculating the contour change comprises:

[0039] If the similarity is not greater than a third threshold, then calculating the number of pixels of the object contour in each real-time video frame;

[0040] The pixel quantity difference of the object contour between adjacent real-time video frames is calculated, and a plurality of pixel quantity differences are added to obtain the contour pixel change amount.

[0041] Furthermore, if the contour change amount is greater than a fourth threshold, the step of confirming that the recognition result is a pet or a person includes:

[0042] Extracting a current object in a first abnormal image region in a plurality of real-time video frames;

[0043] Superimposing the current object with the white layer to obtain an image to be recognized;

[0044] Setting the pixel points of the current object in the image to be identified to the first pixel value, and setting the pixel points of the non-current object in the image to be identified to the second pixel value;

[0045] Extracting the midpoint of the current object in the horizontal coordinate direction;

[0046] Based on the longitudinal coordinate direction of the current object and the midpoint, forming a segmentation line;

[0047] Overlap the sub-image areas on both sides of the segmentation line and count the difference points; the difference point refers to the pixel values ​​of two overlapping pixel points being the first pixel value and the second pixel value respectively;

[0048] Calculate statistical difference points corresponding to multiple real-time video frames respectively;

[0049] If the plurality of statistical difference points are all greater than a third threshold, then the identification result is confirmed to be a pet;

[0050] If the plurality of statistical difference points are not all greater than the third threshold, the recognition result is confirmed to be a person.

[0051] A second aspect of an embodiment of the present invention provides an abnormality identification device, including:

[0052] An acquisition unit, configured to extract real-time video frames according to a preset sampling frequency, and acquire standard comparison video frames; the standard comparison video frames include historical video frames that are determined to be in a normal state;

[0053] A first comparison unit, used for performing grayscale comparison between the current real-time video frame and the standard comparison video frame to obtain a first abnormal image region;

[0054] An expansion unit, used for expanding the first abnormal image region to obtain an image region to be identified;

[0055] A second comparison unit is used to compare the subsequent real-time video frame with the standard comparison video frame pixel by pixel based on the image area to be identified, so as to obtain a second abnormal image area;

[0056] a judgment unit, configured to send the image area to be identified to a cloud server if the displacement between the first abnormal image area and the second abnormal image area is greater than a first threshold;

[0057] The recognition unit is used to classify the objects in the image area to be recognized through the cloud server to obtain recognition results, and send the recognition results to the user terminal; the recognition results include animals and non-animals.

[0058] A third aspect of an embodiment of the present invention provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for abnormality identification described in the first aspect when executing the computer program.

[0059] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for abnormality identification described in the first aspect are implemented.

[0060] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: by extracting real-time video frames according to a preset sampling frequency and obtaining standard comparison video frames (i.e., historical video frames that are judged to be in a normal state), the present solution can realize the detection of real-time abnormal situations. Grayscale comparison between real-time video frames and standard comparison video frames can effectively identify the first abnormal image area, thereby improving the real-time and accuracy of abnormal detection. After the first abnormal image area is identified, the image area to be identified is obtained by area expansion, and the second abnormal image area is obtained by pixel-by-pixel comparison in subsequent real-time video frames. This process ensures the accurate positioning of the abnormal area and reduces the false detection rate. When the displacement between the first abnormal image area and the second abnormal image area is greater than the first threshold, the system sends the image area to be identified to the cloud server. The cloud server uses its computing power to classify objects in the image area to be identified, which can well distinguish between people, pets and non-animals, thereby providing accurate recognition results. Through cloud server processing, local computing power limitations can be broken. In summary, this technical solution improves the computing power adaptability, accurate anomaly detection, effective data processing and clear identification feedback of surveillance cameras, and effectively solves the application difficulties of surveillance cameras in environments with high computing power requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 A schematic flow chart of an abnormality identification method provided by the present invention is shown;

[0063] Figure 2 A schematic diagram of an abnormality identification device provided by an embodiment of the present invention is shown;

[0064] Figure 3 A schematic diagram of a terminal device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0065] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0066] The embodiments of the present invention provide a method and device for abnormality identification to solve the technical problem that abnormal situations such as people, pets and non-animals cannot be well distinguished.

[0067] First, the present invention provides a method for abnormality identification. Figure 1 , Figure 1 FIG. 2 shows a schematic flow chart of a method for abnormality identification provided by the present invention. Figure 1 As shown, the method for abnormality identification may include the following steps:

[0068] Step 101: extracting real-time video frames according to a preset sampling frequency, and obtaining standard comparison video frames; the standard comparison video frames include historical video frames that are determined to be in a normal state;

[0069] The abnormality recognition in this embodiment is to identify whether there is an object (pet or person) illegally intruding in the local monitoring area by the camera. Since the movement of the object is continuous, in order to reduce unnecessary calculation, this embodiment extracts real-time video frames according to the preset sampling frequency.

[0070] By acquiring the standard comparison video frame of the normal state, the real-time video frame is compared with the grayscale to identify the suspected abnormal area. The normal state includes no object entering, and the abnormal state includes animal entering or non-animal entering.

[0071] Step 102: performing grayscale comparison between the current real-time video frame and the standard comparison video frame to obtain a first abnormal image region;

[0072] The current real-time video frame refers to the real-time video frame ranked first among the multiple real-time video frames obtained. Specifically, step 102 specifically includes steps 1021 to 1026:

[0073] Step 1021: dividing the current real-time video frame into a plurality of first sub-images, and dividing the standard comparison video frame into a plurality of second sub-images;

[0074] The current real-time video frame and the standard comparison video frame are respectively divided into a plurality of sub-images so as to compare them block by block and locate the first abnormal image area.

[0075] Step 1022: Calculate the first grayscale mean and the second grayscale mean corresponding to the first sub-image and the second sub-image in the same area respectively;

[0076] For each sub-image area, a first grayscale mean value in the current real-time video frame and a second grayscale mean value in the standard comparison video frame are calculated.

[0077] Step 1023: if a first difference between the first grayscale mean and the second grayscale mean is greater than a second threshold, matching other sub-images adjacent to the first sub-image to form a first image region;

[0078] The difference between the first grayscale mean and the second grayscale mean of the same area is calculated. If the difference is greater than the preset second threshold, it means that the brightness of the area has changed significantly and there may be an abnormality. This difference calculation can identify significant differences between the current video frame and the standard comparison video frame.

[0079] If the grayscale mean difference of a sub-image is greater than the second threshold, other sub-images adjacent to the sub-image are also included in the inspection to form a larger image area (the first image area) to achieve "circle-by-circle diffusion" to find the complete abnormal area.

[0080] Step 1024: in the first image region, calculating the second grayscale mean and the third grayscale mean corresponding to the first sub-image and the second sub-image in the same region respectively;

[0081] Step 1025: Calculate a second difference between the second grayscale mean and the third grayscale mean of the same area;

[0082] Step 1026: If the number of the second difference values ​​that are greater than the second threshold is greater than a preset number, match other sub-images adjacent to the first image area to form a second image area, and use the second image area as the first image area, and loop through the steps of calculating the second grayscale mean and the third grayscale mean corresponding to the first sub-image and the second sub-image of the same area in the first image area and subsequent steps until the number of the second difference values ​​that are greater than the second threshold is no greater than a preset number, thereby obtaining the first abnormal image area; the preset number includes 0.

[0083] According to the result of the second difference, if the abnormal situation persists and exceeds the preset number of thresholds, other sub-images adjacent to the current abnormal area are also included in the inspection, and the abnormal area continues to be expanded. In this way, larger abnormal areas can be gradually identified until the abnormal area no longer expands significantly. When the process of expanding the abnormal area reaches a certain stable state, that is, the number of second difference values ​​greater than the second threshold value no longer exceeds the preset number (that is, the number of grayscale mean differences of other adjacent sub-images that are not greater than the threshold value is 0), the expansion is stopped and the final abnormal image area is determined to achieve "circle-by-circle diffusion" to find the first abnormal image area.

[0084] In this embodiment, by performing a grayscale comparison between a real-time video frame and a standard comparison video frame, the abnormal image area can be effectively identified and located. Specifically, each frame of video is first divided into multiple sub-images, and the corresponding grayscale means are calculated to identify areas with significant grayscale differences. If the grayscale mean difference in a certain area exceeds a set threshold, the adjacent areas are further expanded and matched to form a larger abnormal image area. By repeating the above steps and judging based on the number of grayscale differences and the preset value, the scope of the abnormal area can be gradually and accurately determined. This method effectively improves the accuracy and robustness of anomaly detection, and is particularly suitable for rapid response to abnormal behaviors or objects in real-time video monitoring and image analysis.

[0085] Step 103: Expanding the first abnormal image region to obtain an image region to be identified;

[0086] In order to reduce unnecessary calculation amount, this embodiment performs expansion based on the first abnormal image region to obtain the image region to be identified, and subsequent steps all perform image processing based on the image region to be identified.

[0087] Since the object is mobile, in order to capture the subsequent motion state of the object, the first abnormal image area needs to be expanded.

[0088] Step 104: Based on the image area to be identified, a pixel-by-pixel comparison is performed between the subsequent real-time video frame and the standard comparison video frame to obtain a second abnormal image area;

[0089] The processing logic of step 104 is similar to that of step 102 and will not be described in detail here.

[0090] Step 105: If the displacement between the first abnormal image region and the second abnormal image region is greater than a first threshold, the image region to be identified is sent to a cloud server;

[0091] If the displacement is greater than the first threshold, it means that the object may be a pet or a person and needs further identification, so the image area to be identified is sent to the cloud server for identification processing.

[0092] Step 106: Classify the objects in the image area to be identified by the cloud server to obtain identification results, and send the identification results to the user terminal; the identification results include animals and non-animals.

[0093] Specifically, step 106 specifically includes steps 1061 to 1068:

[0094] Step 1061: extracting the color, number of pixels and aspect ratio of the current object in the first abnormal image area;

[0095] The purpose of this step is to extract the basic features of the current object from the abnormal image area. Color is used to distinguish the appearance features of different objects, the number of pixels reflects the size of the object, and the aspect ratio is used to describe the shape of the object. The extraction of these features provides the basis for the subsequent feature vector construction.

[0096] Step 1062: Calculate the moving speed of the object in the image area to be identified;

[0097] By comparing image frames or using motion detection algorithms, the speed of an object can be calculated. This feature can further differentiate between stationary and moving objects and help identify objects with different motion patterns.

[0098] Specifically, step 1062 specifically includes steps A1 to A5:

[0099] Step A1: extracting a target real-time video frame at a preset interval from a plurality of real-time video frames;

[0100] Since the motion of the object is continuous, in order to further reduce the amount of calculation, this embodiment extracts target real-time video frames at preset intervals from multiple real-time video frames.

[0101] Step A2: obtaining position parameters corresponding to each object in a plurality of target real-time video frames;

[0102] Step A3: Calculating the difference between the position parameters of objects in adjacent sorted real-time video frames;

[0103] Calculate the difference between the object's position parameters in adjacent frames (for example, frame n and frame n+1). This can determine the displacement of the object between the two frames, providing the necessary information for calculating the speed.

[0104] Step A4: Dividing the difference by the time interval to obtain the instantaneous speed; the time interval refers to the interval between two adjacent target real-time video frames;

[0105] The calculated displacement difference is divided by the time interval (i.e., the time difference between two frames) to obtain the instantaneous velocity.

[0106] Step A5: Obtain the median of the multiple instantaneous speeds, and take the average value of the multiple instantaneous speeds greater than the median as the moving speed.

[0107] Since an object may stop and go during its movement, in order to obtain the normal movement speed, the obviously smaller instantaneous speed is eliminated, and the average of multiple instantaneous speeds greater than the median is taken as the moving speed.

[0108] In this embodiment, by extracting the target frame at a preset interval from multiple real-time video frames and obtaining the object position parameters, the difference of the object position parameters in adjacent video frames is calculated. By dividing these differences with the time interval, the instantaneous speed is obtained. Then, the scheme calculates the median of multiple instantaneous speeds and averages the instantaneous speeds greater than the median as the final moving speed. This method can effectively reduce the fluctuation of the instantaneous speed and provide a more stable and reliable moving speed measurement result.

[0109] Step 1063: constructing the color, number of pixels, aspect ratio and moving speed of the current object into a real-time feature vector;

[0110] This step combines the features extracted above into a multi-dimensional feature vector, including color, number of pixels, aspect ratio, and movement speed. These features combined can more comprehensively describe the properties of the object and provide information for accurate identification.

[0111] Step 1064: Obtain the historical feature vector corresponding to the historical object;

[0112] Feature vectors related to historically identified objects are extracted from a server or database. These historical features are used to compare the benchmark data of the current object to help identify whether the current object matches an object in the historical record.

[0113] Step 1065: Calculate the similarity between the real-time feature vector and the historical feature vector;

[0114] The similarity calculation method (such as cosine similarity, Euclidean distance, etc.) is used to evaluate the similarity between the feature vector of the current object and the historical feature vector. The degree of similarity determines whether the current object is consistent with the known historical object.

[0115] Step 1066: If the similarity is greater than a third threshold, confirming that the recognition result is a historical object;

[0116] If the similarity between the real-time feature vector and the historical feature vector exceeds a set threshold, the current object can be determined to be a known historical object, thereby avoiding repeated classification.

[0117] Step 1067: if the similarity is not greater than the third threshold, classify the object according to the contour pixel change to obtain a recognition result;

[0118] If the similarity does not reach the threshold, it means that the current object may be a new or unrecognized object. At this time, the object's contour pixel changes (such as shape features or appearance changes) are further analyzed for classification to determine the object type. The specific principles are as follows:

[0119] Specifically, step 1067 specifically includes steps B1 to B3:

[0120] Step B1: if the similarity is not greater than a third threshold, calculating the contour change;

[0121] Specifically, step B1 specifically includes: if the similarity is not greater than the third threshold, calculating the number of pixels of the object contour in each real-time video frame; calculating the difference in the number of pixels of the object contour between adjacent real-time video frames, and adding multiple pixel number differences to obtain the contour pixel change.

[0122] The number of pixels of the object contour refers to the number of edge pixel points of the object. The difference in the number of edge pixel points between adjacent real-time video frames is calculated respectively, and the pixel number difference is added as the contour pixel change amount. It can be understood that the contour pixel change amount is composed of the total amount of changes in the number of edge pixel points corresponding to multiple real-time video frames.

[0123] Step B2: if the contour change is greater than a fourth threshold, confirming that the recognition result is a pet or a person;

[0124] For pets and people, their overall shapes will change greatly when they move. However, the overall shape of non-animals will not change or will change slightly. The change in shape will inevitably lead to a change in the number of edge pixels. Therefore, based on this rule, this embodiment compares the contour change with the second threshold, and identifies objects with contour changes greater than the second threshold as pets or people. Objects with contour changes not greater than the second threshold are identified as non-living things. Among them, further judgment is required for pets or people, and the specific logic is as follows:

[0125] Specifically, step B2 specifically includes steps B21 to B29:

[0126] Step B21: extracting the current object in the first abnormal image area in multiple real-time video frames;

[0127] Step B22: superimposing the current object with the white layer to obtain an image to be recognized;

[0128] In order to better capture the characteristic information of the current object, the current object is superimposed with the white layer to facilitate subsequent processing and analysis.

[0129] Step B23: setting the pixel points of the current object in the image to be identified as the first pixel value, and setting the pixel points of non-current objects in the image to be identified as the second pixel value;

[0130] The current object area and the non-current object area are binarized to better explore the object feature information.

[0131] Step B24: extracting the midpoint of the current object in the horizontal coordinate direction;

[0132] Step B25: forming a segmentation line based on the longitudinal coordinate direction of the current object and the midpoint;

[0133] Step B26: Overlap the sub-image areas on both sides of the segmentation line and count the difference points; the difference point refers to the pixel values ​​of two overlapping pixel points being the first pixel value and the second pixel value respectively;

[0134] Overlap the sub-image areas on both sides of the segmentation line to obtain a plurality of corresponding pixel pairs. If the pixel values ​​of the pixel pairs are the first pixel value and the second pixel value, they are used as difference points. If the pixel values ​​of the pixel pairs are the same, they are not used as difference points.

[0135] Step B27: Calculate the statistical difference points corresponding to each of the multiple real-time video frames respectively;

[0136] Step B28: If the plurality of statistical difference points are all greater than the third threshold, confirming that the identification result is a pet;

[0137] Since the human body usually walks upright on two legs, it will be symmetrical in the longitudinal direction at multiple times. However, pets usually walk on four limbs, so they will be asymmetrical in the longitudinal direction. Therefore, the size relationship between the difference point and the third threshold can be statistically analyzed, and the object corresponding to the statistical difference point greater than the third threshold is confirmed as a pet.

[0138] Step B29: If the plurality of statistical difference points are not all greater than a third threshold, then the recognition result is confirmed to be a person.

[0139] Since the human body includes a standing phase and a swinging phase during walking, when in the standing phase, the longitudinal axis direction of the image area is in a symmetrical state. When in the swinging phase, the longitudinal axis direction of the image area is in an asymmetrical state. Therefore, if the statistical difference point is not uniformly greater than the third threshold, the recognition result is confirmed to be a person.

[0140] In this embodiment, the current object in the first abnormal image area is extracted from multiple real-time video frames, and the object is superimposed with the white layer to generate an image to be identified. This process highlights the characteristics of the current object. By setting the pixel point of the current object in the image to be identified to the first pixel value and the pixel point of the non-current object to the second pixel value, the contrast between the object and the background is further enhanced. Subsequently, based on the midpoint of the horizontal coordinate and the vertical coordinate position of the current object, a segmentation line is formed to segment the object into two left and right sub-image areas. The sub-image areas on both sides of the segmentation line are overlapped, and the difference points are counted (that is, the pixel values ​​of the two overlapping pixels are the first pixel value and the second pixel value, respectively). This process effectively captures the symmetry and structural characteristics of the image, which is convenient for subsequent recognition and analysis. The statistical difference points corresponding to each of the multiple real-time video frames are calculated respectively. If these difference points all exceed the set third threshold, the recognition result is confirmed to be a pet; otherwise, the recognition result is a person. Through the implementation of this technical solution, real-time and accurate recognition of pets and people is achieved. In this embodiment, there is no need to use a large deep learning model, and accurate recognition effect can be achieved with less computing power. Compared with traditional deep learning models, this implementation has the advantages of low computing power and high accuracy.

[0141] Step B3: If the contour change is not greater than the fourth threshold, it is confirmed that the identification result is not an animal.

[0142] Step 1068: Send the recognition result to the user terminal.

[0143] The recognition result is sent to the user terminal through the cloud server. The user terminal can receive the recognition result for subsequent processing or display.

[0144] In this embodiment, by extracting the features of the object in the image area, such as color, number of pixels, aspect ratio, and moving speed, and constructing a real-time feature vector, and comparing the similarity with the historical feature vector, the object can be accurately identified. If the similarity between the real-time feature vector and the historical feature vector is greater than a preset threshold, the system will confirm that the recognition result is a known historical object; otherwise, the object is classified by the change in the contour pixels. Finally, the recognition result will be sent to the user terminal to ensure that the user obtains timely and accurate object recognition information.

[0145] In this embodiment, by extracting real-time video frames according to a preset sampling frequency and obtaining standard comparison video frames (i.e., historical video frames that are judged to be in a normal state), this solution can realize the detection of real-time abnormal situations. Grayscale comparison between real-time video frames and standard comparison video frames can effectively identify the first abnormal image area, thereby improving the real-time and accuracy of abnormal detection. After identifying the first abnormal image area, the image area to be identified is obtained by area expansion, and the second abnormal image area is obtained by pixel-by-pixel comparison in subsequent real-time video frames. This process ensures the accurate positioning of the abnormal area and reduces the false detection rate. When the displacement between the first abnormal image area and the second abnormal image area is greater than the first threshold, the system sends the image area to be identified to the cloud server. The cloud server uses its computing power to classify objects in the image area to be identified, thereby providing accurate recognition results. Through processing by the cloud server, the local computing power limitation can be broken. In summary, this technical solution improves the computing power adaptation capability, accurate abnormality detection, effective data processing and clear recognition feedback of the surveillance camera, and effectively solves the application problems of the surveillance camera in an environment with high computing power requirements.

[0146] like Figure 2 The present invention provides a device for abnormal identification, see Figure 2 , Figure 2 A schematic diagram of an abnormality identification device provided by the present invention is shown, Figure 2 The device for abnormality identification includes:

[0147] The acquisition unit 21 is used to extract real-time video frames according to a preset sampling frequency, and acquire standard comparison video frames; the standard comparison video frames include historical video frames that are determined to be in a normal state;

[0148] A first comparison unit 22, used for performing grayscale comparison between the current real-time video frame and the standard comparison video frame to obtain a first abnormal image region;

[0149] An expansion unit 23, configured to expand the first abnormal image region to obtain an image region to be identified;

[0150] A second comparison unit 24 is used to perform pixel-by-pixel comparison between the subsequent real-time video frame and the standard comparison video frame based on the image area to be identified, so as to obtain a second abnormal image area;

[0151] A judgment unit 25, configured to send the image area to be identified to a cloud server if the displacement between the first abnormal image area and the second abnormal image area is greater than a first threshold;

[0152] The identification unit 26 is used to classify the objects in the image area to be identified through the cloud server to obtain the identification result, and send the identification result to the user terminal; the identification result includes animals and non-animals.

[0153] The present invention provides a device for abnormal identification. By extracting real-time video frames according to a preset sampling frequency and obtaining standard comparison video frames (i.e., historical video frames judged to be in a normal state), the scheme can realize the detection of real-time abnormal situations. Grayscale comparison is performed between real-time video frames and standard comparison video frames, and the first abnormal image area can be effectively identified, thereby improving the real-time and accuracy of abnormal detection. After the first abnormal image area is identified, the image area to be identified is obtained by area expansion, and the second abnormal image area is obtained by pixel-by-pixel comparison in subsequent real-time video frames. This process ensures the accurate positioning of the abnormal area and reduces the false detection rate. When the displacement between the first abnormal image area and the second abnormal image area is greater than the first threshold, the system sends the image area to be identified to the cloud server. The cloud server uses its computing power to classify objects in the image area to be identified, thereby providing accurate recognition results. Through processing by the cloud server, the local computing power limitation can be broken. In summary, the technical scheme improves the computing power adaptation capability, accurate abnormality detection, effective data processing and clear recognition feedback of the surveillance camera, and effectively solves the application problem of the surveillance camera in an environment with high computing power requirements.

[0154] Figure 3 Schematic diagram of a terminal device provided by an embodiment of the present invention. Figure 3 As shown, a terminal device 3 of this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as an abnormality identification program. When the processor 30 executes the computer program 32, the steps in the above-mentioned abnormality identification method embodiments are implemented, such as Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of each unit in the above-mentioned device embodiments are realized, for example, Figure 2 Function of the unit shown.

[0155] Exemplarily, the computer program 32 may be divided into one or more units, which are stored in the memory 31 and executed by the processor 30 to complete the present invention. The one or more units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 32 in the terminal device 3. For example, the computer program 32 may be divided into the following specific functions of each unit:

[0156] An acquisition unit, configured to extract real-time video frames according to a preset sampling frequency, and acquire standard comparison video frames; the standard comparison video frames include historical video frames that are determined to be in a normal state;

[0157] A first comparison unit, used for performing grayscale comparison between the current real-time video frame and the standard comparison video frame to obtain a first abnormal image region;

[0158] An expansion unit, used for expanding the first abnormal image region to obtain an image region to be identified;

[0159] A second comparison unit is used to compare the subsequent real-time video frame with the standard comparison video frame pixel by pixel based on the image area to be identified, so as to obtain a second abnormal image area;

[0160] a judgment unit, configured to send the image area to be identified to a cloud server if the displacement between the first abnormal image area and the second abnormal image area is greater than a first threshold;

[0161] The recognition unit is used to classify the objects in the image area to be recognized through the cloud server to obtain recognition results, and send the recognition results to the user terminal; the recognition results include animals and non-animals.

[0162] The terminal device includes but is not limited to a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of a terminal device 3 and does not constitute a limitation on the terminal device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0163] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0164] The memory 31 may be an internal storage unit of the terminal device 3, such as a hard disk or memory of the terminal device 3. The memory 31 may also be an external storage device of the terminal device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the roaming control device. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0165] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0166] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0167] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0168] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0169] An embodiment of the present invention provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0170] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera device / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0171] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0172] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0173] In the embodiments provided by the present invention, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0174] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed over multiple network units.

[0175] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0176] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0177] As used in the present specification and the appended claims, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to monitoring, depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is monitored" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is monitored" or "in response to monitoring [described condition or event]", depending on the context.

[0178] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0179] References to "one embodiment" or "some embodiments" etc. described in the specification of the present invention mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0180] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for abnormality identification, characterized in that: The method for abnormality identification comprises: Step 101: extracting real-time video frames according to a preset sampling frequency, and obtaining standard comparison video frames; the standard comparison video frames include historical video frames that are determined to be in a normal state; the normal state includes no object entering, and the abnormal state includes animal entering or non-animal entering; Step 102: dividing the real-time video frame into a plurality of first sub-images, and dividing the standard comparison video frame into a plurality of second sub-images; Step 103: Calculate the first grayscale mean and the second grayscale mean corresponding to the first sub-image and the second sub-image in the same area respectively; Step 104: if a first difference between the first grayscale mean and the second grayscale mean is greater than a second threshold, matching other sub-images adjacent to the first sub-image to form a first image region; Step 105: In the first image region, calculating the second grayscale mean and the third grayscale mean corresponding to the first sub-image and the second sub-image in the same region respectively; Step 106: Calculate a second difference between the second grayscale mean and the third grayscale mean in the same area; Step 107: if the number of the second difference values ​​greater than the second threshold is greater than a preset number, then match other sub-images located adjacent to the first image area to form a second image area, and use the second image area as the first image area, and repeat steps 103 to 107 until the number of the second difference values ​​greater than the second threshold is not greater than a preset number, thereby obtaining a first abnormal image area; the preset number includes 0; Step 108: Expand the first abnormal image region to obtain an image region to be identified; Step 109: Based on the image area to be identified, a pixel-by-pixel comparison is performed between the subsequent real-time video frame and the standard comparison video frame to obtain a second abnormal image area; Step 110: If the displacement between the first abnormal image region and the second abnormal image region is greater than a first threshold, the image region to be identified is sent to a cloud server; Step 111: extracting the color, number of pixels and aspect ratio of the current object in the first abnormal image area; Step 112: Calculate the moving speed of the object in the image area to be identified; Step 113: constructing the color, number of pixels, aspect ratio and moving speed of the current object into a real-time feature vector; Step 114: Obtain the historical feature vector corresponding to the historical object; Step 115: Calculate the similarity between the real-time feature vector and the historical feature vector; Step 116: If the similarity is greater than a third threshold, confirming that the recognition result is a historical object; Step 117: if the similarity is not greater than the third threshold, classify the object according to the contour pixel change to obtain a recognition result; Step 118: Send the recognition result to the user terminal; the recognition result includes animals and non-animals.

2. The method for abnormality identification according to claim 1, characterized in that: The step of calculating the moving speed of the object in the image area to be identified comprises: Extracting a target real-time video frame at a preset interval from a plurality of real-time video frames; Obtaining position parameters corresponding to each object in multiple target real-time video frames; Calculate the difference between the position parameters of objects in adjacent sorted real-time video frames; The instantaneous speed is obtained by dividing the difference value by the time interval; the time interval refers to the interval between two adjacent target real-time video frames; The median of the multiple instantaneous speeds is obtained, and the average value of the multiple instantaneous speeds greater than the median is used as the moving speed.

3. The method for abnormality identification according to claim 1, characterized in that: If the similarity is not greater than the third threshold, the step of classifying the object according to the contour pixel change to obtain the recognition result includes: If the similarity is not greater than a third threshold, calculating the contour change amount; If the contour change is greater than a fourth threshold, confirming that the recognition result is a pet or a person; If the contour change is not greater than the fourth threshold, it is confirmed that the identification result is not an animal.

4. The method for abnormality identification according to claim 3, characterized in that: If the similarity is not greater than the third threshold, the step of calculating the contour change comprises: If the similarity is not greater than a third threshold, then calculating the number of pixels of the object contour in each real-time video frame; The pixel quantity difference of the object contour between adjacent real-time video frames is calculated, and a plurality of pixel quantity differences are added to obtain the contour pixel change amount.

5. The method for abnormality identification according to claim 3, characterized in that: If the contour change is greater than a fourth threshold, the step of confirming that the recognition result is a pet or a person includes: Extracting a current object in a first abnormal image region in a plurality of real-time video frames; Superimposing the current object with the white layer to obtain an image to be recognized; Setting the pixel points of the current object in the image to be identified to the first pixel value, and setting the pixel points of the non-current object in the image to be identified to the second pixel value; Extracting the midpoint of the current object in the horizontal coordinate direction; Based on the longitudinal coordinate direction of the current object and the midpoint, forming a segmentation line; Overlap the sub-image areas on both sides of the segmentation line and count the difference points; the difference point refers to the pixel values ​​of two overlapping pixel points being the first pixel value and the second pixel value respectively; Calculate statistical difference points corresponding to multiple real-time video frames respectively; If the plurality of statistical difference points are all greater than a third threshold, then the identification result is confirmed to be a pet; If the plurality of statistical difference points are not all greater than the third threshold, the recognition result is confirmed to be a person.

6. A device for abnormality identification, characterized in that: The abnormality identification device comprises: An acquisition unit is used to extract real-time video frames according to a preset sampling frequency, and acquire standard comparison video frames; the standard comparison video frames include historical video frames that are determined to be in a normal state; the normal state includes no object entering, and the abnormal state includes animal entering or non-animal entering; A first comparison sub-unit, used for dividing the real-time video frame into a plurality of first sub-images, and dividing the standard comparison video frame into a plurality of second sub-images; A second comparison subunit is used to calculate a first grayscale mean and a second grayscale mean respectively corresponding to the first sub-image and the second sub-image in the same area; a third comparing sub-unit, configured to match other sub-images adjacent to the first sub-image to form a first image region if a first difference between the first grayscale mean and the second grayscale mean is greater than a second threshold; A fourth comparison subunit is used to calculate, in the first image region, a second grayscale mean value and a third grayscale mean value corresponding to the first sub-image and the second sub-image in the same region respectively; a fifth comparing subunit, used for calculating a second difference between the second grayscale mean and the third grayscale mean of the same area; a sixth comparing subunit, configured to, if the number of the second difference values ​​greater than the second threshold value is greater than a preset number, match other sub-images located adjacent to the first image area to form a second image area, and use the second image area as the first image area, and cyclically execute the steps corresponding to the second comparing subunit to the sixth comparing subunit until the number of the second difference values ​​greater than the second threshold value is not greater than a preset number, thereby obtaining a first abnormal image area; the preset number includes 0; An expansion unit, used for expanding the first abnormal image region to obtain an image region to be identified; A second comparison unit is used to compare the subsequent real-time video frame with the standard comparison video frame pixel by pixel based on the image area to be identified, so as to obtain a second abnormal image area; a judgment unit, configured to send the image area to be identified to a cloud server if the displacement between the first abnormal image area and the second abnormal image area is greater than a first threshold; A first recognition subunit, used to extract the color, number of pixels and aspect ratio of the current object in the first abnormal image area; A second recognition subunit is used to calculate the moving speed of the object in the image area to be recognized; A third recognition subunit is used to construct the color, number of pixels, aspect ratio and moving speed of the current object into a real-time feature vector; A fourth identification subunit, used to obtain a historical feature vector corresponding to the historical object; a fifth identification subunit, configured to calculate the similarity between the real-time feature vector and the historical feature vector; a sixth recognition subunit, configured to confirm that the recognition result is a historical object if the similarity is greater than a third threshold; a seventh recognition subunit, configured to classify the object according to the change in contour pixels to obtain a recognition result if the similarity is not greater than a third threshold; The eighth identification subunit is used to send the identification result to the user terminal; the identification result includes animals and non-animals.

7. A terminal device, characterized in that: The terminal device comprises: a memory, a processor, and an abnormality identification program stored in the memory and executable on the processor, wherein the abnormality identification program is configured to implement the steps in the abnormality identification method as claimed in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps in the method for abnormality identification according to any one of claims 1 to 5 are implemented.

Images