Method, apparatus, and electronic device for identifying movable object, and readable medium

Abstract

The present disclosure provides a movable object identification method and device, an electronic device and a readable medium, wherein the movable object identification method comprises: determining the similarity between a static image set and a dynamic image set of a movable object, and determining the identification result of the movable object based on the similarity. Through the embodiments of the present disclosure, a large number of sample images do not need to be collected for pre-training, and the efficiency, accuracy and reliability of identifying the movable object are improved.

Description

Technical Field

[0001] This disclosure relates to the field of image recognition technology, and more specifically, to a method, apparatus, electronic device, and readable medium for recognizing movable objects. Background Technology

[0002] Currently, the need for movable object identification exists in various scenarios of daily life, especially in the catering and logistics warehousing industries. There is a need to detect movable objects with pest and disease characteristics, such as rats, cockroaches, and termites, in order to reduce the adverse effects of pests and diseases in a timely manner.

[0003] In related technologies, the identification of movable objects is usually achieved based on image comparison and model recognition.

[0004] However, the aforementioned image recognition technologies require the collection of a large amount of training data and the setting and adjustment of model parameters, which not only leads to low efficiency and poor generalization of image recognition, but also causes a sharp increase in the amount of data maintenance.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a method, apparatus, electronic device, and readable medium for identifying movable objects, which at least partially overcomes the problem of large sampling data volume in image recognition schemes due to limitations and defects in related technologies.

[0007] According to a first aspect of the present disclosure, a method for identifying a movable object is provided, comprising: determining the similarity between a set of static images and a set of dynamic images of the movable object, and determining an identification result of the movable object based on the similarity.

[0008] In one exemplary embodiment of this disclosure, before determining the similarity between the static image set and the dynamic image set of the movable object, the method further includes: acquiring images of a specified area within a specified time period; extracting B-channel sub-images and G-channel sub-images from the acquired images; determining the absolute value of the difference between the B-channel sub-images and the G-channel sub-images; and determining a suspected infrared image in the image based on the absolute value of the difference.

[0009] In one exemplary embodiment of this disclosure, before determining the similarity between the static image set and the dynamic image set of the movable object, the method further includes: dynamically detecting the movable object based on the suspected infrared image.

[0010] In one exemplary embodiment of this disclosure, dynamic detection of the movable object based on the suspected infrared image includes: determining the (n-1)th frame image and the nth frame image in the suspected infrared image; converting the (n-1)th frame image into the (n-1)th frame image and converting the nth frame image into the nth grayscale image; calculating the (n-1)th frame image and the nth frame grayscale image using the SSIM algorithm to obtain a structured image; binarizing the structured image to obtain a corresponding binarized image; filtering the binarized image according to a preset binarization range to determine the motion region and the movable object; and generating the dynamic image set based on the movable object.

[0011] In one exemplary embodiment of this disclosure, before determining the similarity between the static image set and the dynamic image set of the movable object, the method further includes: augmenting the suspected infrared image based on the motion region and the movable object; and training the YOLOv5 framework based on the augmented suspected infrared image.

[0012] In an exemplary embodiment of this disclosure, augmenting the suspected infrared image based on the motion region and the movable object includes: determining a set of object contour points based on the movable object; determining the coordinate information of two diagonal points in the object contour point set; segmenting the movable object into sub-images based on the coordinate information of the two diagonal points; determining the R channel, G channel, and B channel of the sub-images, and creating an alpha channel corresponding to the sub-images; merging the R channel, the G channel, the B channel, and the alpha channel; saving the merged image as an image in a specified format; merging the specified format image into a motion region that does not contain the movable object using an alphablind algorithm to augment the suspected infrared image; and generating the static image set based on the augmented suspected infrared image.

[0013] In one exemplary embodiment of this disclosure, training the YOLOv5 framework based on the augmented suspected infrared image includes: cropping the suspected infrared images in the static image set according to a preset overlap ratio; labeling the suspected infrared image according to the position of the movable object in the suspected infrared image; and training the YOLOv5 framework based on the labeled suspected infrared image.

[0014] In an exemplary embodiment of this disclosure, before determining the similarity between the static image set and the dynamic image set of the movable object, the method further includes: in response to an instruction to identify the image to be identified in the specified region, sending the image to be identified to a trained YOLOv5 framework for movable object identification; cropping the identified image to be identified according to a preset cropping rule to obtain a sub-image to be identified; detecting the sub-image to be identified based on the model corresponding to the movable object; regressing the detected sub-image to be identified back to the image to be identified according to the preset cropping rule; filtering the regressed image to be identified using the NMS algorithm for overlapping boxes; determining the coordinates of the movable object in the image to be identified based on the filtering result, and saving them to the static image set.

[0015] In an exemplary embodiment of this disclosure, determining the similarity between a set of static images and a set of dynamic images of the movable object, and determining the recognition result of the movable object based on the similarity, includes: obtaining a first subset of the dynamic image set and obtaining a second subset of the static image set; determining a first movable object subset in the first subset using grayscale histogram statistics, and determining a second movable object subset in the second subset using grayscale histogram statistics; calculating the cosine similarity between the first movable object subset and the second movable object subset; comparing whether the cosine similarity is greater than a preset similarity; if the cosine similarity is greater than the preset similarity, then determining that the image to be recognized contains the movable object; if the cosine similarity is less than or equal to the preset similarity, then determining that the image to be recognized does not contain the movable object.

[0016] According to a second aspect of the present disclosure, a device for identifying a movable object is provided, comprising: an identification module configured to determine the similarity between a set of static images and a set of dynamic images of the movable object, and to determine an identification result of the movable object based on the similarity.

[0017] According to a third aspect of this disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to perform the method as described in any of the preceding methods based on instructions stored in the memory.

[0018] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a program stored thereon that, when executed by a processor, implements the method for identifying a movable object as described in any of the preceding claims.

[0019] In this embodiment of the disclosure, by determining the similarity between a set of static images and a set of dynamic images of the movable object, and determining the recognition result of the movable object based on the similarity, it is not necessary to collect a large number of sample images for pre-training, thereby improving the efficiency, accuracy and reliability of recognizing movable objects.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0022] Figure 1 A schematic diagram of an exemplary system architecture for a movable object recognition scheme to which embodiments of the present invention can be applied is shown;

[0023] Figure 2 This is a flowchart of a method for identifying a movable object in an exemplary embodiment of this disclosure;

[0024] Figure 3 This is a flowchart of another method for identifying movable objects in an exemplary embodiment of this disclosure;

[0025] Figure 4 This is a flowchart of another method for identifying movable objects in an exemplary embodiment of this disclosure;

[0026] Figure 5 This is a flowchart of another method for identifying movable objects in an exemplary embodiment of this disclosure;

[0027] Figure 6 This is a flowchart of another method for identifying movable objects in an exemplary embodiment of this disclosure;

[0028] Figure 7 This is a flowchart of another method for identifying movable objects in an exemplary embodiment of this disclosure;

[0029] Figure 8 This is a flowchart of another method for identifying movable objects in an exemplary embodiment of this disclosure;

[0030] Figure 9 This is a flowchart of another method for identifying movable objects in an exemplary embodiment of this disclosure;

[0031] Figure 10This is a flowchart of another method for identifying movable objects in an exemplary embodiment of this disclosure;

[0032] Figure 11 This is an overall flowchart of another movable object recognition scheme in an exemplary embodiment of this disclosure;

[0033] Figure 12 This is a block diagram of a movable object identification device according to an exemplary embodiment of the present disclosure;

[0034] Figure 13 This is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0035] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0036] Furthermore, the accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0037] Figure 1 A schematic diagram of an exemplary system architecture for identifying movable objects to which embodiments of the present invention can be applied is shown.

[0038] like Figure 1 As shown, system architecture 100 may include one or more of terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0039] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, networks, and servers. For example, server 105 could be a server cluster composed of multiple servers.

[0040] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers, etc.

[0041] In some embodiments, the movable object identification method provided in this invention is generally executed by server 105, and correspondingly, the movable object identification device is generally disposed in terminal device 103 (or terminal device 101 or 102). In other embodiments, some terminals may have functions similar to those of the server device to execute this method.

[0042] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0043] Figure 2 This is a flowchart of a method for identifying movable objects in an exemplary embodiment of this disclosure.

[0044] refer to Figure 2 Methods for identifying movable objects may include:

[0045] Step S202: Determine the similarity between the static image set and the dynamic image set of the movable object, and determine the recognition result of the movable object based on the similarity.

[0046] In this embodiment of the disclosure, by determining the similarity between a set of static images and a set of dynamic images of the movable object, and determining the recognition result of the movable object based on the similarity, it is not necessary to collect a large number of sample images for pre-training, thereby improving the efficiency, accuracy and reliability of recognizing movable objects.

[0047] The following section provides a detailed explanation of each step in the method for identifying movable objects.

[0048] In one exemplary embodiment of this disclosure, such as Figure 3 As shown, before determining the similarity between the static image set and the dynamic image set of the movable object, the process further includes:

[0049] Step S302: Acquire images of a specified area within a specified time period.

[0050] Step S304: Extract the B channel sub-image and G channel sub-image from the acquired image.

[0051] Step S306: Determine the absolute value of the difference between the B channel sub-image and the G channel sub-image.

[0052] Step S308: Determine the suspected infrared image in the image based on the absolute value of the difference.

[0053] In the above embodiments, the B channel is the blue light channel in RGB and the G channel is the green light channel in RGB. Images with an absolute difference value less than a preset absolute value are identified as suspected infrared images, which can effectively reduce computational costs, improve detection efficiency, and effectively distinguish the features of movable objects in suspected infrared images.

[0054] In one exemplary embodiment of this disclosure, such as Figure 4 As shown, before determining the similarity between the static image set and the dynamic image set of the movable object, the process further includes:

[0055] Step S402: Dynamically detect the movable object based on the suspected infrared image.

[0056] In one exemplary embodiment of this disclosure, such as Figure 5 As shown, dynamic detection of the movable object based on the suspected infrared image includes:

[0057] Step S502: Determine the (n-1)th frame and the nth frame in the suspected infrared image.

[0058] Step S504: Convert the (n-1)th frame image into the (n-1)th frame image, and convert the nth frame image into the nth grayscale image.

[0059] Step S506: Calculate the structured image by using the SSIM algorithm on the (n-1)th frame image and the nth frame grayscale image.

[0060] In the above embodiments, SSIM (structural similarity) is a full-reference image quality evaluation index that measures the similarity between two images from three aspects: brightness, contrast, and structure. A higher value is better, with a maximum of 1. As an implementation of structural similarity theory, the structural similarity index defines structural information from the perspective of image composition as an attribute that reflects the structure of objects in a scene, independent of brightness and contrast. It models distortion as a combination of three different factors: brightness, contrast, and structure. The mean is used as an estimate of brightness, the standard deviation as an estimate of contrast, and the covariance as a measure of structural similarity. SSIM can take into account the perceptual ambiguity of structural information in images. This model also introduces some perceptual phenomena related to perceptual changes. Structural information refers to the internal dependencies between pixels, especially spatially close pixels.

[0061] Step S508: The structured image is binarized to obtain the corresponding binarized image.

[0062] Step S510: Filter the binarized image according to a preset binarization range to determine the motion area and movable objects.

[0063] In the above embodiments, a binarized image is obtained by binarization, and an object region, i.e. a motion region, suitable for the size of a movable object is extracted by a preset binarization range. Movable objects are white, and non-movable objects are black.

[0064] Step S512: Generate the dynamic image set based on the movable object.

[0065] In one exemplary embodiment of this disclosure, such as Figure 6 As shown, before determining the similarity between the static image set and the dynamic image set of the movable object, the process further includes:

[0066] Step S602: Expand the suspected infrared image based on the motion area and the movable object.

[0067] Step S604: Train the YOLOv5 framework based on the augmented suspected infrared image.

[0068] In one exemplary embodiment of this disclosure, such as Figure 7 As shown, augmenting the suspected infrared image based on the motion region and the movable object includes:

[0069] Step S702: Determine the object contour point set based on the movable object.

[0070] Step S704: Determine the coordinate information of two diagonal points in the object contour point set.

[0071] In the above embodiments, the coordinates of the upper left corner and the lower right corner of the movable object in the object contour point set are denoted as (xmin, ymin) and (xmax, ymax), respectively.

[0072] Step S706: Based on the coordinate information of the two diagonal points, the movable object is divided into sub-graphs.

[0073] Step S708: Determine the R channel, G channel and B channel of the sub-image, and create the alpha channel corresponding to the sub-image.

[0074] Step S710: Merge the R channel, the G channel, the B channel, and the alpha channel.

[0075] Step S712: Save the merged image as an image in the specified format.

[0076] In the above embodiments, the specified format can be png or jpg.

[0077] Step S714: The specified format image is merged into a motion region that does not contain the movable object using the alphablind algorithm, thereby expanding the suspected infrared image.

[0078] In the above embodiments, the alphablind algorithm includes the following processing steps:

[0079] 1) Separate the RGB color components of the source pixel and the target pixel;

[0080] 2) Multiply the three color components of the source pixel by the value of Alpha, and multiply the three color components of the target pixel by the inverse value of Alpha.

[0081] 3) Add the results together according to their corresponding color components;

[0082] 4) Divide the result of each component obtained in the end by the maximum value of Alpha (this step is usually done by bit shifting, which is why the maximum value of Alpha is always a power of two).

[0083] 5) Finally, combine the three color components back into a single pixel for output.

[0084] As can be seen from the above alphablind algorithm process, the larger the alpha value, the weaker the transparency effect. That is, when the alpha value reaches its maximum, it is opaque. Conversely, if the alpha value is 0, it is completely transparent.

[0085] Based on this, the specified format image is merged into a motion region that does not contain the movable object. The Alpha parameter is the saved Alpha value in PNG format. To ensure the realism of the texture, a click event can be set for the motion region. The movable object is added to the motion region according to the click position in the motion region to synthesize a new static image containing the movable object. This realizes the sample expansion of the suspected infrared image. The sample expansion helps to improve the accuracy of the recognition model and further improves the reliability and accuracy of movable object recognition.

[0086] Step S716: Generate the static image set based on the expanded suspected infrared image.

[0087] In one exemplary embodiment of this disclosure, such as Figure 8 As shown, training the YOLOv5 framework based on the augmented suspected infrared image includes:

[0088] Step S802: The suspected infrared images in the static image set are cropped according to a preset overlap ratio.

[0089] In the above embodiments, the suspected infrared image can be slidably cut into a window with an overlap area of ​​30% and a size of 960×544, and a labeling file can be generated based on the position.

[0090] Step S804: Mark the suspected infrared image according to the position of the movable object in the suspected infrared image.

[0091] Step S806: Train the YOLOv5 framework based on the labeled suspected infrared images.

[0092] In one exemplary embodiment of this disclosure, such as Figure 9 As shown, before determining the similarity between the static image set and the dynamic image set of the movable object, the process further includes:

[0093] Step S902: In response to the instruction to recognize the image to be recognized in the specified region, the image to be recognized is sent to the trained YOLOv5 framework for recognition of movable objects.

[0094] Step S904: The identified image to be identified is cropped according to a preset cropping rule to obtain a sub-image of the image to be identified.

[0095] Step S906: Detect the sub-image of the image to be identified based on the model corresponding to the movable object.

[0096] Step S908: The detected sub-image of the image to be identified is reverted to the image to be identified according to the preset cropping rules.

[0097] Step S910: The overlapping box filtering of the regressed image to be identified is performed using the NMS algorithm.

[0098] In the above embodiments, the NMS (Non-max suppression) algorithm finds local maxima, not maximum values, and suppresses the remaining values ​​in their neighborhood. In object detection, multiple candidate boxes may be predicted for an object; in this case, NMS can be used to filter out some redundant boxes. Generally, the output of each predicted box includes the box's location information and confidence score.

[0099] The NMS algorithm process includes:

[0100] (1) Filter out boxes with low confidence by applying a confidence threshold to all boxes.

[0101] (2) Select the box with the highest confidence and save it into the output list.

[0102] (3) Calculate the IOU value of the box with the other remaining boxes in turn. Then remove the boxes with a large IOU with the box with the highest confidence (i.e., the boxes that are close to this box) by using an IOU threshold.

[0103] (4) Continue to perform (2) and (3) operations on the remaining boxes until the traversal is complete.

[0104] Step S902: Determine the coordinates of the movable objects in the image to be identified based on the filtering results, and save them to the static image set.

[0105] In one exemplary embodiment of this disclosure, such as Figure 10 As shown, determining the similarity between the static image set and the dynamic image set of the movable object, and determining the recognition result of the movable object based on the similarity, includes:

[0106] Step S1002: Obtain a first subset of the dynamic image set and a second subset of the static image set.

[0107] Step S1004: Determine the first movable object subset in the first subset by grayscale histogram statistics, and determine the second movable object subset in the second subset by grayscale histogram statistics.

[0108] In the above embodiments, the first subset and the second subset are filtered by grayscale histogram statistics. If the ratio of pixel values ​​greater than a threshold to those less than a threshold is not less than a certain threshold, then the object is considered a movable object.

[0109] Step S1006: Calculate the cosine similarity between the first movable object subset and the second movable object subset.

[0110] Step S1008: Compare whether the cosine similarity is greater than a preset similarity.

[0111] Step S1010: If it is determined that the cosine similarity is greater than the preset similarity, then it is determined that the image to be identified contains the movable object.

[0112] Step S1012: If it is determined that the cosine similarity is less than or equal to the preset similarity, then it is determined that the image to be identified does not contain the movable object.

[0113] In the above embodiments, by combining dynamic detection results, static detection results, and the grayscale histogram ratio of the mouse for confirmation, and taking into account the characteristics of movable objects such as volume and eye reflection in suspected infrared images, the movable object is confirmed by the grayscale histogram ratio, thereby improving the reliability and accuracy of movable object identification.

[0114] Based on the above Figures 2 to 10 The illustrated embodiment, using a mouse as a movable object, specifically illustrates the movable object identification scheme of this disclosure. Figure 11 As shown, the movable object recognition scheme includes the following steps:

[0115] S1: Input process, which involves connecting the terminal's IP address to the detection system based on the terminal camera's location, uniquely marking the terminal's location for easy alarm dissemination to that area after an alarm is triggered. Once the camera is connected to the system, the video stream of that scene can be retrieved in real time. Configure the mouse detection algorithm's activation time and alarm interval.

[0116] S2: Infrared mouse image acquisition module, which filters mouse colors. Because mice are nocturnal and afraid of people, mouse detection in a kitchen setting can use these characteristics to initially screen suspected infrared images as images where mice might be present.

[0117] First, the mouse color was filtered using a suspected infrared image, and the acquired RGB channels were separated into B channel sub-images and G channel sub-images at 50-pixel intervals.

[0118] Secondly, since the pixel values ​​between channels after the suspected infrared image is formed are R≈B≈G, the absolute values ​​of the B channel and the G channel are calculated.

[0119] Ideally, if the absolute value is 0, the image is an infrared image. However, due to potential noise during the imaging process, images with an absolute value less than 2 are considered potential infrared images. Infrared images with a high mouse presence rate are then selected, and images likely to contain mice are sent to the dynamic mouse detection module.

[0120] S3: The moving mouse detection module performs dynamic mouse detection.

[0121] Kitchen scenes are usually empty late at night, so the scene is static. The appearance of a mouse should be due to movement, so dynamic mouse detection is needed first.

[0122] First, a queue Q of size 1 is designed. The previous frame image I1 is saved into the queue Q. The images I1 and the current frame I2 are converted into grayscale images, and the structured image diff is obtained through the SSIM algorithm. I1 and I2 are of equal size and neither is resized.

[0123] Secondly, the diff image is binarized to obtain a black and white binary image. The lower limit of the binarization threshold is 90 and the lower limit is 255 to ensure that moving objects are white and non-moving objects are black.

[0124] The SSIM algorithm formula is as follows:

[0125] SSIM(x, y) = (2u x u y +c1)(σ xy +c2) / ((u) x 2 +u y 2 +c1)(σ x 2 +σ y 2 +c2)),

[0126] Where ux and uy represent the mean of x and y respectively, σx and σy represent the standard deviation of x and y respectively, and σ xy c1 represents the covariance of x and y, and c2 represents the region of the moving mouse extracted.

[0127] Finally, suitable moving object regions for mice are filtered in the binarized image using two thresholds of 10×10 and 50×50, and the coordinates of the moving regions are stored in set C1, while the moving object sub-images are saved to set L1.

[0128] S4: The static mouse detection module performs static mouse training.

[0129] Due to the influence of various objects such as changing lighting, flying insects, other crawling insects, and human bodies, dynamic object detection cannot accurately detect whether a moving object is a mouse. Therefore, static mouse detection needs to be added. Static mouse detection is trained based on the YOLOv5 deep learning framework. However, because mice are not easy to detect, the dataset is small, and the accuracy and recall of the trained model are low. Therefore, this disclosure proposes a mouse data augmentation method.

[0130] First, the mouse outline is marked according to the segmentation method to obtain the mouse outline point set P. Then, the coordinates of the top left corner and bottom right corner of the mouse position are obtained according to the coordinate values ​​in the point set P and recorded as (xmin, ymin) and (xmax, ymax). Then, the mouse sub-image is segmented, and the mouse sub-image is divided into three channels: R, G, and B. An Alpha channel with a size of the image size and a channel of 1 is created. The R, G, B, and Alpha channels are merged and saved as a PNG format image.

[0131] The Alphablind algorithm is used to blend mice into suspected infrared images that do not contain mice. The Alpha parameter is the saved Alpha value in PNG format. To ensure the realism of the texture, a mouse click event is set to expand the mouse based on the mouse click position.

[0132] Since the mice are very small, they cannot be resized. Therefore, a training image cropping method is used, which directly cuts the image according to the 30% overlap ratio and generates a label file based on the position.

[0133] Finally, the YOLOv5 algorithm was used for training. During detection, the test images were first cropped, and the trained mouse model was used to detect all cropped images. Then, all detection results were regressed to the original images according to the cropping rules. Finally, the NMS algorithm was used to filter overlapping boxes. The coordinates of the detected mice, combined with C2, and the subset of detected mice were obtained and saved to set L2.

[0134] S5: The mouse identification and confirmation module performs joint confirmation of mice. Both dynamic and static mouse detection can detect mice, but neither can guarantee accuracy, easily generating false alarms and frequent alerts, which can affect user experience. Therefore, a joint mouse confirmation scheme is designed.

[0135] First, obtain the mouse subset from image sets L1 and L2. Using grayscale histogram statistics, the mouse is defined as a pixel whose pixel value is greater than 150 and less than 150 in a ratio of not less than 95%.

[0136] Secondly, the mice in the subsets are converted into one-dimensional vectors, and the cosine similarity between the two subsets is calculated using the N:N ratio, as shown in the following formula:

[0137] cosθ=(x1x2+y1y2) / ((x1 2 +y1 2 (x2) 2 +y2 2 )),

[0138] Here, x and y are the coordinates of two vectors, respectively. If the final cosine value is greater than 0.8, it is identified as a mouse, and stationary objects are filtered out as false positives. Combining the two confirmation methods mentioned above, the final score is calculated with a weighting of 6:4. Scores higher than 0.5 indicate that the final result is a mouse.

[0139] S6: Output mouse alarm. Mouse detection requires alerting the user to detect and detect mice, making it easier for the user to catch them. Therefore, after the algorithm generates a mouse detection result, it returns the mouse's location and alarm signal, recording the mouse's location, time, and camera information in the system. The system can also send a notification to the user via SMS.

[0140] Corresponding to the above method embodiments, this disclosure also provides a movable object identification device, which can be used to execute the above method embodiments.

[0141] Figure 12 This is a block diagram of a movable object identification device according to an exemplary embodiment of the present disclosure.

[0142] refer to Figure 12 The movable object recognition device 1200 may include:

[0143] The recognition module 1202 is configured to determine the similarity between the static image set and the dynamic image set of the movable object, and to determine the recognition result of the movable object based on the similarity.

[0144] In one exemplary embodiment of this disclosure, before determining the similarity between the static image set and the dynamic image set of the movable object, the method further includes: acquiring images of a specified area within a specified time period; extracting B-channel sub-images and G-channel sub-images from the acquired images; determining the absolute value of the difference between the B-channel sub-images and the G-channel sub-images; and determining a suspected infrared image in the image based on the absolute value of the difference.

[0145] In an exemplary embodiment of this disclosure, before determining the similarity between the static image set and the dynamic image set of the movable object, the identification module 1202 is further configured to: dynamically detect the movable object based on the suspected infrared image.

[0146] In an exemplary embodiment of this disclosure, the identification module 1202 is further configured to: determine the (n-1)th frame image and the nth frame image in the suspected infrared image; convert the (n-1)th frame image into the (n-1)th frame image and convert the nth frame image into the nth grayscale image; perform calculations on the (n-1)th frame image and the nth frame grayscale image using the SSIM algorithm to obtain a structured image; perform binarization processing on the structured image to obtain a corresponding binarized image; filter the binarized image according to a preset binarization range to determine the motion region and movable objects; and generate the dynamic image set based on the movable objects.

[0147] In an exemplary embodiment of this disclosure, before determining the similarity between the static image set and the dynamic image set of the movable object, the recognition module 1202 is further configured to: augment the suspected infrared image based on the motion region and the movable object; and train the YOLOv5 framework based on the augmented suspected infrared image.

[0148] In an exemplary embodiment of this disclosure, the identification module 1202 is further configured to: determine a set of object contour points based on the movable object; determine the coordinate information of two diagonal points in the set of object contour points; segment the movable object into sub-images based on the coordinate information of the two diagonal points; determine the R channel, G channel, and B channel of the sub-images, and create an alpha channel corresponding to the sub-images; merge the R channel, the G channel, the B channel, and the alpha channel; save the merged image as an image of a specified format; merge the image of the specified format into a motion region that does not contain the movable object using the alphablind algorithm, thereby expanding the suspected infrared image; and generate the static image set based on the expanded suspected infrared image.

[0149] In an exemplary embodiment of this disclosure, the identification module 1202 is further configured to: crop the suspected infrared images in the static image set according to a preset overlap ratio; label the suspected infrared images according to the position of the movable object in the suspected infrared images; and train the YOLOv5 framework based on the labeled suspected infrared images.

[0150] In an exemplary embodiment of this disclosure, before determining the similarity between the static image set and the dynamic image set of the movable object, the recognition module 1202 is further configured to: in response to an instruction to recognize the image to be recognized in the specified region, send the image to be recognized to the trained YOLOv5 framework for movable object recognition; crop the recognized image to be recognized according to a preset cropping rule to obtain a sub-image of the image to be recognized; detect the sub-image of the image to be recognized based on the model corresponding to the movable object; regress the detected sub-image of the image to be recognized to the image to be recognized according to the preset cropping rule; perform overlapping box filtering on the regressed image to be recognized using the NMS algorithm; determine the coordinates of the movable object in the image to be recognized based on the filtering result, and save them to the static image set.

[0151] In an exemplary embodiment of this disclosure, the recognition module 1202 is further configured to: acquire a first subset of the dynamic image set and acquire a second subset of the static image set; determine a first movable object subset in the first subset and a second movable object subset in the second subset through grayscale histogram statistics; calculate the cosine similarity between the first movable object subset and the second movable object subset; compare whether the cosine similarity is greater than a preset similarity; if the cosine similarity is greater than the preset similarity, determine that the image to be recognized contains the movable object; if the cosine similarity is less than or equal to the preset similarity, determine that the image to be recognized does not contain the movable object.

[0152] Since the functions of the movable object identification device 1200 have been described in detail in their respective method embodiments, they will not be repeated here.

[0153] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0154] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0155] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as “circuit,” “module,” or “system.”

[0156] The following reference Figure 13 To describe an electronic device 1300 according to this embodiment of the present invention. Figure 13 The electronic device 1300 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0157] like Figure 13 As shown, the electronic device 1300 is presented in the form of a general-purpose computing device. The components of the electronic device 1300 may include, but are not limited to: at least one processing unit 1310, at least one storage unit 1320, and a bus 1330 connecting different system components (including storage unit 1320 and processing unit 1310).

[0158] The storage unit stores program code that can be executed by the processing unit 1310, causing the processing unit 1310 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 1310 can perform the method shown in the embodiments of this disclosure.

[0159] Storage unit 1320 may include readable media in the form of volatile storage units, such as random access memory (RAM) 13201 and / or cache memory 13202, and may further include read-only memory (ROM) 13203.

[0160] Storage unit 1320 may also include a program / utility 13204 having a set (at least one) of program modules 13205, such program modules 13205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0161] Bus 1330 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0162] Electronic device 1300 can also communicate with one or more external devices 1340 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1300, and / or any device that enables electronic device 1300 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1350. Furthermore, electronic device 1300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1360. As shown, network adapter 1360 communicates with other modules of electronic device 1300 via bus 1330. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0163] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0164] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section of this specification.

[0165] The program product for implementing the above-described method according to embodiments of the present invention may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0166] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0167] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0168] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0169] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0170] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0171] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and concept of this disclosure are indicated by the claims.

Claims

1. A method for identifying movable objects, characterized in that, include: Acquire images of a specified area within a specified time period; Extract B-channel and G-channel sub-images from the acquired images; Determine the absolute value of the difference between the B-channel subgraph and the G-channel subgraph; The suspected infrared image in the image is determined based on the absolute value of the difference; Determine the similarity between the static image set and the dynamic image set of the movable object, and determine the recognition result of the movable object based on the similarity; The dynamic image set is generated through the following steps: Identify the (n-1)th frame and the nth frame in the suspected infrared image; The (n-1)th frame image is converted into the (n-1)th frame image, and the nth frame image is converted into the nth frame grayscale image; The structured image is obtained by calculating the (n-1)th frame image and the nth frame grayscale image using the SSIM algorithm; The structured image is binarized to obtain the corresponding binarized image; The binarized image is filtered according to a preset binarization range to determine the motion region and movable objects; The dynamic image set is generated based on the movable object; The set of static images is generated through the following steps: Determine the object contour point set based on the movable object; Determine the coordinate information of two diagonal points in the object contour point set; Based on the coordinate information of the two diagonal points, the movable object is divided into sub-graphs; Determine the R, G, and B channels of the subgraph, and create the corresponding alpha channel for the subgraph; Combine the R channel, the G channel, the B channel, and the alpha channel; Save the merged image as an image in the specified format; The image of the specified format is merged into a motion region that does not contain the movable object by an image synthesis algorithm, so as to expand the suspected infrared image; The static image set is generated based on the expanded suspected infrared image.

2. The method for identifying movable objects as described in claim 1, characterized in that, Before determining the similarity between the static image set and the dynamic image set of the movable object, the method further includes: The movable object is dynamically detected based on the suspected infrared image.

3. The method for identifying movable objects as described in claim 1, characterized in that, Before determining the similarity between the static image set and the dynamic image set of the movable object, the method further includes: The suspected infrared image is augmented based on the motion area and the movable object; The YOLOv5 framework is trained based on the augmented suspected infrared image.

4. The method for identifying movable objects as described in claim 1, characterized in that, Training the YOLOv5 framework based on the augmented suspected infrared image includes: The suspected infrared images in the static image set are cropped according to a preset overlap ratio; The suspected infrared image is labeled according to the position of the movable object in the suspected infrared image; The YOLOv5 framework is trained based on the labeled suspected infrared images.

5. The method for identifying movable objects as described in claim 4, characterized in that, Before determining the similarity between the static image set and the dynamic image set of the movable object, the method further includes: In response to an instruction to identify the image to be identified in the specified region, the image to be identified is sent to the trained YOLOv5 framework for the identification of movable objects. The image to be identified is cropped according to a preset cropping rule to obtain a sub-image of the image to be identified; The sub-image of the image to be identified is detected based on the model corresponding to the movable object; The detected sub-image of the image to be identified is regressed to the image to be identified according to the preset cropping rules; The NMS algorithm is used to filter overlapping boxes in the regressed image to be identified. Based on the filtering results, the coordinates of the movable objects in the image to be identified are determined and saved to the static image set.

6. The method for identifying a movable object as described in any one of claims 1-5, characterized in that, Determining the similarity between the static image set and the dynamic image set of the movable object, and determining the recognition result of the movable object based on the similarity, includes: Obtain a first subset of the dynamic image set and a second subset of the static image set; The first movable object subset in the first subset is determined by gray-level histogram statistics, and the second movable object subset in the second subset is determined by gray-level histogram statistics. Calculate the cosine similarity between the first movable object subset and the second movable object subset; Compare whether the cosine similarity is greater than a preset similarity; If the cosine similarity is determined to be greater than the preset similarity, then the image to be identified is determined to contain the movable object. If the cosine similarity is determined to be less than or equal to the preset similarity, then the image to be identified does not contain the movable object.

7. A device for identifying movable objects, characterized in that, include: The recognition module is configured to acquire images of a specified area within a specified time period; Extract B-channel and G-channel sub-images from the acquired images; Determine the absolute value of the difference between the B-channel subgraph and the G-channel subgraph; The suspected infrared image in the image is determined based on the absolute value of the difference; Determine the similarity between the static image set and the dynamic image set of the movable object, and determine the recognition result of the movable object based on the similarity; The dynamic image set is generated through the following steps: Identify the (n-1)th frame and the nth frame in the suspected infrared image; The (n-1)th frame image is converted into the (n-1)th frame image, and the nth frame image is converted into the nth frame grayscale image; The structured image is obtained by calculating the (n-1)th frame image and the nth frame grayscale image using the SSIM algorithm; The structured image is binarized to obtain the corresponding binarized image; The binarized image is filtered according to a preset binarization range to determine the motion region and movable objects; The dynamic image set is generated based on the movable object; The set of static images is generated through the following steps: Determine the object contour point set based on the movable object; Determine the coordinate information of two diagonal points in the object contour point set; Based on the coordinate information of the two diagonal points, the movable object is divided into sub-graphs; Determine the R, G, and B channels of the subgraph, and create the corresponding alpha channel for the subgraph; Combine the R channel, the G channel, the B channel, and the alpha channel; Save the merged image as an image in the specified format; The image of the specified format is merged into a motion region that does not contain the movable object by an image synthesis algorithm, so as to expand the suspected infrared image; The static image set is generated based on the expanded suspected infrared image.

8. An electronic device, characterized in that, include: Memory; as well as A processor coupled to the memory, the processor being configured to execute the movable object identification method as described in any one of claims 1-6 based on instructions stored in the memory.

9. A computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the method for identifying a movable object as described in any one of claims 1-6.

Images