A method and device for anti-theft monitoring based on vision

Abstract

The present invention provides a vision-based anti-theft monitoring method and device, wherein the method includes: acquiring a first image at a target time node and a second image before the target time node; generating a corresponding differential image based on the difference between the first image and the second image, and determining whether there is an abnormal area in the differential image; when there is an abnormal area, determining the target object and the behavior of the target object in the first image and the second image, and determining the existence of a theft suspicion when the behavior of the target object is abnormal. The vision-based anti-theft monitoring method and device provided by the embodiment of the present invention can automatically monitor whether there is a theft suspicion in the user's blind spot, thereby improving the security in the blind spot; using the differential image to determine whether it is necessary to identify the object and the object behavior in the image can reduce the processing volume of image recognition, so that anti-theft detection can be carried out in a timely manner, and the timeliness of anti-theft monitoring can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a vision-based anti-theft monitoring method and device. Background Art

[0002] The driver has a blind spot in the vehicle. Although the rearview mirror can expand the driver's field of vision, there is still a large blind spot in large logistics and transportation trucks. First, during the driving process, the driver's attention is mainly focused on the front of the vehicle. If someone climbs on the vehicle, forcibly enters the cargo hold, or other unknown behaviors, the driver rarely notices it. Second, due to the driving characteristics of large trucks or commercial vehicles, many drivers have the characteristic of resting in the car, or when the driver enters the hotel to rest, the safety of the cargo in the cargo hold and the safety of the gasoline barrels become a problem. Summary of the invention

[0003] To solve the above problems, an object of the embodiments of the present invention is to provide a vision-based anti-theft monitoring method and device.

[0004] In a first aspect, an embodiment of the present invention provides a visual-based anti-theft monitoring method, comprising:

[0005] Acquire a first image at a target time node and a second image before the target time node;

[0006] generating a corresponding differential image according to the difference between the first image and the second image, and determining whether there is an abnormal area in the differential image;

[0007] When there is an abnormal area, a target object in the first image and the second image and a behavior of the target object are determined, and when the behavior of the target object is abnormal, it is determined that there is a suspicion of theft.

[0008] In a possible implementation manner, generating a corresponding differential image according to a difference between the first image and the second image, and determining whether there is an abnormal area in the differential image includes:

[0009] performing differential processing on the first image and the second image, and generating a differential image representing a difference between the first image and the second image according to a differential processing result;

[0010] Determine the pixel value of each pixel in the differential image, and take the pixel whose pixel value is within a preset abnormal range as an abnormal pixel;

[0011] The total number of the abnormal pixel points that are adjacent in sequence is determined, and when the total number is greater than a preset threshold, an area where the abnormal pixel points that are adjacent in sequence are located is taken as an abnormal area.

[0012] In a possible implementation manner, generating a differential image representing a difference between the first image and the second image according to a differential processing result includes:

[0013] Generating a differential initial image according to the differential processing result, and performing binarization processing on the differential image to generate a differential binary image;

[0014] The differential binary image is dilated, and the differential binary image after the dilation process is used as the generated differential image.

[0015] In a possible implementation manner, the first image includes a first left-eye image and a first right-eye image, and the second image includes a second left-eye image and a second right-eye image.

[0016] In a possible implementation manner, determining the target object in the first image and the second image and the behavior of the target object includes:

[0017] Determine a first region image corresponding to the abnormal region in the first image, and a second region image corresponding to the abnormal region in the second image;

[0018] Feature matching processing is performed on the first area image and the second area image, matching objects in the first area image and the second area image are used as target objects, and the behavior of the target object is determined according to the position change of the target object in the first area image and the second area image.

[0019] In a possible implementation manner, determining the behavior of the target object according to the position change of the target object in the first area image and the second area image includes:

[0020] Determining first three-dimensional coordinates of the target object according to the position of the target object in the first area image, and determining second three-dimensional coordinates of the target object according to the position of the target object in the second area image;

[0021] determining a movement trend of the target object according to a difference between the first three-dimensional coordinates and the second three-dimensional coordinates, and determining a movement speed of the target object according to a time difference between the first region image and the second region image and the movement trend;

[0022] The behavior of the target object is compared with the preset illegal behavior to determine whether the behavior of the target object is abnormal; wherein the behavior of the target object includes the movement trend and the movement speed.

[0023] In a possible implementation, when there is an abnormal area, the method further includes:

[0024] Determine whether the image at the previous time node of the target time node is abnormal;

[0025] When an abnormality exists in the image at the previous time node, determining a target object in the first image and the second image and a behavior of the target object;

[0026] When there is no abnormality in the image at the previous time node, determine the target object in the first image and the second image, and perform face recognition processing on the target object; when the identified face of the target object does not match the preset legal face, determine the behavior of the target object, and determine that there is an abnormality in the first image at the target time node.

[0027] In a second aspect, an embodiment of the present invention further provides a vision-based anti-theft monitoring device, comprising:

[0028] An acquisition module, used to acquire a first image at a target time node and a second image before the target time node;

[0029] A difference module, used to generate a corresponding difference image according to the difference between the first image and the second image, and determine whether there is an abnormal area in the difference image;

[0030] The processing module is used to determine the target object in the first image and the second image and the behavior of the target object when there is an abnormal area, and determine that there is a suspicion of theft when the behavior of the target object is abnormal.

[0031] In a third aspect, an embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are used in any one of the above-mentioned vision-based anti-theft monitoring methods.

[0032] In a fourth aspect, an embodiment of the present invention further provides an electronic device, including:

[0033] at least one processor; and,

[0034] a memory communicatively connected to the at least one processor; wherein,

[0035] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the vision-based anti-theft monitoring methods described above.

[0036] In the solution provided in the first aspect of the embodiment of the present invention, a differential image that can represent the difference between images at two different time points is used to determine whether there is an abnormal area in the image. When an abnormal area exists, it can be determined whether there is a suspicion of theft based on the behavior of the moving object. This method can automatically monitor whether there is a suspicion of theft in the user's blind spot, which can improve the security in the blind spot. And by using the differential image to determine whether it is necessary to identify the object and the object's behavior in the image, the processing amount of image recognition can be reduced, so that anti-theft detection can be carried out in a timely manner, improving the timeliness of anti-theft monitoring.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flow chart of a visual-based anti-theft monitoring method provided by an embodiment of the present invention is shown;

[0040] Figure 2 Three schematic diagrams of differential images in the vision-based anti-theft monitoring method provided by an embodiment of the present invention are shown;

[0041] Figure 3a A schematic diagram of an abnormal binocular difference image in the vision-based anti-theft monitoring method provided by an embodiment of the present invention is shown;

[0042] Figure 3b A schematic diagram showing mapping an abnormal area to a binocular image in a vision-based anti-theft monitoring method provided by an embodiment of the present invention;

[0043] Figure 4 A schematic structural diagram of a vision-based anti-theft monitoring device provided by an embodiment of the present invention is shown;

[0044] Figure 5 A schematic structural diagram of an electronic device for executing a vision-based anti-theft monitoring method provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0045] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0046] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0047] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0048] The embodiment of the present invention provides a visual anti-theft monitoring method, see Figure 1 As shown, including:

[0049] Step 101: Acquire a first image at a target time node and a second image before the target time node.

[0050] In the embodiment of the present invention, a pre-installed image acquisition device is used to acquire images of areas that require anti-theft detection (e.g., blind spots on both sides of the vehicle, inside a cargo hold, etc.). The image acquisition device may be a camera, a webcam, etc. The image acquisition device may acquire images in real time. Each frame of the image acquired by the image acquisition device corresponds to an image at a time node. In this embodiment, the target time node corresponds to a frame. The corresponding first image is a frame of the image acquired by the image acquisition device; and the second image is another frame of the image before the first image. Optionally, the second image and the first image are adjacent images on the time axis, that is, the second image is the previous frame of the first image.

[0051] Before step 101, the image acquisition device needs to be calibrated with internal parameters, that is, the internal parameter matrix related to the image acquisition device, such as focal length, principal point, lens distortion, etc., needs to be determined. Optionally, the image acquisition device can be a binocular camera. In this case, in addition to calibrating the above-mentioned internal parameter matrix, the rotation matrix and translation vector between the left camera and the right camera also need to be determined. The calibrated image acquisition device can capture images of the area where anti-theft detection is required.

[0052] Step 102: Generate a corresponding differential image according to the difference between the first image and the second image, and determine whether there is an abnormal area in the differential image.

[0053] In the embodiment of the present invention, the first image and the second image are images collected at different time points of the same anti-theft detection area, and the difference between the first image and the second image can represent the difference between the two, that is, the differential image can be used to represent the difference between the first image and the second image. When there is no abnormality, that is, when there is no theft, the first image and the second image should be exactly the same; when there is a theft, since there is an active object (i.e., a thief) at this time, the images collected at different time points will be slightly different, that is, there is an abnormal area in the differential image at this time. In this embodiment, by judging whether there is an abnormal area in the differential image, it is convenient and quick to judge whether there is a suspicion of theft at the current target time node.

[0054] Optionally, the above step 102 of “generating a corresponding differential image according to the difference between the first image and the second image, and determining whether there is an abnormal area in the differential image” specifically includes:

[0055] Step A1: performing differential processing on the first image and the second image, and generating a differential image representing the difference between the first image and the second image according to the differential processing result.

[0056] In the embodiment of the present invention, after determining the first image and the second image, a differential process can be performed on the two images, and a corresponding differential image can be generated according to the differential process result. Specifically, the above-mentioned "generating a differential image for representing the difference between the first image and the second image according to the differential process result" includes:

[0057] Step A11: Generate a differential initial image according to the differential processing result, and perform binarization processing on the differential image to generate a differential binary image.

[0058] In the embodiment of the present invention, the image that can be directly generated based on the difference processing result of the first image and the second image is the difference initial image. cur , the second image is img last , the corresponding difference processing result is img d=img cur -img last , where img d It is also the above-mentioned differential initial image, that is, the pixel value of each pixel in the differential initial image is the difference between the pixel values ​​of the corresponding pixel in the first image and the second image. If there is no theft, there is no abnormality in the scene. In theory, the first image and the second image are the same, that is, the differential initial image img d = 0, that is, the difference initial image img d is a zero matrix. However, due to the system noise in the image acquisition device, the first image and the second image are not exactly the same, that is, img d ≠ 0. In this embodiment, the differential initial image is binarized to reduce the influence of system noise and simplify the subsequent process of determining whether an abnormality exists based on the differential image.

[0059] Specifically, the differential initial image img d The pixel value of each pixel in the th The difference initial image img is compared based on the comparison result. d Convert to a binary image, that is, a differential binary image img dn .

[0060] For example, Among them, I uv Represents the difference binary image img dn The pixel value of the pixel point with coordinates (u, v), i uv Represents the differential initial image img d The pixel value of the pixel with coordinates (u,v).

[0061] Step A12: dilate the difference binary image, and use the difference binary image after the dilation process as the generated difference image.

[0062] In the embodiment of the present invention, the difference binary image img dn Perform dilation processing to highlight the difference binary image img dn The image after dilation is the difference image img in this embodiment. dnd .

[0063] Step A2: Determine the pixel value of each pixel in the differential image, and take the pixel whose pixel value is within a preset abnormal range as an abnormal pixel.

[0064] In the embodiment of the present invention, if the difference image img dnd If there are a large number of concentrated abnormal points in the differential image img dndThere is an abnormality. Specifically, an abnormal range is pre-set. If the pixel value of a pixel in the differential image is within the abnormal range, the pixel is considered abnormal, that is, the pixel is regarded as an abnormal pixel. For example, if the differential image is a binary image, the pixel with a pixel value of 255 can be regarded as an abnormal pixel, that is, the abnormal range at this time can be: the pixel value is 255. Specifically, the pixel with a pixel value of 255 is displayed as a white dot, and the differential image img dnd See for some cases Figure 2 shown.

[0065] exist Figure 2 middle, Figure 2 The difference image img on the left dnd1 It is a standard normal differential image, that is, the pixel value of each pixel in the differential image is 0 (all black); Figure 2 The middle difference image img dnd2 is a normal differential image with noise. Due to the noise in the camera, there may be noise in the differential image, but the distribution of noise is generally uniform. In this case, the differential image img dnd2 It is normal; Figure 2 The difference image img on the right dnd3 is an abnormal difference image, the difference image img dnd3 There are a large number of concentrated abnormal pixels (i.e., white pixels) in the differential image img dnd3 It is abnormal, that is, a certain area in the first image is different from that in the second image, which indicates that there is an active object in the area.

[0066] Step A3: Determine the total number of sequentially adjacent abnormal pixel points, and when the total number is greater than a preset threshold, take the area where the sequentially adjacent abnormal pixel points are located as the abnormal area.

[0067] In the embodiment of the present invention, if the abnormal pixel points are scattered, it means that the abnormal pixel points may be noise points, and the abnormal pixel points can be ignored; if the abnormal pixel points are concentrated and the number is large (such as Figure 2 ), it means that the abnormal pixel corresponds to a moving object. Specifically, the "adjacent abnormal pixel points in sequence" in this embodiment means that at least one edge (or vertex) of the abnormal pixel point is adjacent to another abnormal pixel point A, and at least one edge (or vertex) of the abnormal pixel point A can also be adjacent to other abnormal pixel point B... That is, the adjacent relationship of the abnormal pixels is sequential. The adjacent abnormal pixels in sequence can form an abnormal pixel set, in which each abnormal pixel point is adjacent to at least one other abnormal pixel point, that is, a certain edge is a common edge, or a certain vertex is a common vertex.

[0068] In this embodiment, the abnormal pixel set is composed of relatively concentrated abnormal pixels. If the total number of abnormal pixels in the abnormal pixel set (i.e., the total number of abnormal pixels adjacent to each other) is greater than a preset threshold, it means that the abnormal pixel set contains a large number of concentrated abnormal pixels, and the differential image can be considered abnormal at this time. At the same time, the area where the abnormal pixels in the abnormal pixel set are located can be used as the abnormal area of ​​the differential image. Among them, those skilled in the art can understand that for the same differential image, there can be multiple groups of abnormal pixels adjacent to each other, that is, there can be multiple abnormal pixel sets, and each abnormal pixel set can correspond to an abnormal area.

[0069] Step 103: When there is an abnormal area, determine the target object and the behavior of the target object in the first image and the second image, and determine that there is a suspicion of theft when the behavior of the target object is abnormal.

[0070] In the embodiment of the present invention, when there is an abnormal area in the differential image, it means that there is a large difference between a certain area of ​​the first image and the second image. At this time, it can be said that there is a moving object (such as a person, an animal, a moving vehicle, etc.) in the area. In this embodiment, the moving object is taken as the target object to be processed. Specifically, the target object in the first image and the second image can be determined based on the image recognition technology, and the behavior of the target object can be determined according to the position change of the target object between the first image and the second image, for example, the target object is approaching the warehouse, or climbing the warehouse, etc.; when the behavior of the target object is similar to the theft behavior, it is determined that the behavior of the target object is abnormal, so that it can be determined that there is a theft suspicion at this time, and then the alarm module can be used to warn the driver that there is a theft suspicion in the driver's blind spot. Since the processing amount of identifying the object and the object behavior in the image is large, it may lead to the problem of untimely processing, but the processing amount of using the differential image to determine whether there is an abnormal area is small. In this embodiment, the differential image is used to determine whether it is necessary to identify the object and the object behavior in the image, which can reduce the processing amount of image recognition, so that anti-theft detection can be carried out in time, and the timeliness of anti-theft monitoring can be improved.

[0071] The embodiment of the present invention provides a vision-based anti-theft monitoring method, which uses a differential image that can represent the difference between images at two different time points to determine whether there is an abnormal area in the image. When there is an abnormal area, it can be determined whether there is a theft suspicion based on the behavior of the moving object. This method can automatically monitor whether there is a theft suspicion in the user's blind spot, which can improve the security in the blind spot. And using the differential image to determine whether it is necessary to identify the object and the object behavior in the image can reduce the processing volume of image recognition, so that anti-theft detection can be carried out in a timely manner, improving the timeliness of anti-theft monitoring.

[0072] Based on the above embodiment, step 103 of “determining the target object and the behavior of the target object in the first image and the second image” includes:

[0073] Step B1: determining a first region image corresponding to the abnormal region in the first image, and a second region image corresponding to the abnormal region in the second image.

[0074] In the embodiment of the present invention, since the differential image completely corresponds to the first image and the second image, the abnormal area of ​​the differential image also corresponds to a certain area in the first image and the second image, and the abnormal area maps the first area image in the first image and corresponds to the second area image in the second image; wherein the position of the abnormal area in the differential image, the position of the first area image in the first image, and the position of the second area image in the second image are all the same.

[0075] Step B2: Perform feature matching processing on the first region image and the second region image, take the matching objects in the first region image and the second region image as the target object, and determine the behavior of the target object according to the position change of the target object in the first region image and the second region image.

[0076] In an embodiment of the present invention, there are active objects in both the first region image and the second region image. Since the active objects are not completely the same in the first region image and the second region image, this embodiment matches the objects in the first region image and the second region image based on a feature matching method, and the objects are the target objects. Before feature matching, the first image and the second image can be subjected to distortion correction based on the predetermined internal parameters of the image acquisition device; in this embodiment, since only the first region image and the second region image need to be processed, only the first region image and the second region image can be subjected to distortion correction, thereby further reducing the processing amount. Optionally, when the image acquisition device is a binocular camera, binocular stereo correction can also be performed on the first region image and the second region image.

[0077] In the embodiment of the present invention, after determining the first region image and the second region image, feature point extraction and feature point matching operations can be performed to determine the target object therein; wherein, the feature point extraction and matching can be performed using an ORB (Oriented FAST and Rotated BRIEF) algorithm, an optical flow method, etc., which is not limited in this embodiment. After determining the target object, the movement speed, movement trend, etc. of the target object can be determined based on the position change of the target object in the first region image and the second region image of different frames, that is, the behavior of the target object can be determined, such as determining whether the target object is approaching a vehicle, or whether it is climbing a vehicle, etc.

[0078] Optionally, the first image and the second image can be collected based on a binocular camera, and accordingly, the first image includes a first left-eye image and a first right-eye image, and the second image includes a second left-eye image and a second right-eye image. After collecting the first and second binocular images, it is also possible to determine whether the behavior of the target object is abnormal based on the above steps 101 to 103. Specifically, since the image collected by the binocular camera includes a left-eye image and a right-eye image, a corresponding left-eye differential image can be generated according to the difference between the first left-eye image and the second left-eye image in step 102, and a corresponding right-eye differential image can be generated according to the difference between the first right-eye image and the second right-eye image, and the left-eye differential image and the right-eye differential image are complete differential images. In addition, an image can also be selected from the left-eye image or the right-eye image, or the left-eye image and the right-eye image can be used as a complete image to determine the corresponding differential image, which is not limited in this embodiment.

[0079] In the above step 103, when it is necessary to map the abnormal area to the first image and the second image, it is necessary to map the abnormal area of ​​the left eye image to the corresponding left eye image, and map the abnormal area of ​​the right eye image to the corresponding right eye image; after the feature points are matched, the three-dimensional coordinates of all matched feature points in the camera coordinate system can be solved using the principle of binocular stereo vision, and the instantaneous motion speed and motion trend of the target object can be calculated according to the position change of the target object in different frames. Specifically, the above step B2 "determining the behavior of the target object according to the position change of the target object in the first area image and the second area image" includes:

[0080] Step B21: determining the first three-dimensional coordinates of the target object according to the position of the target object in the first area image, and determining the second three-dimensional coordinates of the target object according to the position of the target object in the second area image.

[0081] Step B22: determining the movement trend of the target object according to the difference between the first three-dimensional coordinates and the second three-dimensional coordinates, and determining the movement speed of the target object according to the time difference between the first area image and the second area image and the movement trend.

[0082] Step B23: Compare the behavior of the target object with the preset illegal behavior to determine whether the behavior of the target object is abnormal; wherein the behavior of the target object includes movement trend and movement speed.

[0083] In this embodiment, the first image and the second image are both binocular images, and the first area image and the second area image are also binocular images. Based on the coordinate conversion relationship of the binocular camera, the pixel coordinates of the target object in the first area image can be converted into three-dimensional coordinates in the world coordinate system, that is, the first three-dimensional coordinates; similarly, the second three-dimensional coordinates corresponding to the target object in the second area image can also be determined, and the movement trend and movement speed of the target object can be determined based on the changes of the two three-dimensional coordinates. At the same time, the movement trend and movement speed that match the theft behavior are pre-set, that is, the illegal behavior is pre-set. If the movement trend and movement speed of the target object match the illegal behavior, it means that there may be a thief at this time, that is, there is a suspicion of theft.

[0084] Among them, an abnormal differential image of the binocular camera can be found in Figure 3a As shown, Figure 3a img dndL Represents the left eye difference image, img dndR represents the right eye difference image, where the area marked by the white rectangular frame is the corresponding abnormal area. Those skilled in the art will appreciate that other shapes other than the rectangular frame can also be used to select the abnormal area corresponding to the abnormal pixel point. For a schematic diagram of mapping the abnormal area to the first image, see Figure 3b As shown, Figure 3b img curL Represents the first left-eye image in the first image, img curR represents the first right-eye image in the first image, Figure 3b The area marked by the white rectangular frame is the first area image selected from the first image; mapping the abnormal area to the second image to determine the second area image is similar to this, and will not be described in detail here.

[0085] In an embodiment of the present invention, by mapping the abnormal area to the first image and the second image, the first area image and the second area image corresponding to the abnormal area are selected. When performing subsequent image recognition and behavior judgment, only the first area image and the second area image need to be processed, which can further reduce the processing amount and effectively ensure real-time monitoring.

[0086] On the basis of the above embodiment, it is also possible to determine whether the target object is legal based on face recognition technology, and then identify the behavior of the target object when the target object is illegal. Specifically, when there is an abnormal area, the method also includes:

[0087] Step C1: Determine whether there is any abnormality in the image at the previous time node of the target time node.

[0088] In an embodiment of the present invention, when it is determined that there is an abnormal area in the differential image, it can be preliminarily considered that there is an abnormality in the first image. At the same time, the method can be executed in a loop, so that it can be determined whether each frame of the image is abnormal. For example, after the image acquisition device acquires the image, the operation starts from the second frame of the image (i.e., the first image), and the differential image between the second frame of the image and the first frame of the image (i.e., the second image) is calculated, so that it can be determined whether there is an abnormality in the second frame of the image; then the third frame of the image can be used as the first image and the second frame of the image can be used as the second image, and the above steps 101 to 103 are repeated again, so that it can be determined whether there is an abnormality in the third frame of the image. Specifically, when the behavior of the target object in the abnormal area is abnormal, the first image can be used as an abnormal image, that is, it is considered that there is an abnormality in the first image.

[0089] Optionally, if a frame of image is abnormal, an abnormal mark can be added to the image, and then whether the image is abnormal can be determined based on whether the image contains the abnormal mark, that is, whether the image at the previous time node is abnormal. The previous time node is the previous time node of the target time node; specifically, the image at the previous time node can be the previous frame of the first image.

[0090] Step C2: when there is an abnormality in the image at the previous time node, determine the target object and the behavior of the target object in the first image and the second image.

[0091] Step C3: When there is no abnormality in the image at the previous time node, determine the target object in the first image and the second image, and perform face recognition processing on the target object; when the identified face of the target object does not match the preset legal face, determine the behavior of the target object.

[0092] In an embodiment of the present invention, legal face samples are collected in advance, that is, faces that do not need to be monitored, such as the driver's face, etc. The face samples cover multiple angles of the face as much as possible, and legal faces are generated based on the face samples. If the image at the current time node (i.e., the target time node) has an abnormality, and the image at the previous time node does not have an abnormality, it means that the abnormality is first collected at the current time node. At this time, it is necessary to perform face recognition processing on the target image in the image. If the target object is identified as a preset legal face, it can be considered that the behavior of the target object at this time is also legal, that is, there is no abnormality in its behavior; optionally, the first image at this time can be backed up, and then the image of the next frame can be selected as the first image to continue to determine whether there is an abnormality.

[0093] If there is an abnormality in the image at the current time node and there is an abnormality in the image at the previous time node, it means that the behavior of the target object has been judged to be abnormal at the previous time node, and face recognition has been performed at the previous time node or an earlier time node, that is, it has been determined in the previous processing process that the target object is not a legal face, so there is no need to repeat the face recognition process at this time, thereby further reducing the processing volume.

[0094] The embodiment of the present invention provides a visual anti-theft monitoring method, which uses a differential image that can represent the difference between images at two different time nodes to determine whether there is an abnormal area in the image. When there is an abnormal area, it can be determined whether there is a suspicion of theft based on the behavior of the moving object. This method can automatically monitor whether there is a suspicion of theft in the blind area of ​​the user's visual field, and can improve the safety in the blind area of ​​the visual field. And using the differential image to determine whether it is necessary to identify the object and the object behavior in the image can reduce the processing amount of image recognition, so that anti-theft detection can be performed in time, and the timeliness of anti-theft monitoring can be improved. By mapping the abnormal area to the first image and the second image, the first area image and the second area image corresponding to the abnormal area are selected. When performing image recognition and behavior judgment in the subsequent process, only the first area image and the second area image need to be processed, which can further reduce the processing amount and effectively ensure the realization of real-time monitoring. Using face recognition technology can further screen out illegal target objects, and by determining whether there is an abnormality at the previous time node, repeated face recognition can be effectively avoided, which can also reduce the processing amount in the anti-theft monitoring process.

[0095] The above describes in detail the process of the vision-based anti-theft monitoring method. The method can also be implemented by a corresponding device. The structure and function of the device are described in detail below.

[0096] The embodiment of the present invention provides a visual anti-theft monitoring device, see Figure 4 As shown, including:

[0097] An acquisition module 41 is used to acquire a first image at a target time node and a second image before the target time node;

[0098] A difference module 42, configured to generate a corresponding difference image according to the difference between the first image and the second image, and determine whether there is an abnormal area in the difference image;

[0099] The processing module 43 is used to determine the target object and the behavior of the target object in the first image and the second image when there is an abnormal area, and determine that there is a suspicion of theft when the behavior of the target object is abnormal.

[0100] Based on the above embodiment, the difference module 42 includes:

[0101] a differential processing unit, configured to perform differential processing on the first image and the second image, and generate a differential image representing a difference between the first image and the second image according to a differential processing result;

[0102] an abnormal pixel point determination unit, used to determine the pixel value of each pixel point in the differential image, and to take the pixel points whose pixel values ​​are within a preset abnormal range as abnormal pixels;

[0103] The abnormal region determining unit is used to determine the total number of the abnormal pixel points that are adjacent to each other in sequence, and when the total number is greater than a preset threshold, the region where the abnormal pixel points that are adjacent to each other in sequence are located is regarded as the abnormal region.

[0104] Based on the above embodiment, the difference processing unit generates a difference image representing the difference between the first image and the second image according to the difference processing result, including:

[0105] Generating a differential initial image according to the differential processing result, and performing binarization processing on the differential image to generate a differential binary image;

[0106] The differential binary image is dilated, and the differential binary image after the dilation process is used as the generated differential image.

[0107] On the basis of the above embodiment, the first image includes a first left-eye image and a first right-eye image, and the second image includes a second left-eye image and a second right-eye image.

[0108] Based on the above embodiment, the processing module 43 determines the target object in the first image and the second image and the behavior of the target object including:

[0109] Determine a first region image corresponding to the abnormal region in the first image, and a second region image corresponding to the abnormal region in the second image;

[0110] Feature matching processing is performed on the first area image and the second area image, matching objects in the first area image and the second area image are used as target objects, and the behavior of the target object is determined according to the position change of the target object in the first area image and the second area image.

[0111] On the basis of the above embodiment, the processing module 43 determines the behavior of the target object according to the position change of the target object in the first area image and the second area image, including:

[0112] Determining first three-dimensional coordinates of the target object according to the position of the target object in the first area image, and determining second three-dimensional coordinates of the target object according to the position of the target object in the second area image;

[0113] determining a movement trend of the target object according to a difference between the first three-dimensional coordinates and the second three-dimensional coordinates, and determining a movement speed of the target object according to a time difference between the first region image and the second region image and the movement trend;

[0114] The behavior of the target object is compared with the preset illegal behavior to determine whether the behavior of the target object is abnormal; wherein the behavior of the target object includes the movement trend and the movement speed.

[0115] Based on the above embodiment, the processing module 43 is further used for:

[0116] When there is an abnormal area, determine whether there is an abnormality in the image of the previous time node of the target time node;

[0117] When an abnormality exists in the image at the previous time node, determining a target object in the first image and the second image and a behavior of the target object;

[0118] When there is no abnormality in the image at the previous time node, determine the target object in the first image and the second image, and perform face recognition processing on the target object; when the identified face of the target object does not match the preset legal face, determine the behavior of the target object, and determine that there is an abnormality in the first image at the target time node.

[0119] The embodiment of the present invention provides a visual anti-theft monitoring device, which uses a differential image that can represent the difference between images at two different time nodes to determine whether there is an abnormal area in the image. When there is an abnormal area, it can be determined whether there is a suspected theft according to the behavior of the moving object. This method can automatically monitor whether there is a suspected theft in the blind area of ​​the user's visual field, and can improve the safety in the blind area of ​​the visual field. And using the differential image to determine whether it is necessary to identify the object and the object behavior in the image can reduce the processing volume of image recognition, so that anti-theft detection can be performed in time, and the timeliness of anti-theft monitoring can be improved. By mapping the abnormal area to the first image and the second image, the first area image and the second area image corresponding to the abnormal area are selected. When performing image recognition and behavior judgment in the subsequent process, only the first area image and the second area image need to be processed, which can further reduce the processing volume and effectively ensure the realization of real-time monitoring. Using face recognition technology can further screen out illegal target objects, and by determining whether there is an abnormality at the previous time node, repeated face recognition can be effectively avoided, which can also reduce the processing volume in the anti-theft monitoring process.

[0120] An embodiment of the present invention further provides a computer storage medium storing computer executable instructions, which includes a program for executing the above-mentioned vision-based anti-theft monitoring method. The computer executable instructions can execute the method in any of the above-mentioned method embodiments.

[0121] Among them, the computer storage medium can be any available medium or data storage device that can be accessed by the computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0122] Figure 5 The block diagram of the structure of an electronic device of another embodiment of the present invention is shown. The electronic device 1100 may be a host server with computing capability, a personal computer PC, or a portable computer or terminal, etc. The specific embodiment of the present invention does not limit the specific implementation of the electronic device.

[0123] The electronic device 1100 includes at least one processor 1110, a communications interface 1120, a memory array 1130, and a bus 1140. The processor 1110, the communications interface 1120, and the memory array 1130 communicate with each other via the bus 1140.

[0124] The communication interface 1120 is used to communicate with network elements, where the network elements include, for example, a virtual machine management center, shared storage, etc.

[0125] The processor 1110 is used to execute programs. The processor 1110 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0126] The memory 1130 is used for executable instructions. The memory 1130 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory. The memory 1130 may also be a memory array. The memory 1130 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules. The instructions stored in the memory 1130 can be executed by the processor 1110, so that the processor 1110 can execute the visual-based anti-theft monitoring method in any of the above method embodiments.

[0127] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A visual-based anti-theft monitoring method, characterized in that: include: Acquire a first image at a target time node and a second image before the target time node; generating a corresponding differential image according to the difference between the first image and the second image, and determining whether there is an abnormal area in the differential image; When there is an abnormal area, determining a target object in the first image and the second image and a behavior of the target object, and determining that there is a suspicion of theft when the behavior of the target object is abnormal; The determining of the target object in the first image and the second image and the behavior of the target object includes: Determine a first region image corresponding to the abnormal region in the first image, and a second region image corresponding to the abnormal region in the second image; Performing feature matching processing on the first region image and the second region image, taking the matching objects in the first region image and the second region image as target objects, and determining the behavior of the target object according to the position change of the target object in the first region image and the second region image; Among them, when there is an abnormal area, it also includes: Determine whether the image at the previous time node of the target time node is abnormal; When an abnormality exists in the image at the previous time node, determining a target object in the first image and the second image and a behavior of the target object; When there is no abnormality in the image at the previous time node, determine the target object in the first image and the second image, and perform face recognition processing on the target object; when the identified face of the target object does not match the preset legal face, determine the behavior of the target object, and determine that there is an abnormality in the first image at the target time node.

2. The method according to claim 1, characterized in that Generating a corresponding differential image according to the difference between the first image and the second image, and judging whether there is an abnormal area in the differential image, comprises: performing differential processing on the first image and the second image, and generating a differential image representing a difference between the first image and the second image according to a differential processing result; Determine the pixel value of each pixel in the differential image, and take the pixel whose pixel value is within a preset abnormal range as an abnormal pixel; The total number of the abnormal pixel points that are adjacent to each other in sequence is determined, and when the total number is greater than a preset threshold, an area where the abnormal pixel points that are adjacent to each other in sequence are located is taken as an abnormal area.

3. The method according to claim 2, characterized in that The step of generating a differential image representing a difference between the first image and the second image according to a differential processing result includes: Generating a differential initial image according to the differential processing result, and performing binarization processing on the differential image to generate a differential binary image; The differential binary image is dilated, and the differential binary image after the dilation process is used as the generated differential image.

4. The method according to claim 1, characterized in that: The first image includes a first left-eye image and a first right-eye image, and the second image includes a second left-eye image and a second right-eye image.

5. The method according to claim 1, characterized in that The determining of the behavior of the target object according to the position change of the target object in the first area image and the second area image comprises: Determining first three-dimensional coordinates of the target object according to the position of the target object in the first area image, and determining second three-dimensional coordinates of the target object according to the position of the target object in the second area image; determining a movement trend of the target object according to a difference between the first three-dimensional coordinates and the second three-dimensional coordinates, and determining a movement speed of the target object according to a time difference between the first region image and the second region image and the movement trend; The behavior of the target object is compared with the preset illegal behavior to determine whether the behavior of the target object is abnormal; wherein the behavior of the target object includes the movement trend and the movement speed.

6. A visual anti-theft monitoring device, characterized in that: include: An acquisition module, used to acquire a first image at a target time node and a second image before the target time node; A difference module, used to generate a corresponding difference image according to the difference between the first image and the second image, and determine whether there is an abnormal area in the difference image; a processing module, configured to determine, when an abnormal area exists, a target object in the first image and the second image and a behavior of the target object, and determine that there is a suspicion of theft when the behavior of the target object is abnormal; The processing module is specifically used to determine a first region image corresponding to the abnormal region in the first image, and a second region image corresponding to the abnormal region in the second image; Performing feature matching processing on the first region image and the second region image, taking the matching objects in the first region image and the second region image as target objects, and determining the behavior of the target object according to the position change of the target object in the first region image and the second region image; The processing module is further specifically used to determine whether the image of the previous time node of the target time node is abnormal when there is an abnormal area; When an abnormality exists in the image at the previous time node, determining a target object in the first image and the second image and a behavior of the target object; When there is no abnormality in the image at the previous time node, determining the target object in the first image and the second image, and performing face recognition processing on the target object; When the identified face of the target object does not match the preset legal face, the behavior of the target object is determined, and it is determined that the first image of the target time node is abnormal.

7. A computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, and the computer executable instructions are used to execute the vision-based anti-theft monitoring method according to any one of claims 1-5.

8. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the vision-based anti-theft monitoring method described in any one of claims 1-5.

Images