Method, device, electronic device and storage medium for detecting abnormality of an object

Abstract

The present disclosure provides a method for detecting an abnormality of an object, the method comprising: in response to determining that a first target object exists in an image to be detected, determining a first overlapping relationship between the first target object and a predetermined detection area; and determining whether the first target object in the image to be detected is abnormal based on the first overlapping relationship, wherein the image to be detected includes the predetermined detection area. The present disclosure also provides a device for detecting an abnormality of an object, an electronic device, and a computer-readable storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet technology, and more specifically, to a method, device, electronic device, and storage medium for detecting abnormality of an object. Background Art

[0002] In order to ensure safety, items are usually placed in a safe location. Sometimes, due to operational errors or forgetfulness, the placement of items may be problematic. Inevitably, the placement of items needs to be checked to ensure that there is no abnormal placement of items.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: abnormal placement of currency usually causes safety hazards and brings inconvenience to the management of currency by various enterprises. Summary of the invention

[0004] In view of this, the present disclosure provides a method, an apparatus, an electronic device, and a storage medium for detecting an abnormality of an object, which are capable of improving detection accuracy.

[0005] One aspect of the present disclosure provides a method for detecting an abnormality of an object, comprising: in response to determining that a first target object exists in an image to be detected, determining a first overlapping relationship between the first target object and a predetermined detection area; and based on the first overlapping relationship, determining whether the first target object in the image to be detected is abnormal, wherein the image to be detected includes the predetermined detection area.

[0006] Optionally, determining whether the first target object in the image to be detected is abnormal based on the first overlapping relationship includes: determining that the first target object in the image to be detected is abnormal when the first overlapping relationship indicates that there is no overlap between the first target object and the predetermined detection area.

[0007] Optionally, the above-mentioned determining whether the above-mentioned first target object in the above-mentioned image to be detected is abnormal based on the above-mentioned first overlapping relationship includes: when the above-mentioned first overlapping relationship indicates that the above-mentioned first target object overlaps with the above-mentioned predetermined detection area, determining whether there is a second target object in the above-mentioned image to be detected; when the above-mentioned second target object does not exist, determining that the above-mentioned first target object in the above-mentioned image to be detected is abnormal.

[0008] Optionally, the above-mentioned determining whether the above-mentioned first target object in the above-mentioned image to be detected is abnormal based on the above-mentioned first overlapping relationship also includes: in the case that the above-mentioned second target object exists, determining the second overlapping relationship between the above-mentioned second target object and the above-mentioned predetermined detection area; and determining whether the above-mentioned first target object in the above-mentioned image to be detected is abnormal based on the above-mentioned second overlapping relationship.

[0009] Optionally, the above-mentioned determining whether the above-mentioned first target object in the above-mentioned image to be detected is abnormal based on the above-mentioned second overlapping relationship includes: determining the category and quantity of the above-mentioned first target object when the above-mentioned second overlapping relationship indicates that the above-mentioned second target object does not overlap with the above-mentioned predetermined detection area; determining the value of the above-mentioned first target object based on the category and quantity of the above-mentioned first target object; and determining that the above-mentioned first target object in the above-mentioned image to be detected is abnormal when the above-mentioned value is greater than a predetermined value.

[0010] Optionally, the method further includes: determining the number of images in which the first target object is abnormal among a plurality of images to be detected included in the video to be detected; and determining that the first target object is abnormal when the number of images is greater than a predetermined ratio.

[0011] Optionally, the predetermined detection area includes multiple areas; determining the second overlapping relationship between the second target object and the predetermined detection area includes: determining the area where the first target object is located in the multiple areas as the target area; and determining the overlapping relationship between the second target object and the target area.

[0012] Another aspect of the present disclosure provides a device for detecting object abnormalities, comprising: a relationship determination module, for determining a first overlapping relationship between the first target object and a predetermined detection area in response to determining that there is a first target object in the image to be detected; and a first abnormality discrimination module, for determining whether the first target object in the image to be detected is abnormal based on the first overlapping relationship, wherein the image to be detected includes the predetermined detection area.

[0013] Optionally, the first abnormality identification module is used to determine that the first target object in the image to be detected is abnormal when the first overlapping relationship indicates that the first target object does not overlap with the predetermined detection area.

[0014] Optionally, the first abnormality discrimination module includes: an object determination submodule, used to determine whether a second target object exists in the image to be detected when the first overlapping relationship indicates that the first target object overlaps with the predetermined detection area; and a discrimination submodule, used to determine that the first target object in the image to be detected is abnormal when the second target object does not exist.

[0015] Optionally, the above-mentioned first abnormality discrimination module also includes: a relationship determination submodule, used to determine the second overlapping relationship between the above-mentioned second target object and the above-mentioned predetermined detection area when the above-mentioned second target object exists; and the above-mentioned discrimination submodule is used to determine whether the above-mentioned first target object in the above-mentioned image to be detected is abnormal based on the above-mentioned second overlapping relationship.

[0016] Optionally, the above-mentioned discrimination submodule includes: an object information determination unit, used to determine the category and quantity of the above-mentioned first target object when the above-mentioned second overlapping relationship indicates that the above-mentioned second target object does not overlap with the above-mentioned predetermined detection area; a value determination unit, used to determine the value of the above-mentioned first target object based on the category and quantity of the above-mentioned first target object; and a discrimination unit, used to determine that the above-mentioned first target object in the above-mentioned image to be detected is abnormal when the above-mentioned value is greater than a predetermined value.

[0017] Optionally, the device also includes: an image quantity determination module, used to determine the number of images of the above-mentioned first target object that are abnormal among multiple images to be detected included in the video to be detected; and a second abnormality judgment module, used to determine that the above-mentioned first target object is abnormal when the number of the above-mentioned images is greater than a predetermined ratio.

[0018] Optionally, the predetermined detection area includes multiple areas; the relationship determination submodule includes: a target area determination unit, used to determine the area where the first target object is located among the multiple areas as the target area; and an overlap determination unit, used to determine the overlap relationship between the second target object and the target area.

[0019] Another aspect of the present disclosure provides an electronic device, which includes computer executable instructions. When the instructions are executed, they are used to implement the method of the embodiment of the present disclosure.

[0020] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method of the embodiment of the present disclosure when executed.

[0021] According to the embodiments of the present disclosure, since a technical means is adopted to determine whether the first target object in the image to be detected is abnormal based on the first overlapping relationship between the first target object in the image to be detected and the predetermined detection area, the technical problem of the safety hazard caused by the abnormal placement of objects and the inconvenience brought to the enterprise in the management of objects is at least partially overcome, thereby achieving the technical effect of avoiding the safety hazard caused by the abnormal placement of currency and facilitating the enterprise's management of objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0023] Figure 1 An exemplary system architecture of a method and apparatus for detecting object abnormality according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 2 A flowchart of a method for detecting abnormality of an object according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 3 A flowchart of a method for detecting abnormality of an object according to another embodiment of the present disclosure is schematically shown;

[0026] Figure 4 A flowchart of a method for detecting abnormality of an object according to another embodiment of the present disclosure is schematically shown;

[0027] Figure 5 A flowchart of a method for detecting abnormality of an object according to another embodiment of the present disclosure is schematically shown;

[0028] Figure 6 A flowchart of a method for detecting abnormality of an object according to another embodiment of the present disclosure is schematically shown;

[0029] Figure 7 A flowchart of a method for determining a second overlapping relationship according to an embodiment of the present disclosure is schematically shown;

[0030] Figure 8 A block diagram of a device for detecting abnormality of an object according to an embodiment of the present disclosure is schematically shown;

[0031] Fig. 9 A block diagram schematically shows an apparatus for detecting abnormality of an object according to another embodiment of the present disclosure; and

[0032] Fig.10 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0034] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0035] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0036] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0037] An embodiment of the present disclosure provides a method for detecting object abnormality. The method includes: in response to determining that a first target object exists in an image to be detected, determining a first overlapping relationship between the first target object and a predetermined detection area; and determining whether the first target object in the image to be detected is abnormal based on the first overlapping relationship, wherein the image to be detected includes the predetermined detection area.

[0038] Figure 1 The exemplary system architecture of the method and apparatus for detecting object abnormality according to an embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0039] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0040] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0041] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0042] The server 105 may be a server that provides various services, such as a background management server that provides support for websites browsed by users using the terminal devices 101 , 102 , and 103 (only an example).

[0043] It should be noted that the method for detecting object abnormality provided in the embodiment of the present disclosure can generally be executed by the terminal device 101, 102, 103 or the server 105. Accordingly, the apparatus for detecting object abnormality provided in the embodiment of the present disclosure can generally be set in the terminal device 101, 102, 103 or the server 105.

[0044] It should be understood that Figure 1 The number and type of terminal devices, networks and servers in the embodiment are only for illustration purposes. Any number and type of terminal devices, networks and servers may be provided as required.

[0045] The method for detecting object abnormality in the disclosed embodiment can be applied to detecting currency abnormally placed at a counter of a financial institution. The currency may not be placed in the working area of ​​the staff of the financial institution. It should be understood that when the currency is not placed in the working area of ​​the staff of the financial institution, it can be confirmed that the currency is abnormal.

[0046] Figure 2 The flowchart of the method for detecting abnormality of an object according to an embodiment of the present disclosure is schematically shown.

[0047] like Figure 2 As shown, the method includes operations S210 to S220.

[0048] In operation S210 , in response to determining that a first target object exists in the image to be detected, a first overlapping relationship between the first target object and a predetermined detection area is determined.

[0049] In operation S220, based on the first overlapping relationship, it is determined whether the first target object in the image to be detected is abnormal. The image to be detected includes a predetermined detection area.

[0050] According to the embodiments of the present disclosure, target detection can be performed on the image to be detected, and the detected target is a predetermined detection area, so as to determine the position of the predetermined detection area in the image to be detected. Alternatively, the predetermined detection area to be detected can be marked by the detection personnel in the image to be detected. Through the above two methods, the position information of the detection frame of the predetermined detection area in the image to be detected can be obtained. For example, the detection frame of the predetermined detection area can be a rectangular frame, and the position information can include the coordinate value of the center point of the detection frame of the predetermined detection area in the image to be detected and the height and width of the detection frame of the predetermined detection area in the image to be detected.

[0051] Exemplarily, the predetermined detection area may be a working area of ​​a staff of a financial institution, and the image to be detected may be an image in a video captured by a camera of the financial institution. The position information of the working area of ​​the staff of the financial institution in the image to be processed is determined in the image to be detected.

[0052] According to an embodiment of the present disclosure, while determining the position of a predetermined detection area in an image to be detected, target detection may be performed on the image to be detected to determine whether a first target object exists in the image to be detected.

[0053] Exemplarily, the first target object may be currency, the area to be detected may be the working area of ​​a staff member of a financial institution, and the image to be detected may be an image in a video captured by a camera of a financial institution. Target detection is performed on the image to be detected in the video captured by the camera of the financial institution to determine whether there is currency in the image to be detected. If there is currency in the image to be detected, the position information of the detection frame of the currency in the image to be detected can be obtained through target detection. For example, the detection frame of the currency may be a rectangular frame, and the position information may include the height and width of the detection frame of the currency in the image to be detected and the coordinate value of the center point of the detection frame of the currency in the image to be detected.

[0054] For example, a target detection model based on deep learning can be used for target detection. Since the currency is small, it is difficult to detect as a detection target. In order to ensure the accuracy and efficiency of detection, the yolov5 algorithm can be used to build a target detection model. Among them, the yolov5 algorithm is a single-stage target detection algorithm.

[0055] If the first target object exists in the image to be detected, in response to determining that the first target object exists in the image to be detected, a first overlapping relationship between the first target object and the predetermined detection area is determined. The first target object may overlap with the predetermined detection area, or may not overlap with the predetermined detection area. The intersection-and-union ratio between the first target object and the predetermined detection area may be calculated based on the position information of the detection frame of the first target object and the position information of the detection frame of the predetermined detection area. When the intersection-and-union ratio is greater than 0, the first target object overlaps with the predetermined detection area. When the intersection-and-union ratio is not greater than 0, the first target object does not overlap with the predetermined detection area.

[0056] According to an embodiment of the present disclosure, based on the first overlapping relationship, it can be determined whether the first target object in the image to be detected is abnormal. If the first target object does not overlap with the predetermined detection area, it can be determined that the first target object in the image to be detected is abnormal. If the first target object overlaps with the predetermined detection area, it can be determined that the first target object in the image to be detected is normal.

[0057] Exemplarily, when the currency does not overlap with the working area of ​​the financial institution staff, it can be determined that the currency is not in the working area, thereby determining that the placement of the currency in the image to be detected is abnormal. On the contrary, when the currency overlaps with the working area of ​​the financial institution staff, it can be determined that the placement of the currency in the image to be detected is normal.

[0058] The disclosed embodiment determines whether the first target object in the image to be detected is abnormal by determining whether the first target object overlaps with the predetermined detection area, and can accurately and efficiently confirm that the currency is placed abnormally when the currency is not placed in the working area of ​​the financial institution staff. Therefore, the potential safety hazards caused by the abnormal placement of currency can be avoided, which is beneficial to the management of currency by financial institutions. Compared with the related art, the time for manually determining the abnormal placement of currency can be saved, thereby saving manpower.

[0059] According to an embodiment of the present disclosure, determining whether the first target object in the image to be detected is abnormal based on the first overlapping relationship includes: determining that the first target object in the image to be detected is abnormal when the first overlapping relationship indicates that there is no overlap between the first target object and a predetermined detection area.

[0060] According to an embodiment of the present disclosure, based on the first overlapping relationship, it can be determined whether the first target object in the image to be detected is abnormal. If the first overlapping relationship indicates that the first target object does not overlap with the predetermined detection area, it can be determined that the first target object in the image to be detected is abnormal.

[0061] For example, if there is currency in the image to be detected in the video captured by the camera of the financial institution, the intersection-and-union ratio between the currency detection frame and the predetermined detection area is calculated based on the position information of the currency detection frame and the position information of the detection frame of the predetermined detection area. When the intersection-and-union ratio is not greater than 0, there is no overlap between the currency and the predetermined detection area, and it can be determined that the currency in the image to be detected is abnormal.

[0062] In the embodiment of the present disclosure, when the first overlapping relationship indicates that there is no overlap between the first target object and the predetermined detection area, it is determined that the first target object in the image to be detected is abnormal, and the currency abnormality can be confirmed when the currency is not placed in the working area of ​​the financial institution staff.

[0063] Figure 3 A flow chart of a method for detecting abnormality of an object according to another embodiment of the present disclosure is schematically shown.

[0064] like Figure 3 As shown, the method includes operations S210, S310-S320.

[0065] In operation S210, in response to determining that a first target object exists in the image to be detected, a first overlapping relationship between the first target object and a predetermined detection area is determined. Figure 2 The operation S210 shown is similar and will not be described again.

[0066] In operation S310, when a first overlapping relationship indicates an overlap between a first target object and a predetermined detection area, it is determined whether a second target object exists in the image to be detected.

[0067] In operation S320 , when the second target object does not exist, it is determined that the first target object in the to-be-detected image is abnormal.

[0068] Determine whether there is a first target object in the image to be detected, and determine a first overlapping relationship between the first target object and a predetermined detection area. In the case where the first overlapping relationship indicates that the first target object overlaps with the predetermined detection area, determine whether there is a second target object in the image to be detected. If there is a second target object, the position information of the detection frame of the second target object in the image to be detected can be obtained through target detection. For example, the detection frame of the second target object can be a rectangular frame, and the position information can include the height and width of the detection frame of the second target object in the image to be detected and the coordinate value of the center point of the detection frame of the second target object in the image to be detected.

[0069] Exemplarily, the second target object may be a staff member of a financial institution, the area to be detected may be a working area of ​​a staff member of a financial institution, and the image to be detected may be an image to be detected included in a video shot by a camera of a financial institution. Target detection is performed on the image to be detected included in the video shot by the camera of the financial institution to determine whether there is a person in the image to be detected. If there is a person in the image to be detected, the position information of the detection frame for the person in the image to be detected can be obtained by target detection. For example, the detection frame for the person may be a rectangular frame, and the position information may include the height, width and coordinate value of the center point of the detection frame for the person in the image to be detected. In the case where there is a first target object in the image to be detected and the first target object overlaps with the predetermined detection area, but there is no second target object in the image to be detected, it can be determined that the first target object in the image to be detected is abnormal. At this time, the first target object overlaps with the predetermined detection area, or the first target object does not overlap with the predetermined detection area, and it can be determined that the first target object in the image to be detected is abnormal.

[0070] Exemplarily, when there is currency in the image to be detected included in the video shot by the camera of a financial institution, the currency overlaps with the working area of ​​the financial institution staff, and there is no person in the image to be detected included in the video shot by the camera of the financial institution, it can be determined that the currency in the image to be detected included in the video shot by the camera of the financial institution is abnormal.

[0071] In the embodiment of the present disclosure, when the first target object overlaps with the predetermined detection area and the second target object does not exist in the image to be detected, it is determined that the first target object in the image to be detected is abnormal. When money is placed in the working area of ​​the staff of the financial institution, but there is no person in the image to be detected included in the video captured by the camera of the financial institution, it can be accurately and efficiently confirmed that the money is placed abnormally. Therefore, the potential safety hazard caused by the unattended money can be avoided, which is beneficial to the management of money by the financial institution. Compared with the related art, it can save time for manually determining the abnormal placement of money, thereby saving manpower.

[0072] Figure 4 A flowchart of a method for detecting abnormality of an object according to yet another embodiment of the present disclosure is schematically shown.

[0073] like Figure 4 As shown, the method includes operations S310, S410-S420.

[0074] In operation S310, when a first overlapping relationship indicates an overlap between a first target object and a predetermined detection area, it is determined whether a second target object exists in the image to be detected.

[0075] This operation and Figure 3 The operation S310 shown is the same and will not be described again.

[0076] In operation S410 , in a case where the second target object exists, a second overlapping relationship between the second target object and a predetermined detection area is determined.

[0077] In operation S420, based on the second overlapping relationship, it is determined whether the first target object in the image to be detected is abnormal.

[0078] If the first overlapping relationship indicates that the first target object overlaps with the predetermined detection area, it is determined whether there is a second target object in the image to be detected. If there is a second target object in the image to be detected, a second overlapping relationship between the second target object and the predetermined detection area is determined.

[0079] The second target object may overlap with the predetermined detection area, or may not overlap with the predetermined detection area. The intersection-and-union ratio between the detection frame of the second target object and the predetermined detection area may be calculated based on the position information of the detection frame of the second target object and the position information of the detection frame of the predetermined detection area. When the intersection-and-union ratio is greater than 0, the second target object overlaps with the predetermined detection area. When the intersection-and-union ratio is not greater than 0, the second target object does not overlap with the predetermined detection area.

[0080] According to an embodiment of the present disclosure, when the first overlapping relationship indicates that the first target object overlaps with the predetermined detection area, and there is a second target object in the image to be detected, it is determined whether the first target object in the image to be detected is abnormal based on the second overlapping relationship between the second target object and the predetermined detection area. If the second target object overlaps with the predetermined detection area, the first target object in the image to be detected is normal; otherwise, it is detected that the first target object in the image to be detected is abnormal.

[0081] Exemplarily, when the currency overlaps with the working area of ​​the financial institution staff and there is a financial institution staff in the image to be detected, whether the currency in the image to be detected is abnormal is determined based on whether the person overlaps with the working area of ​​the financial institution staff. If the person overlaps with the working area of ​​the financial institution staff, the currency is normal, otherwise, the currency is abnormal.

[0082] In the disclosed embodiment, when the first target object overlaps with the predetermined detection area and the second target object exists in the image to be detected, whether the first target object in the image to be detected is abnormal can be determined based on whether the second target object overlaps with the predetermined detection area. When the currency is placed in the working area of ​​the staff of the financial institution and there is a task in the image to be detected included in the video captured by the camera of the financial institution, the currency abnormality can be accurately and efficiently confirmed based on whether the task is in the working area of ​​the staff of the financial institution. Therefore, the potential safety hazard when the currency in the working area of ​​the staff of the financial institution is unattended can be avoided, which is beneficial to the management of currency by the financial institution. Compared with the related art, the time of manually determining the abnormal placement of currency can be saved, thereby saving manpower.

[0083] Figure 5 A flowchart of a method for detecting abnormality of an object according to yet another embodiment of the present disclosure is schematically shown.

[0084] like Figure 5 As shown, the method includes operations S310, S410, and S510-S530.

[0085] In operation S310, when a first overlapping relationship indicates an overlap between a first target object and a predetermined detection area, it is determined whether a second target object exists in the image to be detected.

[0086] This operation and Figure 3 The operation S310 shown is the same and will not be described again.

[0087] In operation S410 , in a case where the second target object exists, a second overlapping relationship between the second target object and a predetermined detection area is determined.

[0088] This operation and Figure 4 The operation S410 shown is the same and will not be described again.

[0089] In operation S510 , when the second overlapping relationship indicates that there is no overlap between the second target object and a predetermined detection area, a category and a number of the first target object are determined.

[0090] In operation S520 , the value of the first target object is determined based on the category and the quantity of the first target object.

[0091] In operation S530 , when the value is greater than a predetermined value, it is determined that the first target object in the image to be detected is abnormal.

[0092] In the case where the first overlapping relationship indicates that the first target object overlaps with the predetermined detection area, it is determined whether there is a second target object in the image to be detected. In the case where the second target object exists in the image to be detected, a second overlapping relationship between the second target object and the predetermined detection area is determined. In the case where the second overlapping relationship indicates that the second target object does not overlap with the predetermined detection area, the category and quantity of the first target object are determined.

[0093] Exemplarily, when the currency overlaps with the working area of ​​the financial institution staff, and there is a person in the image to be detected, it is determined whether the person overlaps with the working area of ​​the financial institution staff. If the financial person and the working area of ​​the financial institution staff do not overlap, the type and amount of the currency are determined.

[0094] For example, the categories of currency may include: 1 yuan, 5 yuan, 10 yuan, 20 yuan, 50 yuan, 100 yuan, etc. When handling currency, the counter of a financial institution often handles currency in "bundles", and the number of currency in a bundle is fixed. The number of currencies can be: 1 bundle, 2 bundles, 3 bundles, etc. The category of currency and the number of currencies can be detected by a deep learning-based target detection model, and the yolov5 algorithm can be used to build a target detection model.

[0095] Based on the category and quantity of the first target object, a value calculation is performed on the first target object to determine the value of the first target object.

[0096] For example, the number of currency in a bundle is set to 100. If the number of currency of the 5-yuan category is 1 bundle and the number of currency of the 10-yuan category is 2 bundles, the value of the currency is: 5 yuan × 1 × 100 + 10 yuan × 2 × 100 = 2,500 yuan.

[0097] When the value of the first target object is greater than the predetermined value, it is determined that the first target object in the image to be detected is abnormal. The predetermined value needs to be determined in advance, and the predetermined value can be changed according to demand.

[0098] For example, if the predetermined value is 2,000 yuan and the value of the currency is 2,500 yuan, the value of the currency is greater than the predetermined value, and it is determined that the currency is abnormal.

[0099] In the embodiment of the present disclosure, when the first target object overlaps with the predetermined detection area, and the second target object does not overlap with the predetermined detection area, it is determined whether the first target object in the image to be detected is abnormal based on whether the value of the first target object is greater than the predetermined value. The value of the currency can be determined based on the type and quantity of the currency in the image to be detected when the currency is placed in the working area of ​​the financial institution staff, and the financial institution staff is in the financial institution but not in the working area of ​​the financial institution staff. When the value of the currency is greater than the predetermined value, the abnormal placement of the currency is accurately and efficiently confirmed. Therefore, it is possible to avoid the potential safety hazards caused by the currency in the working area of ​​the financial institution staff being unattended and the currency value being large, which is beneficial to the management of currency by the financial institution. Compared with the related art, it is possible to save time in manually determining the abnormal placement of currency, thereby saving manpower.

[0100] Figure 6 A flowchart of a method for detecting abnormality of an object according to yet another embodiment of the present disclosure is schematically shown.

[0101] like Figure 6 As shown, the method includes operations S210-S220, S610-S620.

[0102] In operation S210 , in response to determining that a first target object exists in the image to be detected, a first overlapping relationship between the first target object and a predetermined detection area is determined.

[0103] This operation and Figure 2 The operation S210 shown is the same and will not be described again.

[0104] In operation S220, based on the first overlapping relationship, it is determined whether the first target object in the image to be detected is abnormal. The image to be detected includes a predetermined detection area.

[0105] This operation and Figure 2 The operation S220 shown is the same and will not be described again.

[0106] In operation S610, the number of images in which a first target object is abnormal among a plurality of images to be detected included in a video to be detected is determined.

[0107] In operation S620, when the number of images is greater than a predetermined ratio, it is determined that the first target object is abnormal.

[0108] The video to be detected may be a video shot by a camera of a financial institution. The video shot by a camera of a financial institution may include multiple images to be detected. In this embodiment, an operation of determining whether the first target object is abnormal may be performed on all images to be detected included in the video. An operation of determining the number of abnormalities of the first target object is performed on multiple images to be detected included in the video. When the number of images determined to be abnormal for the first target object is greater than a predetermined ratio, the first target object is determined to be abnormal.

[0109] Exemplarily, among the multiple images to be detected included in the video captured by the camera of the financial institution, the image to be detected with abnormal currency is determined to be labeled 1, and the image to be detected with normal currency is determined to be labeled 0. Determine the number of images to be detected with label 1 within a time window of 5s in the video captured by the camera of the financial institution. If the proportion of the number of images to be detected with label 1 to the number of all images is greater than a predetermined proportion of 40%, it is determined that the currency is abnormal. Among them, the length of the time window and the value of the predetermined proportion need to be preset and can be changed according to needs.

[0110] In the embodiment of the present disclosure, when the proportion of images with abnormal first target object in the multiple images to be detected included in the video to be detected is greater than a predetermined proportion, the first target object is determined to be abnormal. When the currency in the image to be detected is not placed in the working area of ​​the financial institution staff, it can be confirmed that the placement of the currency in the image to be detected is abnormal. The proportion of the abnormal placement of the currency in the multiple images to be detected included in the video captured by the financial institution’s camera can be counted, and the detection result can be suppressed, avoiding interference with the detection result caused by the financial institution staff not being detected due to temporary actions such as bending down to pick up something. Compared with related technologies, the robustness of the deep target detection model can be increased, and the accuracy of determining currency abnormalities can be improved.

[0111] Figure 7 The flowchart of the method for determining the second overlapping relationship according to the embodiment of the present disclosure is schematically shown.

[0112] like Figure 7 As shown, the method for determining the second overlapping relationship between the second target object and the predetermined detection area may include operations S711 to S712.

[0113] In operation S711, a region where a first target object is located among a plurality of regions is determined as a target region.

[0114] In operation S712, an overlapping relationship between the second target object and the target area is determined.

[0115] There may be multiple predetermined detection areas in the image to be detected. The area where the first target object is located in the multiple areas is determined as the target area. The second overlapping relationship between the second target object and the predetermined detection area is determined, that is, the overlapping relationship between the second target object and the target area is determined.

[0116] When determining whether the first target object in the image to be detected is abnormal, all predetermined detection areas in the image to be detected are detected. When the first target object overlaps with the predetermined detection area, as long as there is a predetermined detection area overlapping with the first target object, it is determined that the first target object in the image to be detected is abnormal.

[0117] According to an embodiment of the present disclosure, the area where the first target object is located in the multiple areas can be used as the target area. The second overlapping relationship between the second target object and the predetermined detection area is the overlapping relationship between the second target object and the target area. When money is placed in the working area of ​​a financial institution staff, the working area can be determined as the target area, and the overlapping relationship between the financial institution staff and the target area can be determined.

[0118] Figure 8 A block diagram of a device for detecting abnormality of an object according to an embodiment of the present disclosure is schematically shown.

[0119] like Figure 8 As shown, the device 800 for detecting object abnormality may include: a relationship determination module 810 and a first abnormality determination module 820. The device for detecting object abnormality may execute the method described in the above reference method embodiment, which will not be described in detail here.

[0120] Specifically, the relationship determination module 810 may be configured to determine a first overlapping relationship between the first target object and a predetermined detection area in response to determining that the first target object exists in the image to be detected.

[0121] The first abnormality determination module 820 may be used to determine whether a first target object in the image to be detected is abnormal based on the first overlapping relationship, wherein the image to be detected includes a predetermined detection area.

[0122] The first abnormality determination module may be configured to determine that the first target object in the image to be detected is abnormal when the first overlapping relationship indicates that the first target object does not overlap with the predetermined detection area.

[0123] The first abnormality determination module may include: an object determination submodule and a determination submodule.

[0124] Specifically, the object determination submodule may be used to determine whether there is a second target object in the image to be detected when the first overlapping relationship indicates that the first target object overlaps with the predetermined detection area.

[0125] The discrimination submodule may be used to determine whether the first target object in the image to be detected is abnormal when the second target object does not exist.

[0126] The first abnormality determination module may further include: a relationship determination submodule.

[0127] Specifically, the relationship determination submodule may be used to determine a second overlapping relationship between the second target object and the predetermined detection area when there is a second target object.

[0128] The discrimination submodule may be used to determine whether the first target object in the image to be detected is abnormal based on the second overlapping relationship.

[0129] The determination submodule may include: an object information determination unit, a value determination unit, and a determination unit.

[0130] Specifically, the object information determining unit may be configured to determine the category and quantity of the first target object when the second overlapping relationship indicates that the second target object does not overlap with the predetermined detection area.

[0131] The value determination unit may be configured to determine the value of the first target object based on the category and quantity of the first target object.

[0132] The discrimination unit may be used to determine that the first target object in the image to be detected is abnormal when the value is greater than a predetermined value.

[0133] Fig. 9 A block diagram of an apparatus for detecting abnormality of an object according to another embodiment of the present disclosure is schematically shown.

[0134] like Fig. 9 As shown, the device 900 for detecting object abnormality may include: a relationship determination module 910 , a first abnormality discrimination module 920 , an image quantity determination module 930 , and a second abnormality discrimination module 940 .

[0135] Specifically, the relationship determination module 910 may be configured to determine a first overlapping relationship between the first target object and a predetermined detection area in response to determining that the first target object exists in the image to be detected.

[0136] The first abnormality determination module 920 may be configured to determine whether a first target object in the image to be detected is abnormal based on the first overlapping relationship, wherein the image to be detected includes a predetermined detection area.

[0137] The image quantity determination module 930 may be used to determine the number of images in which the first target object is abnormal among a plurality of images to be detected included in the video to be detected.

[0138] The second abnormality determination module 940 may be used to determine that the first target object is abnormal when the number of images is greater than a predetermined ratio.

[0139] The relationship determination submodule may include: a target area determination unit and an overlap determination unit.

[0140] Specifically, the target area determination unit may be configured to determine an area among multiple areas where the first target object is located as the target area.

[0141] The overlap determination unit may be used to determine the overlap relationship between the second target object and the target area.

[0142] It should be noted that the embodiments of the device part of the present disclosure are the same or similar to the embodiments of the method part of the present disclosure, and the present disclosure will not repeat them here.

[0143] According to the embodiments of the present invention, any one or more of the modules, submodules, and units, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, and units can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, and units can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, and units can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0144] For example, any multiple of the relationship determination module and the first abnormal discrimination module can be combined in one module / unit / subunit for implementation, or any one of the modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functions of one or more modules / units / subunits in these modules / units / subunits can be combined with at least part of the functions of other modules / units / subunits and implemented in one module / unit / subunit. According to an embodiment of the present disclosure, at least one of the relationship determination module and the first abnormal discrimination module can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the relationship determination module and the first abnormal discrimination module can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0145] Fig.10 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Fig.10 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0146] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0147] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0148] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The system 1000 may further include one or more of the following components connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0149] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0150] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0151] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0152] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 1002 and / or the RAM 1003 described above and / or one or more memories other than the ROM 1002 and the RAM 1003 .

[0153] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0154] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0155] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for detecting anomalies of an object, comprising: In response to determining that a first target object exists in the image to be detected, determining a first overlapping relationship between the first target object and a predetermined detection area; as well as Based on the first overlapping relationship, determining whether the first target object in the image to be detected is abnormal, Wherein, the image to be detected includes the predetermined detection area; The determining whether the first target object in the image to be detected is abnormal based on the first overlapping relationship includes: In a case where the first overlapping relationship indicates that the first target object overlaps with the predetermined detection area, determining whether a second target object exists in the image to be detected; In the absence of the second target object, determining that the first target object in the image to be detected is abnormal; The determining whether the first target object in the image to be detected is abnormal based on the first overlapping relationship further includes: In the case where the second target object exists, determining a second overlapping relationship between the second target object and the predetermined detection area; and Based on the second overlapping relationship, determining whether the first target object in the image to be detected is abnormal; The determining whether the first target object in the image to be detected is abnormal based on the second overlapping relationship includes: When the second overlapping relationship indicates that the second target object does not overlap with the predetermined detection area, determining the category and quantity of the first target object; determining the value of the first target object based on the category and quantity of the first target object; and When the value is greater than a predetermined value, it is determined that the first target object in the image to be detected is abnormal.

2. The method according to claim 1, wherein: The determining whether the first target object in the image to be detected is abnormal based on the first overlapping relationship further includes: When the first overlapping relationship indicates that the first target object does not overlap with the predetermined detection area, it is determined that the first target object in the image to be detected is abnormal.

3. The method according to claim 1, further comprising: Determine the number of images in which the first target object is abnormal among a plurality of images to be detected included in the video to be detected; as well as When the number of images is greater than a predetermined ratio, it is determined that the first target object is abnormal.

4. The method according to claim 1, wherein: The predetermined detection area includes a plurality of areas; Determining a second overlapping relationship between the second target object and the predetermined detection area includes: Determine an area among the multiple areas where the first target object is located as a target area; as well as An overlapping relationship between the second target object and the target area is determined.

5. A device for detecting abnormality of an object, comprising: a relationship determination module, configured to determine a first overlapping relationship between the first target object and a predetermined detection area in response to determining that the first target object exists in the image to be detected; as well as a first abnormality determination module, configured to determine whether the first target object in the image to be detected is abnormal based on the first overlapping relationship; Wherein, the image to be detected includes the predetermined detection area; The determining whether the first target object in the image to be detected is abnormal based on the first overlapping relationship includes: In a case where the first overlapping relationship indicates that the first target object overlaps with the predetermined detection area, determining whether a second target object exists in the image to be detected; In the absence of the second target object, determining that the first target object in the image to be detected is abnormal; The determining whether the first target object in the image to be detected is abnormal based on the first overlapping relationship further includes: In the case where the second target object exists, determining a second overlapping relationship between the second target object and the predetermined detection area; and Based on the second overlapping relationship, determining whether the first target object in the image to be detected is abnormal; The determining whether the first target object in the image to be detected is abnormal based on the second overlapping relationship includes: When the second overlapping relationship indicates that the second target object does not overlap with the predetermined detection area, determining the category and quantity of the first target object; determining the value of the first target object based on the category and quantity of the first target object; and When the value is greater than a predetermined value, it is determined that the first target object in the image to be detected is abnormal.

6. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having executable instructions stored thereon, wherein when the instructions are executed by a processor, the processor implements the method according to any one of claims 1 to 4.

Images