Method for object detection using cropped image
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- AXIS
- Filing Date
- 2023-06-14
- Publication Date
- 2026-04-27
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates generally to the field of camera surveillance, and in particular to a method and control unit for object detection in a video stream captured by means of an image collection device, for example for the purpose of anonymizing objects in the video stream. [Background technology]
[0002] In various camera surveillance applications, it may be necessary to mask objects in the video stream captured by the camera. An important reason for object masking is to ensure privacy about people present in the video stream and to protect other types of personal information that may be captured in the video stream.
[0003] Masking of people may be performed by extracting the image coordinates of the relevant portion of the image frame. Once the image coordinates are known, the relevant portion in the video stream can be masked, pixelated, blurred, or otherwise altered to obscure the identifiers in the video stream. However, before masking can be processed, the object to be masked must be detected in the video stream. Object detection methods may be applied to detect people, for example.
[0004] Object detection requires algorithms, such as those implemented by neural networks, to detect and classify objects in a video stream. To allow the neural network to operate fast enough, it may have to operate at a lower resolution than the camera. This requires downscaling of the images, resulting in a loss of data. Furthermore, the reduction in resolution may not allow the neural network to accurately detect objects that should be masked, which means that an operator may be able to see the object by digitally zooming in on the image.
[0005] Therefore, there is room for improvement regarding object detection, especially for anonymizing objects in video streams. Summary of the Invention
[0006] In view of the above and other drawbacks of the prior art, it is an object of the present invention to provide an improved method for object detection in an image that mitigates at least some of the drawbacks of the prior art.
[0007] According to a first aspect of the present invention, there is therefore provided a method for object detection in an image, the method comprising the steps of: obtaining a first resolution image collected by an image collection device, scaling the first resolution image to a second resolution image having a resolution lower than that of the first resolution image, analysing the second resolution image to determine a first probability that a detected object of a predetermined type is present in an area of the first resolution image, cropping an area in the first resolution image containing the detected object if the first probability is below a first threshold and above or equal to a second threshold, where a probability above or equal to the first threshold indicates that the detected object is of the predetermined type and a probability below the second threshold indicates that the detected object is not of the predetermined type, analysing the cropped image to determine a second probability that the detected object is of the predetermined type, and providing an indication that the detected object is of the predetermined type if the second probability is above or equal to the first threshold.
[0008] The present invention is based on the recognition that image cropping is utilized to improve the accuracy of object detection. More specifically, the inventors have realized that if the first probability is such that it is not possible to know whether an area in the downscaled or scaled image contains a predetermined type of object, the area in the original image collected by the image collection device can be cropped and the cropped image can be analyzed or, more specifically, passed through a neural network again for object detection. This results in a higher accuracy of the detected object.
[0009] A video stream is generally a set of consecutive image frames captured over time, which collectively form the video stream.
[0010] The present invention can be applied to various applications where object detection is used. One such application is for privacy masking, understood as a function for protecting personal privacy by hiding or concealing parts of an image frame with a masked area. In general, privacy masks can be static or dynamic, but the privacy masks most often described herein are dynamic masks. Privacy masks include, for example, edge filters, solid masks, or blurs. Static masks can be applied globally in the same way across the entire or at least part of an image frame, while dynamic masks can be applied, for example, when a face or person is detected in a video stream.
[0011] Scaling an image may be considered as downscaling with the goal of reducing the image resolution to a resolution suitable for the analysis step to detect objects, for example, a first image may be downscaled to a second resolution suitable for an algorithm or neural network used for object detection.
[0012] According to one embodiment, the steps of analyzing the second resolution image and analyzing the cropped image may be performed in a neural network. The neural network is generally capable of operating at a lower resolution than the sensor. The scaling step is to adapt the image resolution to the neural network operating image resolution. In other words, the neural network may be adapted to the image resolution and image size, expecting the input image to have that resolution and size. This is required to enable the operation of the neural network in real time for object detection.
[0013] According to one embodiment, the step of analysing the second resolution image may be performed in a first neural network and the step of analysing the cropped image may be performed in a second neural network. The use of two or more neural networks advantageously allows the object detection method to be performed faster.
[0014] According to one embodiment, the method may include the steps of (a) analyzing the cropped image to determine a further probability that a detected object of a predetermined type is present in a region of the cropped image; (b) cropping the region of the detected object in the cropped image to form a further cropped image if the further probability is below a first threshold and above or equal to a second threshold; (c) analyzing the further cropped image to determine a further probability that the detected object is of the predetermined type; and (d) providing an indication that the detected object is of the predetermined type if the further probability is above or equal to the first threshold, steps (a)-(d) being performed recursively until a predetermined condition is met. In this way, if the analysis still cannot confidently conclude the type of the object, a further image cropping is performed on a smaller region. The recursive analysis and cropping further improves the object detection accuracy.
[0015] The recursively performed method may continue until at least one of several conditions is met: the further probability in step (b) is below a second threshold or exceeds or is equal to the first threshold, for a predetermined number of iterations, and the resolution of the further cropped image is below a predetermined resolution. The predetermined resolution may be, for example, the resolution to which the neural network used to analyze the cropped image is adapted. Furthermore, if the further probability in step (b) is below a second threshold or exceeds or is equal to the first threshold, there is no "detection uncertainty" and therefore no need to continue further recursive iterations.
[0016] According to one embodiment, the step of analysing the first resolution image may include determining a probability that a detected object of a given type is present in two or more regions, each of which is cropped and analysed in the first resolution image. In this way, it is envisaged to advantageously crop two or more regions in order to detect two or more objects.
[0017] According to one embodiment, the method may include the steps of analyzing the set of second resolution images, detecting motion in the set of second resolution images, cropping a further region in a subsequent image frame if the detected motion exceeds or is equal to a motion threshold, analyzing the cropped image, and giving an indication that the detected object is of a predefined type if the second probability exceeds or is equal to the first threshold. Advantageously, a new crop is performed only on the region where motion is detected with a magnitude exceeding or equal to the motion threshold, thereby reducing the time required to take a new crop. In other words, a new crop is taken only where motion is detected. The further region preferably includes the region of the detected object that is moving, detected in the set of second resolution images. To state one possible implementation, the detection and quantifying the motion may be performed by monitoring pixel color changes in the image.
[0018] Furthermore, if the detected motion is below a motion threshold in the first image frame, the results of the analyzing step in the first frame can be reused on the same cropped region in the subsequent image frame. In this way, instead of performing a new crop and re-analyzing regions where a new crop may not be necessary, i.e. regions where no motion is detected, the previous analysis results are reused.
[0019] According to one embodiment, if the first probability exceeds or is equal to the first threshold, an indication is provided that the detected object is of a predetermined type. The indication may further include an indication that object masking is required.
[0020] Preferably, the resolution of the first resolution image is the resolution at which it was captured by the image capture device.
[0021] The resolution of the second resolution image advantageously depends on the size of the neural network used to analyze the second resolution image.
[0022] Preferably, the method is performed at a rate that corresponds substantially to a frame rate of a captured video stream that includes the first resolution images.
[0023] According to a second aspect of the present invention, there is provided a control unit for object detection in an image, the control unit being configured to: obtain a first resolution image collected by an image collection device; scale the first resolution image to a second resolution image having a lower resolution than that of the first resolution image; analyze the second resolution image to determine a first probability that a detected object of a predetermined type is present in a region of the first resolution image; crop a region in the first resolution image containing the detected object if the first probability is below a first threshold and above or equal to a second threshold, wherein a probability above or equal to the first threshold indicates that the detected object is of the predetermined type and a probability below the second threshold indicates that the detected object is not of the predetermined type; analyze the cropped image to determine a second probability that the detected object is of the predetermined type; and provide an indication that the detected object is of the predetermined type if the second probability is above or equal to the first threshold.
[0024] Further embodiments of, and advantages achieved by, this second aspect of the invention are largely similar to the embodiments and advantages described above for the first aspect of the invention.
[0025] According to a third aspect of the invention there is provided a system comprising a control unit according to the second aspect and an image acquisition device for capturing an image of a scene including an object.
[0026] The image acquisition device is preferably a video camera, such as a surveillance camera.
[0027] Further embodiments of, and advantages achieved by, this third aspect of the invention are largely similar to the embodiments and advantages described above for the first and second aspects of the invention.
[0028] According to a fourth aspect of the present invention, there is provided a computer program for object detection in an image, the computer program comprising computer code which, when executed on a processing circuit of a controller, causes a control unit to: acquire a first resolution image collected by an image collection device; scale the first resolution image to a second resolution image having a lower resolution than that of the first resolution image; analyze the second resolution image to determine a first probability that a detected object of a predetermined type is present in a region of the first resolution image; crop a region in the first resolution image containing the detected object if the first probability is below a first threshold and above or equal to a second threshold, where a probability above or equal to the first threshold indicates that the detected object is of the predetermined type and a probability below the second threshold indicates that the detected object is not of the predetermined type; analyze the cropped image to determine a second probability that the detected object is of the predetermined type; and provide an indication that the detected object is of the predetermined type if the second probability is above or equal to the first threshold.
[0029] Further embodiments of, and advantages achieved by, this fourth aspect of the invention are largely similar to the embodiments and advantages described above for the other aspects of the invention.
[0030] Further features and advantages of the present invention will become apparent upon study of the appended claims and the following description. Those skilled in the art will recognize that different features of the present invention can be combined to create embodiments other than those described below without departing from the scope of the present invention.
[0031] Various aspects of the present invention, including particular features and advantages of the present invention, will be readily understood from the following detailed description and the accompanying drawings. [Brief description of the drawings]
[0032] [Figure 1] FIG. 1 conceptually illustrates an exemplary application of an embodiment of the present invention. [Diagram 2] 2 is a flowchart of method steps according to an embodiment of the present invention. [Diagram 3] FIG. 2 conceptually illustrates an image cropping process for object detection according to an embodiment of the present invention. [Figure 4A] FIG. 2 conceptually illustrates a control unit for operating a neural network, according to an embodiment of the present invention. [Figure 4B] FIG. 2 conceptually illustrates a control unit operating two neural networks according to an embodiment of the present invention. [Diagram 5] FIG. 2 conceptually illustrates an image cropping process for object detection according to an embodiment of the present invention. [Figure 6] 2 is a flowchart of method steps according to an embodiment of the present invention. [Figure 7] FIG. 2 conceptually illustrates image cropping in the presence of moving objects, according to an embodiment of the present invention. [Figure 8] 2 is a flowchart of method steps according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0033] The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which presently preferred embodiments of the invention are shown. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather, these embodiments are provided for thoroughness and completeness so as to fully convey the scope of the invention to those skilled in the art. Like reference characters refer to like elements throughout.
[0034] Turning now to the drawings, and in particular to Figure 1, there is shown a scene 1 being monitored by an image collection device 100, e.g. a camera, or more particularly a surveillance camera. Within scene 1 there is a vehicle 104d, for example in a parking lot, and a set of objects 104a-d, which may be several people 104a-c.
[0035] The camera 100 may be mounted on a building, on a pole, or in any other suitable location depending on the particular application at hand. Moreover, the camera 100 may be a fixed camera, or a movable camera, such as pan, tilt, and zoom, or even a body-worn camera. Moreover, the camera 100 may be a visible light camera, an infrared (IR) sensitive camera, or a thermal (long-wave infrared (LWIR)) camera. Additionally, image collection devices employing lidar and radar capabilities may also be contemplated.
[0036] Camera 100 continuously monitors scene 1 by capturing image frames that form a video stream of scene 1. Scene 1 within the field of view of camera 100 is illustrated here as including vehicle 104d and people 104a-c. Camera 100 may transmit the video stream to client 116 or server 118 via wired or wireless communication link 112 connected to a communication network 114, e.g., the cloud.
[0037] The camera 100 comprises an image capture module 2 and a control unit 3 comprising one or more processors capable of operating an image processing pipeline and an encoder. The camera 100 further comprises an input / output interface 210 configured as a communication interface between the camera 100 and a network 114 via a wireless link 112.
[0038] During monitoring of a scene 1 with the camera 100, the control unit 3 is operable to detect objects in the scene and classify them as being of a certain type. Detecting objects in the scene 1 is important as it can be used for masking of objects and people to protect their privacy. To improve the ability of the camera and its associated control unit 3 to detect objects of a certain type, e.g. objects requiring masking, the following method is provided.
[0039] FIG. 2 is a flow chart of method steps according to an embodiment of the invention, and FIG. 3 conceptually illustrates the method steps.
[0040] In step S102, a first resolution image 110 is collected by the image collection device 100. The first resolution image 110 may be a frame of a video stream, here including the vehicle 104d and the people 104a-c. The resolution of the first image is the resolution at which it was captured by the image collection device 100.
[0041] In step S104, the control unit 3 scales the first resolution image 110 to a second resolution image 112 having a resolution lower than that of the first resolution image 110. Scaling the first resolution image 110 to the second resolution image 112 may be performed by downscaling the first resolution image 110 to the second resolution image 112. The resolution of the second resolution image 112 may be scaled down by a factor selected such that the total number of pixels is within the range of 4 to 25, e.g., 4, 9, 16, etc.
[0042] In step S106, the second resolution image is analysed by the control unit 3 in order to determine a first probability that a detected object of a given type is present in an area 111 of the first resolution image. In this way, an analysis is performed on the second resolution image 112 in order to establish areas 111' in the second resolution image 112 in which the detected object may be present. Corresponding areas 111 are determined or found in the original first resolution image 110.
[0043] The analysis is preferably performed in a neural network of a predetermined size and resolution. The size and / or resolution of the neural network corresponds to the size and resolution of the image on which the neural network was trained. Furthermore, the resolution of the second image 112 depends on the size and / or resolution of the neural network used to analyze the second resolution image. For example, the second resolution should be kept higher than the resolution of the neural network. Since the size of the first resolution image is substantially larger than the size of the neural network, the neural network may have difficulty analyzing the image fast enough. To alleviate this, in accordance with the present disclosure, the first resolution image is downscaled to a smaller resolution, which allows the neural network to operate at a high speed to perform the analysis. In this way, by performing the downscaling and subsequent cropping described above, the overall operation of the analysis can also be performed more efficiently.
[0044] If the first probability is below the first threshold and above or equal to the second threshold, in step S108, a region 111 including the detected object 104b-c in the first resolution image is cropped. The first probability and the second probability are adapted such that a probability above or equal to the first threshold indicates that the detected object is of a predetermined type and a probability below the second threshold indicates that the detected object is not of a predetermined type. For example, the predetermined type of objects may include objects that should be masked, such as people 104a-c and license plates of vehicles 104d.
[0045] In some cases, the cropped image 111 is downscaled similarly to the downscaling in step S104, for example when a recursive process is applied as described further below, however, in general it is not strictly necessary that the cropped image is downscaled.
[0046] If the first probability exceeds or is equal to the first threshold, then in step S109 an indication that the detected object is of a predetermined type is given by the control unit 3. When applying the method to masking, a subsequent step may be to perform privacy masking of the detected objects 104c-b in the image 110 if the detected objects are objects that should be masked, such as people or vehicle license plates.
[0047] In a subsequent step S110, the cropped image 111 is analyzed by the control unit 3 to determine a second probability that the detected object 104b is of a predetermined type. If the second probability exceeds or is equal to the first threshold, then in step S112 the control unit 3 provides an indication that the detected object 104b is of a predetermined type.
[0048] As shown diagrammatically in Fig. 4A, a control unit 3 receiving or collecting a video stream 4 from an image capture module 2 scales a first resolution image 110 to be analysed and provides a scaled second resolution image 112 to a neural network 302. The steps of analysing the second resolution 112 image and of analysing the cropped image 111 are performed in the neural network 302. The control unit 3 provides an indication in the form of a data signal 304 that a detected object 104b is of a predetermined type.
[0049] 4B, which conceptually illustrates an alternative to the embodiment of FIG. 4A, the steps of analyzing the second resolution image 112 are performed in a first neural network 302a, and the steps of analyzing the cropped image 111 are performed in a second neural network 302b. In this way, because processing power is split between the two networks 302a and 302b, the second resolution image can be of higher resolution than if only a single neural network were used.
[0050] 5, it may occur that objects 104c and 104b of a given type are detected with a certain probability in two or more regions 111a and 111b of the first resolution image 110 based on an analysis of the second resolution image 112 in which corresponding regions 111a' and 111b' have been detected. In such a case, two or more respective regions 111a and 111b are cropped in the first resolution image 110 and analyzed in a neural network or in two or more neural networks. However, where possible, one region containing both objects 104c and 104b may be cropped and processed in a recursive manner, as will be further explained below.
[0051] According to some embodiments, the method is performed recursively, as illustrated by the steps of the flowchart of FIG. 6 described in conjunction with FIG. 3. The cropping in step S108 is preferably performed so that the cropped image includes all or as many of the detected objects as possible. In this example, the cropped image 111 includes the detected objects 104b and 104c. If the image size of the cropped image 111 is too large for the neural network, the cropped image 111 is downscaled to a reduced-resolution cropped image before analysis. In this way, given the cropped image 111, the cropped image 111 may first be downscaled as described in step S104. Then, in step S602, the cropped image 111 or the reduced-resolution cropped image is analyzed to determine a second or further probability that a detected object 104c of a given type is present in the region 115′ of the cropped image 111. In this way, the cropped image 111, which may here be a scaled cropped image 111, is analyzed in the neural network, similar to the second resolution image 112 in step S106 of Figure 2. In this way, an analysis is performed on the cropped image 111 in order to establish an area 115' in the cropped image 111 in which the detected object 104c may be present.
[0052] If the further probability is below the first threshold and above or equal to the second threshold, then in step S604, a region 115' of the detected object 104c in the cropped image 111 is cropped to form a further cropped image 115.
[0053] If the predetermined condition is not met, the process recursively analyzes the further cropped image again to determine yet another probability that the detected object is of the predetermined type in step S602. If the probability exceeds or is equal to the first threshold, an indication is provided in step S109 that the detected object 104c is of the predetermined type. In such a case, the detected object 104c may be masked in the first resolution image 110.
[0054] The object 104 b may have been determined to be of a certain type in the cropped image, such that the object 104 b is not included in the cropped image 115 .
[0055] The steps of the flowchart of FIG. 6 are executed recursively until at least one of the following predetermined conditions is met: the further probability is below a second threshold or exceeds or equals the first threshold; the number of iterations exceeds or equals a predetermined number; and the resolution of the further cropped image is below a predetermined resolution.
[0056] For example, if the further probability exceeds or is equal to a first threshold, the neural network concludes that the object 104c has been detected with a sufficiently high probability that no further analysis is necessary, and provides an indication thereof in step S109. Similarly, if the further probability is below a second threshold, the neural network concludes that the object is not of a predetermined type, and therefore no further analysis is necessary. In this case, it may be concluded, for example, that no masking is necessary.
[0057] Further, the iterations may continue until the resolution of the further cropped image falls below a predetermined resolution set by the resolution of the neural network.
[0058] If the predetermined condition is met, the method ends in step S610.
[0059] FIG. 7 conceptually illustrates two first resolution image frames 110a and 110b and corresponding scaled second resolution images 112a and 112b, and FIG. 8 is a flowchart of method steps according to an embodiment of the invention.
[0060] As described with respect to FIG. 2, in step S106, the second resolution image 112a and the second resolution image 112b are analyzed.
[0061] Thereafter, step S802 includes detecting motion in the second resolution images 112a-b. This can be done by comparing frame 112a of the second resolution image with frame 112b of the subsequent second resolution image and detecting differences between those frames that may indicate object motion. For example, a pixel-by-pixel comparison can be performed to compare color changes between frames 112a and 112b that may indicate motion. Here, the vehicle 104d is moving, e.g., moving between frames 110a / 112a and 110b / 112b, but the person 104a is not moving.
[0062] If the detected motion exceeds or is equal to the motion threshold, in step S804, the same region in the subsequent image frame is cropped. In this example, since motion was detected in the cropped region 111d' in the frame 112a of the second resolution image, a further region 111e including the moving object 104d is cropped in the second image frame 110b as in the first image frame 110a. The further region 111e corresponds to the region 111e' of the moving object 104d in the second resolution image 112b of the second image frame 110b. Therefore, since a relatively strong degree of motion was detected, it is important to repeat the step S110 of successively cropping the further region 111e and analyzing the cropped region 111e.
[0063] However, if the detected motion falls below the motion threshold in the first image frame 112a, no repeat of cropping is necessary. Instead, the results of the analyzing step in the first frame 112a on the same cropped region are reused in the subsequent image frame 112b. This saves computational power, since the cropping step S108 and the analyzing step S110 do not need to be repeated unless a change in the corresponding region of the scene 1 is detected. For example, no motion is detected in the region 111c' between the frame 112a of the second resolution image and the frame 112b of the second resolution image. Thus, the analysis for detecting objects performed in a previous step on the corresponding region 111c in the image frame 110 is reused, concluding, for example, that the person 104a is still present in the image frame 110b.
[0064] Preferably, the method is performed at a rate that substantially corresponds to the frame rate of the captured video stream that includes the first resolution images, and the neural network, or networks, are configured such that there is object detection on each frame with a delay of at most one frame.
[0065] The methods described herein are computer-implemented methods.
[0066] The control unit includes processing circuitry configured to cause the control unit to perform the steps of the methods described herein.
[0067] Further provided is a computer program product including a computer readable storage medium storing the computer program. The computer readable storage medium may be, for example, non-transitory and may be provided, for example, as a hard disk drive (HDD), a solid state drive (SDD), a USB flash drive, an SD card, a CD / DVD, and / or any other storage medium capable of non-transitory storage of data.
[0068] The computer program includes computer code, which when executed on the processing circuitry of the controller causes the control unit to acquire a first resolution image collected by the image collection device, scale the first resolution image to a second resolution image having a resolution lower than that of the first resolution image, analyze the second resolution image to determine a first probability that a detected object of a predetermined type is present in a region of the first resolution image, crop a region in the first resolution image containing the detected object if the first probability is below a first threshold and above or equal to a second threshold, where a probability above or equal to the first threshold indicates that the detected object is of the predetermined type and a probability below the second threshold indicates that the detected object is not of the predetermined type, cropping the region, analyze the cropped image to determine a second probability that the detected object is of the predetermined type, and provide an indication that the detected object is of the predetermined type if the second probability is above or equal to the first threshold.
[0069] The control unit includes a microprocessor, a microcontroller unit, a programmable digital signal processor, or another programmable device. The control unit may also, or instead, include an application specific integrated circuit, a programmable gate array or programmable array logic, a programmable logic device, or a digital signal processor. If the control unit includes a programmable device, such as a microprocessor, a microcontroller unit, or a programmable digital signal processor as described above, the processor may further include computer executable code for controlling the operation of the programmable device.
[0070] The control functions of the present disclosure may be implemented using existing computer processors, or by dedicated computer processors for suitable systems incorporated for this or another purpose, or by hardwired systems. An embodiment within the scope of the present disclosure includes a program product comprising a machine-readable medium for carrying machine-executable instructions or data structures or having machine-executable instructions or data structures stored thereon. Such machine-readable media may be any available medium that can be accessed by a general purpose or special purpose computer, or other machine having a processor. By way of example, such machine-readable media may comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of machine-executable instructions or data structures and that can be accessed by a general purpose or special purpose computer, or other machine having a processor. When information is transferred or provided to a machine over a network or another communications connection (either wired, wireless, or a combination of wired or wireless), the machine properly views the connection as a machine-readable medium. As such, any such connection is properly referred to as a machine-readable medium. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions comprise, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions.
[0071] Although the diagrams may show a sequence, the order of steps may differ from that shown. Also, two or more steps may be performed simultaneously or with partial concurrence. Such variations depend on the software and hardware systems chosen and on the designer's choice. All such variations are within the scope of this disclosure. Similarly, software implementations may be achieved using standard programming techniques with rule-based logic and other logic to accomplish the various connection, processing, comparison, and decision steps. Furthermore, while the present invention has been described with reference to specific exemplary embodiments thereof, many different changes, modifications, and the like will become apparent to those skilled in the art.
[0072] In addition, variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. Moreover, in the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. [Explanation of symbols]
[0073] 2 Image capture module 3. Control Unit 4. Video Stream 10. System 100 Image Acquisition Device 104a Object 104b Object 104c Object 104d object 110 first resolution image 110a First resolution image frame 110b First resolution image frame 111 areas 111 Cropped Images 111' area 111a area 111a' area 111b area 111b' area 111c area 111c' area 111d area 111d' area 111e area 111e' area 112 Communication Links 112 second resolution image 112a Frame of second resolution image 112b Second resolution image frame 114 Communication Networks 115 Cropped Images 115' area 116 clients 118 Server 210 Input / Output Interface 302 Neural Networks 302a The first neural network 302b Second Neural Network 304 Data Signal
Claims
1. A computer-based method for detecting an object in an image collected by a camera of a surveillance system, The steps include acquiring a first resolution image collected by the aforementioned camera, The steps include scaling the first resolution image to a second resolution image having a lower resolution than the first resolution image, A step of analyzing a second resolution image in order to determine a first probability that a detected object of a predetermined type is present in the region of the first resolution image, wherein the second resolution image is analyzed such that the predetermined type is an object to be masked. If the first probability is below a first threshold and above or equal to a second threshold, the step of cropping the region in the first resolution image that includes the detected object, wherein the probability above or equal to the first threshold indicates that the detected object is of a predetermined type, and the probability below the second threshold indicates that the detected object is not of a predetermined type; The steps include analyzing the cropped image to determine a second probability that the detected object is of the predetermined type, If the second probability exceeds or is equal to the first threshold, the step of indicating that the detected object is of the predetermined type that should be masked, If the second probability is below the first threshold and exceeds or equals the second threshold, (a) The step of analyzing the cropped image to determine the further probability that a detected object of the predetermined type exists within the region of the cropped image, (b) If the further probability is below the first threshold and above or equal to the second threshold, the step of cropping the region of the detected object in the cropped image in order to form a further cropped image, (c) The step of analyzing the further cropped image in order to determine another probability that the detected object is of the predetermined type, (d) If the further probability exceeds or is equal to the first threshold, the step of indicating that the detected object is of the predetermined type: Includes, Steps (a) to (d) described above are performed recursively until a predetermined condition is met. method.
2. The method according to claim 1, wherein the step of analyzing the second resolution image and the step of analyzing the cropped image are performed in a neural network.
3. The method according to claim 1, wherein the step of analyzing the second resolution image is performed in a first neural network, and the step of analyzing the cropped image is performed in a second neural network.
4. Steps (a) to (d) described above are subject to the following conditions, namely, The further probability in step (b) is below the second threshold, or above or equal to the first threshold, A predetermined number of repetitions, and If the resolution of the further cropped image falls below a predetermined resolution The method according to claim 1, which is performed recursively until at least one of the following conditions is met.
5. The method according to any one of claims 1 to 4, wherein the step of analyzing the first resolution image comprises determining the probability that the detected object of a predetermined type is present in two or more regions, and each of the two or more regions is cropped and analyzed in the first resolution image.
6. The steps include analyzing a second set of resolution images, The steps include detecting motion in the set of the second resolution images, If the detected motion exceeds or is equal to a motion threshold, the steps include cropping a further region in the subsequent image frame and analyzing the cropped image. If the second probability exceeds or is equal to the first threshold, the step of indicating that the detected object is of the predetermined type: The method according to any one of claims 1 to 4, including the method described in any one of claims 1 to 4.
7. The method according to claim 6, wherein if the detected motion falls below the motion threshold in the first image frame, the results of the analysis step are reused in the same cropped region in a subsequent image frame to determine the second probability in the first frame.
8. The method according to any one of claims 1 to 4, wherein if the first probability exceeds or is equal to the first threshold, an indication is given that the detected object is of the predetermined type.
9. The method according to any one of claims 1 to 4, wherein the resolution of the first resolution image is the resolution at the time it was captured by the image acquisition device.
10. The method according to any one of claims 1 to 4, wherein the resolution of the second resolution image depends on the size of the neural network used to analyze the second resolution image.
11. The method according to any one of claims 1 to 4, wherein the method is performed at a rate substantially corresponding to the frame rate of the captured video stream including the first resolution image.
12. A processing circuit comprising one or more processors and a memory storing instructions for object detection in an image collected by a camera, wherein the processing circuit To acquire a first resolution image collected by the aforementioned camera, Scaling the first resolution image to a second resolution image having a lower resolution than the first resolution image, Analyzing a second resolution image to determine a first probability that a detected object of a predetermined type is present in the region of the first resolution image, wherein the second resolution image is analyzed in such a way that the predetermined type is an object to be masked. If the first probability is below a first threshold and above or equal to a second threshold, crop the region in the first resolution image that includes the detected object, wherein the probability above or equal to the first threshold indicates that the detected object is of a predetermined type, and the probability below the second threshold indicates that the detected object is not of a predetermined type. To determine a second probability that the detected object is of the predetermined type, the cropped image is analyzed, If the second probability exceeds or is equal to the first threshold, the detected object is indicated to be of the predetermined type that should be masked. If the second probability is below the first threshold and exceeds or equals the second threshold, (a) Analyzing the cropped image to determine the further probability that the detected object of the predetermined type is present in the region of the cropped image, (b) If the further probability is below the first threshold and above or equal to the second threshold, crop the region of the detected object in the cropped image in order to form a further cropped image. (c) Analyzing the further cropped image in order to determine another probability that the detected object is of the predetermined type, (d) If the further probability exceeds or is equal to the first threshold, the detected object is to be indicated as being of the predetermined type. It is configured to do the following: Steps (a) to (d) are executed recursively until a predetermined condition is met. Processing circuit.
13. A system comprising a camera for capturing an image of a scene including an object, and the processing circuit described in claim 12.
14. A non-temporary computer-readable storage medium storing a computer program for object detection in an image, wherein the computer program, when executed on a processing circuit comprising one or more processors, is configured to be used by the processing circuit. To acquire a first resolution image collected by the camera, Scaling the first resolution image to a second resolution image having a lower resolution than the first resolution image, Analyzing a second resolution image to determine a first probability that a detected object of a predetermined type is present in the region of the first resolution image, wherein the second resolution image is analyzed in such a way that the predetermined type is an object to be masked. If the first probability is below a first threshold and above or equal to a second threshold, crop the region in the first resolution image that includes the detected object, wherein the probability above or equal to the first threshold indicates that the detected object is of a predetermined type, and the probability below the second threshold indicates that the detected object is not of a predetermined type. To determine a second probability that the detected object is of the predetermined type, the cropped image is analyzed, If the second probability exceeds or is equal to the first threshold, the detected object is indicated to be of the predetermined type that should be masked. If the second probability is below the first threshold and exceeds or equals the second threshold, (a) Analyzing the cropped image to determine the further probability that the detected object of the predetermined type is present in the region of the cropped image, (b) If the further probability is below the first threshold and above or equal to the second threshold, crop the region of the detected object in the cropped image in order to form a further cropped image. (c) Analyzing the further cropped image in order to determine another probability that the detected object is of the predetermined type, (d) If the further probability exceeds or is equal to the first threshold, the detected object is to be indicated as being of the predetermined type. Includes computer code that causes the following: Steps (a) to (d) are executed recursively until a predetermined condition is met. Non-temporary computer-readable storage medium.