An image processing system, and a method of image processing

Abstract

An image processing system and method capable for use with an ANPR (Fig.2a), comprises: an image capture device (14) arranged to capture an image / video 36 (22) of a designated region; a first processor (18) configured to detect the image of the designated region and divide the image into a plurality of subregions such as a grid overlay (27) and determine 38 the presence of one or more of activity, change or a feature of interest, transmitting 40 to a parallel processor 10 (20) such as a GPU, only the determined regions (26, 24) of the image; and process in parallel the image’s region(s) to determine information there from. The image capture device 14 may be a still photographic camera or a video camera arranged to capture video of a car park. Preferably the ANPR system will have an image capture device with a trigger (44, Fig.5) activated by a car passing over a cable or a heat sensor.

Description

The present invention relates to an image processing system and a method of image processing. The invention also relates to an automatic number plate recognition system incorporating an image processing system and operated according to an image processing method. In conventional processing systems a central processing unit is used for processing and executing instruction. Although modern CPUs are fast, as a result of the manner in which they process data and perform operations there is a natural limit in the available processing speeds. The processing of a CPU may be thought of as serial processing, since a problem to be solved 2 is generally broken down in a series of tasks 4 which are fed into and executed in serial sequence by the CPU 6. Such an arrangement is shown schematically in Figure 1A. The processing of video or images can be particularly slow using a CPU. To address this hardware known as a graphics processor was developed. The schematic structure of such a graphics processor can be seen in Figure 1B. It relies on what may be referred to as parallel processing, in contrast to the series processing of the CPU. The structure and operation enables many calculations or processes to be carried out simultaneously. As a result, graphics processors are generally considered fast and efficient. In terms of hardware structures, a CPU consists typically of four to eight CPU cores, while graphics processor parallel computing operates using scores of hundreds of smaller cores 10. The parallel computing nature of a graphics processor enables it to breakdown a large problem into a number of smaller simpler tasks 8, many of which can be performed simultaneously by a respective one of the plural cores 10 of the graphics processor. US7598958, US7689541 amongst many others disclose the general principles and some application of graphics processors. According to a first aspect of the present invention, there is provided an image processing system, comprising: an image capture device arranged to capture an image or video of a designated region; a first processor configured to detect the image or video of the designated region and divide the image into a plurality of subregions and determine whether or not in any of the subregions there is present one or more of activity, change or a feature of interest; responsive to a positive identification of the presence of one or more of activity, change or a feature of interest transmitting to a smart parallel processor the image or video only of the or each subregion in which the activity, change or feature of interest has been identified; a parallel processor arranged to receive the transmitted image or video of the or each subregion and process in parallel the image to determine information from the received image. A system is provided which is capable of use within an ANPR. An image or video of a designated region, such as, say, a region of a carpark covering plural parking spaces is captured. Subregions are then identified and only images of the subregions are forwarded to the parallel processor. The parallel processor is then able to determine information from the received image or video such as the identity of a vehicle. By making plural such determinations over a period of time it will be possible for the parallel processor is system as whole to determine how long a vehicle has been present in a certain location. This is particularly useful in ANPR systems. Of course, the process of vehicle recognition based on a number plate itself is well known technology and this can be achieved by use of functionality included within the image capture device or the first processor, before identified subregions of an image are sent to the parallel processor for further processing. The processing of the subregions by the parallel processor can be used to determine additional information relating to a vehicle that has been identified using ANPR associated with the first processor. The ANPR could also be done based on the subreqions received at the parallel processor but in practice it is likely that it will be done at the first processor based on the captured image. The parallel processor is tasked with more than identifying a number plate and analysing the data within it. The use of a parallel processor itself speeds up the processing of images and video and this is improved further by forwarding only subregions identified from a captured image. Furthermore, the fact that a first processor is used to identify the subregions of interest within a captured means that in an initial stage there is no bottleneck in terms of the transmission of data from the image capture device to the parallel processor. The captured images can be stored and processed in a convenient time scale for user and does not need to be transferred in real time to the parallel processor. Of course, the video or image data can be transferred in real time and the potential problems of limited bandwidth are reduced as a consequence of not all he data being sent. A parallel processor such as a graphics processor with parallel computing ability, i.e. including a plurality of sub processors, is particularly suitable for the processing of image and video data files. Graphics processing works well, however in some applications there can still be a need to enable faster and more efficient image or video processing. This is particularly the case in applications where a large, monitored region of space changes quickly in different subregions and there is a need to identify and process the subregions in which a change has occurred. In an embodiment, the first processor is arranged and configured to determine whether or not in any of the subregions there is present a vehicle. In an embodiment, the processor is arranged and configured to determine whether or not in any of the subregions there is movement of a vehicle. In an embodiment, the image processing system comprises a memory to store the image or video of the designated region. In an embodiment, the processor is arranged to process the stored image or video and determine whether or not in any of the subregions there is present one or more of activity, change or a feature of interest. According to a second aspect of the present invention, there is provided an automatic number plate recognition system, comprising: an image processing system according to the first aspect of the present invention. In an embodiment, the image capture device is arranged to capture an image or video of a region in which vehicles are expected to be present, either moving or stationary. In an embodiment, the region in which vehicles are expected to be present, either moving or stationary, is a car park or a section of a road. In an embodiment, the automatic number plate recognition system comprises an activation switch, responsive to an input to trigger the image capture device automatically to connect the image capture device to power when a vehicle enters or leaves the region in which vehicles are expected to be present, either moving or stationary. In an embodiment, the activation switch is a pressure activated switch. In an embodiment, the pressure activated switch is arranged to be activated by movement of a vehicle, the image or video of which is to be captured. In an embodiment, the switch is arranged on a region of ground surface in the vicinity of the image capture device, the switch being arranged to activate the image capture device in response to movement of the vehicle over the region of ground surface to capture an image or video. In an embodiment, the switch is activated only if a threshold force is applied to the mechanical switch, the threshold force being greater than 300KG. In an embodiment, the image capture device is integrated into a bollard, a fence post, a brick or other feature. According to a third aspect of the present invention there is provided a method of ANPR using a system, comprising an image capture device arranged to capture an image or video of a vehicle in a designated region, the method comprising, responsive to capture of an image of a region, dividing the image or video into a plurality of subregions; determining whether or not in any of the subregions there is present one or more of activity, change or a feature of interest and responsive to a positive identification of the presence of one or more of activity, change or a feature of interest transmitting to a parallel processor the image or video only of the or each subregion in which the activity, change or feature of interest has been identified; using the parallel processor to identify a vehicle in dependence on the received image or video of the or each subregion. According to a fourth aspect of the present invention there is provided a method of ANPR, comprising using a system according to the first aspect of the present invention. According to a fifth aspect of the present invention, there is provided an image processing system, comprising: an image capture device arranged to capture an image or video of a designated region; a first processor configured to detect the image or video of the designated region and divide the image into a plurality of subregions and determine whether or not in any of the subregions there is present one or more of activity, change ora feature of interest; and, responsive to a positive identification of the presence of one or more of activity, change or a feature of interest transmit to a smart parallel processor the image or video only of the or each subregion in which the activity, change or feature of interest has been identified. Preferably a parallel processor is provided arranged to receive the transmitted image or video of the or each subregion and process in parallel the image to determine information from the received image. According to a sixth aspect of the present invention, there is provided a method of ANPR using a system comprising an image capture device arranged to capture an image or video of a vehicle in a designated region, the method comprising: responsive to capture of an image or video of a region, dividing the image or video into a plurality of subregions; with a first processor determining whether or not in any of the subregions there is present one or more of activity, change or a feature of interest, and responsive to a positive identification of the presence of one or more of activity, change or a feature of interest transmitting to a parallel processor the image or video only of the or each subregion in which the activity, change or feature of interest has been identified. The fifth and sixth aspects of the invention correspond to the first and third aspects but do not include the parallel processor itself. All features that are described above as being provided with the first or third aspects could similarly be provided with the fifth and sixth aspects. Embodiments of the present invention will now be described in detail with reference to the accompanying drawings, in which: Figures 1A and 1B are schematic representations of computer processing systems; Figure 2A is a schematic representation of an example of an automatic number plate recognition system including an image processing system; Figure 2B is a schematic view of a further example of a segmented scene for use with the image processing system described herein; Figure 3 is a schematic representation of another example of an automatic number plate recognition system including an image processing system; Figure 4A is a simplified flow chart showing the steps in the operation of an ANPR system according to the present invention; Figure 4B is a fuller flow chart showing the steps in the operation of an ANPR system according to the present invention; and Figure 5 is a schematic representation of an image capture device for use in the ANPR system of figures 2 or 3; and Figure 2 is schematic representation of an ANPR system 12. The ANPR system 12 comprises an image capture device 14 arranged to capture an image 16 or video of a region, which in this example is shown as a carpark. The region or (image 16 thereof) may be considered the designated region since it encompasses the region of space which is within the view of the image capture device. The image capture device 14 may be a still photographic camera or a video camera arranged to capture video of the car park 16. A first processor 18 is provided coupled to the image capture device 14. The first processor is arranged to perform a review of the captured images 22, which can be either frames of the video or individual still images when taken as a sequence, and identify if there have been any changes. Upon identification of changes in a subregion of the still images (or in frames of the captured video) a determination is made by the first processor 18 that one or more subregions of the image(s) have included a feature or change that is worth further review. The identified subregion(s) of the captured images or frames are then communicated by the first processor 18 to a second processor, preferably forming part of a parallel processor or graphics processor 20, as described above. The communication of data between the image processing system 12 and specifically the first processor 18 thereof is therefore bandwidth efficient and preferred to systems in which all the captured data is transmitted to the graphics processor. Referring to the example of Figure 2 an image 22 of a carpark 16 is shown. The first processor serves to identify activity, change or a feature of interest within any subregions of the image or frame in question. In the example shown two subregions, 24 and 26 contain cars and are thus identified by the first processor as being subregions of interest. These subregions may then be extracted from the entire image 22 and communicated to the graphics processor 20, i.e. the parallel processor. The benefits of the speed of processing provided by graphics processor are thus significantly improved upon since the amount of data that needs to be processed is reduced. It will be appreciated of course that the subregions referred to with reference to Figure 2 may not be identified as regions of interest, if ,say, the vehicles shown in the subregions have been there for a long time and so not indicate any change. In this situation although the subregions would have been communicated to the second processor 20 (the parallel processor) when the vehicles arrived, or when the first frame was identified in which the vehicles could be seen, in subsequent frames the subregions might not be identified as being of interest. The division of the image 22 into subregions may be achieved as shown schematically in figure 2 by use of a grid 27 laid over the image 22. This is preferably done at or by the first processor. Each of the cells of the grid can be considered a potential subregion of interest or alternatively more than one cell of the grid can be considered a subregion of interest. In the example shown, subregion 26 is defined by a single one of the cells, whereas subregion 24 is made up of a plurality (4 in this nonlimiting example) contiguous cells of the grid. It will be appreciated that in the example shown, all the relevant subregions of interest are communicated to the graphics processor 20 whilst at the same time only 5 / 16ths of the image data is sent for processing by the graphics processor 20. Thus, the processing at the graphics processor and the execution of the ANPR processing will be quicker. In the present system, the graphics processor is preferably arranged to perform video analytics. In previous systems, to enable video analytics to work, a processor was expected to look at an entire screen and analyse everything within the field of view. This requires significant processing power. The present system therefore utilizes a first processor, which can be actual or virtual, and is arranged between the camera and the graphics processor and splits the area to be analysed into segments or subregions and only sends those segments or subregions where there has been movement, to be analysed by the graphics processor. The result is that the main processor only analyses the data that needs to be processed which makes the analysis process more efficient. The first processor can either send to the graphics processor still images taken from frames of a captured video or can send video subregions. In addition, the communication channel between the first processor 18 associated with the image processing device and the graphics processor can be smaller as lower volumes of data will be transferred through it than would be if the full frames were to be communicated. The communication of the images of the subregions of interest to the graphics processor 20 may be via a wired or wireless communication means or communication protocol. It will be appreciated that by making plural determinations over a period of time it will be possible for the parallel processor 20 or the system as whole to determine how long a vehicle has been present in a certain location. This is particularly useful in ANPR systems as it can be used to calculate how much a vehicle owner should be required to pay for having parked their vehicle in a particular space in a carpark. Figure 3 is another example of an ANPR using similar components to that of Figure 2. In this example the first processor (not shown separately from the image capture device 22) is arranged to communicate with the graphics processor 30 via communications network 32. Thus, in this example the benefits of reduced volume of data for transmission will be significant. In the example shown the first processor is provided as part of the image capture device 14. The first processor is arranged to perform a review of the captured images 22, which can be either frames of the video or individual still images when taken as a sequence and identify if there have been any changes. Upon identification of changes in a subregion of the still images (or in frames of the captured video) a determination is made by the first processor that one or more subregions of the image(s) have included a feature or change that is worth further review. The identified subregion(s) of the captured images or frames are then communicated by the first processor to a second processor being the graphics processor 30, as described above. The communication of data between the image capture device 14 and the graphics processor 30 is therefore bandwidth efficient and preferred to systems in which all the captured data is transmitted to the graphics processor. Referring to the example of Figure 3 an image 22 of a carpark 16 is shown. The first processor serves to identify activity, change or a feature of interest within any subregions of the image or frame in question. In the example a subregion 34 contains a space for a car. In the example shown it is clear that no car is present. However, there was previously a car present and so the subregion is identified as being of interest due to a change that has occurred (the vehicle leaving). The subregion 34 is extracted from the entire image 22 and communicated to the graphics processor 20 via the network 32. As in the example above described with reference to Figure 2, the benefits of the speed of processing provided by graphics processor are thus significantly improved upon since the amount of data that needs to be processed is reduced. Figures 4A and 4B are schematic flow diagrams showing the steps in a process for ANPR. The process is one which would typically be executed or performed using s system such as that shown in and described with reference to Figure 2 or 3. Referring to Figure 4B, initially at step 36 an image capture device captures an image which can be in the form of a still image or a video. Ideally the image capture device is a video camera arranged to film a region of interest and to store, preferably locally, the captured frames of video. At step 38 a first processor is arranged to identify within the captured still images or frames of the captured video, subregions of interest. These can be subregions in which a change has taken place, as compared to the same area of a previous frame or still image. This process of identifying subregions of interest is preferably performed using image recognition software or it can be performed manually. The subregion of interest will preferably be an area of the still image frame or the captured still image in which one or more of activity, change or a feature of interest are identified. This step can involve the comparison of two consecutive or sequential, i.e. not necessarily directly consecutive, still frames from a video or two consecutive or sequential captured still images. Image subtraction or other known image comparison methods are preferably used. The identified subregions are then extracted and communicated to the graphics processor 10 for further processing as described above. As explained above, the process of ANPR 39 can be performed at the first processor, or at the graphics processor 10. It is expected that the ANPR will be performed at the first processor and further image processing relating to the identified subregions, which could contain identified objects of interest are then provided to the graphics processor for further processing. Thus, instead of processing all of the captured data from the image capture device, only subregions from captured images need to be forwarded on to the graphics processor 10for full processing. In the example of use with ANPR system it will be appreciated that the subregions forwarded for processing to the GPU will be subregions that include an image or representation of the vehicle number plate. The graphics processor is arranged to execute ANPR software which can be any commercially available ANPR system software arranged to run on the graphics processor. Alternatively the ANPR can be performed upstream as described above the graphics processor be arranged to perform other image or video analysis on the received subregions of still or video data The effect of use of a method such as that shown in and described with reference to Figures 4A and 4B is that a significant increase in the speed of result generation can be achieved. Figure 5 is a schematic representation of an automatic number plate recognition system according to a present embodiment. In this example, the ANPR system is integrated into a bollard or post which can be arranged within a car park. In this embodiment, the system provides a self-contained, self-powered camera system that is able to take an image or video only when triggered by movement. Thus, the system is able to capture a vehicle on arrival and / or on departure too. The image capture device 14 is coupled to, or includes a first processor, such as that described above which is therefore able to transmit to an associated graphics processor only the subregions of captured images or video frames which include an indication of one or more of activity, change or a feature of interest. This example further includes power saving components to be described below. The system includes an image capture device 14 having a trigger 44 which is mechanical and is therefore capable of being activated by a vehicle, such as a car passing over a cable or a heat sensor. Accordingly, the image capture device 14, such as a digital video or stills camera, utilizes less power and therefore renders power by a battery pack as a feasible technical solution. The exemplary system 46 includes an image capture device in the form of a digital camera 14 fixably mounted within a housing 48. The housing, in this example, is in the form of a bollard 50 which is preferably anchored via a base 52 to the ground at an appropriate position. The camera will be arranged to capture or film, when activated a region that includes multiple subregions divided in a manner as shown in, say, Figures 2A and 3. The bollard 50 is connected via a wired connection 54 to a region of the ground surface 56 which, in this example is provided with a speed bump 58. The speed bump 58 has arranged on it (not shown) a pressure activated switch 44, with a connector 54 to the digital camera 14. The system is arranged such that when the speed bump 58 is driven over by a vehicle, such as a car, the pressure activated switch 44 causes a signal to be provided to the camera 14. The signal could be an on / off signal and / or it could preferably include power which, is arranged to activate the camera 14 and cause it to capture an image of the car park. Thus, the camera is activated when a vehicle enters the car park. The image capture device 14 is configured to capture an image of a region of the car park which itself includes subregions defined by a division of the captured mage, in a manner as described above. Thus, it will be appreciated that the pressure activated switch 44 could be a switch that causes a power source within the camera to be connected to the camera’s circuitry, thus enabling it to take a picture. Alternatively, it is also possible that no permanent power source is provided within the camera and instead the pressure activated switch includes power generation functionality. In this case, in response to pressure being applied (by the vehicle driving over the speed bump 58 and activating the switch 44) sufficient power is generated to activate the camera and cause it to take a picture, and subsequently to store it or to transmit it (or subregions of it) to wherever it might be needed for analysis, as described above. In other words, in one nonlimiting example the pressure activated switch 44 is a microgenerator positioned such that when a vehicle enters or leaves the designated region, the vehicle engages with the microgenerator to cause it to generate power for the image capture device. The power is communicated to the image capture device typically by a wired connection which causes the image capture device to “wake up” and capture the image of the vehicle entering the space. The power generated is preferably sufficient to also cause the image capture device to transmit the captured image, or subregions thereof, to a connected server for processing in known ways. As described, in the example shown in Figure 1, the pressure activated switch 44provided on the speed bump is activated when a vehicle, such as a car, drives over it. In an alternative example, the switch is provided in the form of a heat sensor which is triggered to provide an on / off signal and / or to the camera when a vehicle is in close proximity to it or engages it by driving over it. The system 46 shown in Figure 5 comprises a bollard 50 in which the image capture device 14 is housed. The first processor described above is also preferably included within the bollard 50. It can be provided as integrated part of the image capture device 14 or separate connected component, as described above with reference to Figure 2. It will be appreciated that other forms of housing for the image capture device 14 and the first processor can be provided. For example, a fence post can be used or alternatively the image capture device 14 and the first processor can be integrated into a brick or the wall of a building in which a vehicle is to be parked. One advantage of the system described with reference to Figure 5 is that the ANPR system can be entirely in sleep mode or disconnected from a power source (and drawing no power from it) unless and until a vehicle enters the car park and causes activation of the image capture device 14, by virtue of triggering the switch 44. The image capture device and first processor 14 are only activated when required to capture an image of the car park due to a vehicle either leaving or arriving. This means that the power drawn from a power source in the image capture device 14 (if provided in the form of a battery) is minimal or in fact zero. The system at most times does not need any power at all. This means that a small replaceable battery, e.g. AA, AAA etc, can be used and can be expected to have a long active life whilst still enabling efficient and correct functioning of the ANPR system. Embodiments of the present invention have been described with particular reference to the examples illustrated. However, it will be appreciated that variations and modifications may be made to the examples described within the scope of the present invention.

Claims

1. An image processing system, comprising:an image capture device arranged to capture an image or video of a designated region;a first processor configured to detect the image or video of the designated region and divide the image or video into a plurality of subregions and determine whether or not in any of the subregions there is present one or more of activity, change or a feature of interest;responsive to a positive determination of the presence of one or more of activity, change or a feature of interest transmitting to a parallel processor the image or video only of the or each subregion in which the activity, change or feature of interest has been identified;a parallel processor arranged to receive the transmitted image or video of the or each subregion and process in parallel the image or video to determine information from the received image.

2. An image processing system according to claim 1, in which the first processor is arranged and configured to determine whether or not in any of the subregions there is present a vehicle.

3. An image processing system according to claim 1, in which the first processor is arranged and configured to determine whether or not in any of the subregions there is movement of a vehicle.

4. An image processing system according to any of claims 1 to 3, comprising a memory to store the image or video of the designated region.

5. An image processing system according to claim 4, in which the first processor is arranged to process the stored image or video and determine whether or not in any of the subregions there is present one or more of activity, change or a feature of interest.

6. An image processing system according to any of claims 1 to 5, arranged to process the captured image or video of the designated region at the first processor andperform an automatic number plate recognition process on the captured image to identify any vehicle in the image or video of the designated region7. An automatic number plate recognition system, comprising: an image processing system according to any of claims 1 to 6.

8. An automatic number plate recognition system according to claim 7, in which the image capture device is arranged to capture an image or video of a region in which vehicles are expected to be present, either moving or stationary.

9. An automatic number plate recognition system according to claims 7 or 8, in which the region in which vehicles are expected to be present, either moving or stationary, is a car park or a section of a road.

10. An automatic number plate recognition system according to any of claims 7 to 9, comprising an activation switch, responsive to an input to trigger the image capture device automatically to connect the image capture device to power when a vehicle enters or leaves the region in which vehicles are expected to be present, either moving or stationary.11 A system according to claim 10, in which the activation switch is a pressure activated switch.

12. A system according to claim 10 or 11, in which the pressure activated switch is arranged to be activated by movement of a vehicle, the image or video of which is to be captured.

13. A system according to claim 11 or 12, in which the switch is arranged on a region of ground surface in the vicinity of the image capture device, the switch being arranged to activate the image capture device in response to movement of the vehicle over the region of ground surface.

14. A system according to any of claims 11 to 13, in which the switch is activated only if a threshold force is applied to the mechanical switch, the threshold force being greater than 300KG.

15. A system according to any of claims 10 to 14, in which the image capture device is integrated into a bollard, a fence post, a brick or other feature16. A method of ANPR using a system comprising an image capture device arranged to capture an image or video of a vehicle in a designated region, the method comprising:responsive to capture of an image or video of a region, dividing the image or video into a plurality of subregions;with a first processor determining whether or not in any of the subregions there is present one or more of activity, change or a feature of interestand responsive to a positive identification of the presence of one or more of activity, change or a feature of interest transmitting to a parallel processor the image or video only of the or each subregion in which the activity, change or feature of interest has been identified;using the first processor or the parallel processor to identify a vehicle in dependence on the captured image or video or the received image or video of the or each subregion.

17. A method of ANPR, comprising using a system according to any of claims 1 to 15.

18. An image processing system, comprising:an image capture device arranged to capture an image or video of a designated region;a first processor configured to detect the image or video of the designated region and divide the image into a plurality of subregions and determine whether or not in any of the subregions there is present one or more of activity, change or a feature of interest; and, responsive to a positive identification of the presence of one or more of activity, change or a feature of interest transmit to a smart parallel processor the image or video only of the or each subregion in which the activity, change or feature of interest has been identified.

19. An image processing system according to claim 18, in which the first processor is arranged and configured to determine whether or not in any of the subregions there is present a vehicle.

20. An image processing system according to claim 18 or 19, in which the first processor is arranged and configured to determine whether or not in any of the subregions there is movement of a vehicle.

21. A method of ANPR using a system comprising an image capture device arranged to capture an image or video of a vehicle in a designated region, the method comprising: responsive to capture of an image or video of a region, dividing the image or video into a plurality of subregions;with a first processor determining whether or not in any of the subregions there is present one or more of activity, change or a feature of interestand responsive to a positive identification of the presence of one or more of activity, change or a feature of interest transmitting to a parallel processor the image or video only of the or each subregion in which the activity, change or feature of interest has been identified

Images