Method, apparatus, device and storage medium for detecting an object

Abstract

According to an embodiment of the present disclosure, a method, an apparatus, a device and a storage medium for detecting an object are provided. The method comprises determining a candidate image frame potentially containing at least one object from a video. At least one reference image frame associated with the candidate image frame is determined from the video. Then, based on a potential position of the at least one object in the candidate image frame, a plurality of image segments potentially containing the at least one object in the candidate image frame and in each of the at least one reference image frame are determined. Finally, based on at least one differential image segment between the plurality of image segments, the at least one object is detected in the candidate image frame. Thus, by determining the image segments first and then detecting the object through the image segments, the interference of noise in the background region can be avoided, and thus the robustness and accuracy of object detection can be improved.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and more particularly to methods, apparatus, devices, and computer-readable storage media for detecting objects. Background Technology

[0002] With the increasing demand for scenario-based monitoring and intelligent analysis across various industries, video-based object detection technology has received widespread attention and rapid development. This technology enables efficient and accurate identification of dynamic targets in video or image sequences through intelligent analysis of video footage. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for detecting an object is provided. The method includes: determining candidate image frames from a video that potentially contain at least one object; determining at least one reference image frame associated with the candidate image frames from the video; determining a plurality of image segments potentially containing at least one object in each of the candidate image frames and the at least one reference image frame, based on the potential location of at least one object in the candidate image frames; and detecting at least one object in the candidate image frames based on at least one differential image segment between the plurality of image segments.

[0004] In a second aspect of this disclosure, an apparatus for detecting an object is provided. The apparatus includes: a first determining module configured to determine candidate image frames potentially containing at least one object from a video; a second determining module configured to determine at least one reference image frame associated with the candidate image frames from the video; a third determining module configured to determine, based on the potential location of at least one object in the candidate image frames, a plurality of image segments potentially containing at least one object in each of the candidate image frames and the at least one reference image frame; and a detection module configured to detect at least one object in the candidate image frames based on at least one differential image segment between the plurality of image segments.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the method of the first aspect.

[0008] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A flowchart of a process for detecting an object according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A schematic diagram illustrating a process for detecting fireworks according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A flowchart is shown illustrating a process for generating a training dataset according to some embodiments of the present disclosure;

[0014] Figure 5 A schematic diagram illustrating the process of generating a training dataset for training a smoke sequence determination model according to some embodiments of the present disclosure is shown.

[0015] Figure 6 A schematic diagram of a process for model training according to some embodiments of the present disclosure is shown;

[0016] Figure 7 A block diagram of an apparatus for detecting an object according to some embodiments of the present disclosure is shown; and

[0017] Figure 8 A block diagram of an apparatus capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0020] In this document, unless explicitly stated otherwise, performing a step in response to A does not mean that the step is performed immediately after A, but may include one or more intermediate steps.

[0021] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0022] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0023] The solutions described in this specification and embodiments, if involving the processing of personal information, will be processed only on the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be processed within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0024] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," a "machine learning network," or simply a "network," and these terms are used interchangeably.

[0025] For ease of description, several concepts and terms are first defined here.

[0026] Image classification is a core task in computer vision. Its goal is to distinguish different categories of images based on the different features reflected in the image information. It involves selecting a category label for a given input image from a known set of category labels. Examples include classifying cat and dog images, classifying whether mechanical parts are defective, and classifying whether the content / quality of a photo is acceptable.

[0027] Object detection is one of the core problems in the field of computer vision. Its task is to find all objects of interest in an image and determine their category and location.

[0028] Frame difference method is one of the most commonly used methods for moving target detection and segmentation. The basic principle is to extract the moving region in the image by using pixel-based temporal difference between two or three adjacent frames of the image sequence and then applying thresholding.

[0029] Optical flow is a method that uses the temporal changes of pixels in an image sequence and the correlation between adjacent frames to find the correspondence between the previous frame and the current frame, thereby calculating the motion information of objects between adjacent frames.

[0030] Feature detection, a concept in computer vision and image processing, refers to using computers to extract information from images and determine whether each point in an image belongs to a specific image feature. The result of feature detection is the division of points in an image into different subsets, which often belong to isolated points, continuous curves, or continuous regions. Main classifications include: edges, corners, regions, and ridges. Classical feature extraction operators include Histogram of Oriented Gradients (HOG), Scale-Invariant Feature Transform (SIFT), and Local Binary Pattern (LBP).

[0031] Video compression refers to the use of compression technology to remove redundant information from digital video, reducing the storage required to represent the original video, thus facilitating the transmission and storage of video data. Video compression can significantly reduce video transmission and storage costs, improve the real-time performance and energy efficiency of video surveillance systems, and is one of the core technical tasks in the security field.

[0032] Bitrate is the number of bits of data transmitted per unit of time during data transmission. It's usually measured in kbps (kilobits per second) and is also known as sampling rate. At the same resolution, a higher bitrate results in a lower compression ratio, higher picture quality, but a larger file size. Sampling rate is not the same as sampling frequency. Sampling frequency is measured in Hz and represents the number of samples taken per second.

[0033] Dynamic bitrate refers to the process of dynamically adjusting the bitrate during audio and video encoding based on factors such as scene complexity and content changes. Fixed bitrate, on the other hand, uses a uniform bitrate throughout the encoding process without adjustment based on scene complexity or content variations. Compared to fixed bitrate, dynamic bitrate can better adapt to scene changes and reduce bitrate while maintaining image quality, thus improving transmission efficiency.

[0034] Video flicker refers to the phenomenon of pixel value fluctuations at the same location during video recording. Visually, it manifests as brightness / color flickering. The causes of video flicker are numerous. There is a certain relationship between video flicker and video bitrate; low bitrates / dynamic bitrates exacerbate the problem. If the bitrate is set too low, it may lead to a decrease in video quality, resulting in issues such as pixelation and blurring, which can in turn cause video flicker. Dynamic bitrates further intensify the flickering.

[0035] Signal-to-noise ratio (SNR) is the ratio of signal to noise in a system. Both SNR and signal-to-interference-plus-noise ratio (SINR) are technical indicators for measuring the reliability of communication system quality. The difference is that SNR is the ratio of signal to noise, while SINR incorporates the impact of interference, making it the ratio of the signal to the sum of interference and noise.

[0036] As briefly mentioned earlier, video-based object detection technology can achieve efficient and accurate identification of dynamic targets in video or image sequences through intelligent analysis of video footage. For example, frame differencing can be used to identify dynamic targets in videos. Typically, frame differencing for identifying dynamic targets in videos can include multi-frame recognition and image sequence recognition methods.

[0037] For multi-frame recognition, a recognition scheme based on extracted frames from two or more videos can be constructed using classic machine vision algorithms, classic machine learning, deep learning, etc., outputting the location information and confidence score of the target region. Typically, multi-frame recognition methods can be categorized into the following three types:

[0038] 1) Secondary filtering, in which target detection can be performed on a certain image frame among multiple image frames first, then "energy determination" can be performed based on the frame difference image of multiple image frames, and finally, comprehensive discrimination of multiple image frames can be performed based on the strategy;

[0039] 2) Feature extraction, in which target detection can be performed on multiple consecutive image frames first, dynamic and static features can be fused based on the frame difference image of multiple image frames, then pixel value statistics can be performed to distinguish between targets and false detections, and finally a comprehensive discrimination of multiple image frames can be performed based on the strategy.

[0040] 3) Potential target region extraction, which can first calculate the frame difference of multiple image frames, extract motion / foreground regions, perform pixel value statistics, further distinguish between targets / false detections, and finally perform comprehensive discrimination of multiple image frames based on the strategy.

[0041] For image sequence recognition, target recognition schemes can be constructed based on image segments, utilizing classic machine vision algorithms, classic machine learning, and deep learning. The output results are the location information and confidence score of the target region. Typically, this method processes multiple image frames, but the number of image frames input to a single model is greater. It should be understood that, unlike multi-frame recognition methods, image sequence recognition usually uses multiple image frames simultaneously as model input, fusing multiple image frames or features with frame difference images or features for target recognition.

[0042] However, the above-mentioned recognition methods rely on empirical thresholds. Since these thresholds vary significantly across different application scenarios, they lack transferability and cannot be applied on a large scale (adapting parameters individually for each scenario / time period results in extremely high maintenance costs), leading to poor robustness. Furthermore, because the target signal intensity in the frame difference image is reduced compared to the original image, the frame difference feature intensity in the moving region often exceeds that in the target region. Therefore, motion interference cannot be distinguished, requiring the integration of other information to remove noise, resulting in low recognition accuracy.

[0043] Especially in the field of video surveillance, factors such as video compression, motion noise, and background noise commonly cause video flickering. Video flicker severely damages frame difference images, not only directly destroying frame difference features but also introducing background semantic noise.

[0044] Typically, video compression techniques such as bitrate reduction and dynamic bitrate adjustment are used, which introduce noise. For example, for static background regions (non-target regions) in frame difference images, the introduced noise can include background frame difference pixel differences and background semantic noise; for dynamic foreground regions (target regions) in frame difference images, the introduced noise can include background frame difference pixel differences and background semantic noise.

[0045] For example, due to video quality fluctuations, the features of the target region in the frame difference image are diluted, and strong background semantic noise (e.g., human figures and actions) also appears. This leads to a decrease in the signal-to-noise ratio of the target region features in the frame difference image, and the extent of this decrease is related to factors such as the degree of video compression and the difference between background and foreground semantics, and therefore fluctuates considerably.

[0046] For example, due to video quality jitter, non-target areas on the frame difference image may exhibit similar visual features to the target area, making it impossible to distinguish between target and non-target areas by the maximum or average value of the frame difference pixel values.

[0047] In other words, even with weak background semantic noise, the methods described above cannot distinguish between real and fake targets in scenarios with shaky video quality. Distinguishing between real and fake targets becomes even more difficult under strong background semantic noise.

[0048] Embodiments of this disclosure propose a scheme for detecting objects. According to various embodiments of this disclosure, candidate image frames potentially containing objects are first determined from a video. Further, at least one reference image frame associated with the candidate image frames is determined from the video. Based on the potential location of the object in the candidate image frames, multiple image segments potentially containing the object are determined in each image frame of the candidate image frames and the at least one reference image frame. The object is detected in the candidate image frames based on at least one differential image segment between the multiple image segments. In this manner, by first determining image segments and then detecting objects using those image segments, interference from background noise can be avoided, thereby improving the robustness and accuracy of object detection.

[0049] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0050] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. See also Figure 1 Environment 100 may include electronic device 110 and image acquisition device 120.

[0051] In environment 100, image acquisition device 120 can be used to acquire video of target area 130 and send the acquired video to electronic device 110. Electronic device 110 can process the received video. For example, it can perform target detection, target tracking, etc., based on the received video.

[0052] It should be understood that Figure 1 The number of electronic devices 110, image acquisition devices 120, and target areas 130 included in environment 100 is merely illustrative. In practical applications, the number of electronic devices 110, image acquisition devices 120, and target areas 130 can be greater. Furthermore, the deployment of electronic devices 110 and image acquisition devices 120 can adopt any suitable deployment method in the related art, and the embodiments of this disclosure are not limited in this regard.

[0053] Image acquisition device 120 can be a camera, webcam, camcorder, infrared / thermal imaging device, etc. Electronic device 110 can be any type of device with computing capabilities, including terminal devices or server devices.

[0054] Terminal devices can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof.

[0055] Server-side devices can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server-side devices may include, for example, computing systems / servers such as mainframes, edge computing nodes, and computing devices in cloud environments, etc.

[0056] The communication connection between the electronic device 110 and the image acquisition device 120 can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections; the embodiments disclosed herein are not limited in this respect.

[0057] It should be understood that the structure and function of environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0058] Figure 2 A flowchart of a process 200 for detecting an object according to some embodiments of the present disclosure is shown. For example, process 200 may be implemented in Figure 1 In electronic device 110. Next, in conjunction with... Figure 1 A detailed description of process 200 is provided.

[0059] Electronic device 110 can acquire video to be processed. Such video can be a sequence of videos or images containing an object to be detected, or a sequence of videos or images potentially containing an object. Objects can include static objects and / or dynamic objects. Static objects can, for example, refer to stationary people or objects. Dynamic objects can, for example, include human bodies, animals, vehicles, smoke, open flames, etc. In some embodiments, electronic device 110 can acquire video to be processed from image acquisition device 120 in real time. In other embodiments, electronic device 110 can also acquire video containing the object to be processed from local or other storage devices (e.g., other electronic devices or third-party data platforms). When video to be processed is acquired by electronic device 110 from local or other storage devices, the video can be acquired by image acquisition device 120 and stored in the local or other storage devices of electronic device 110 (such video can also be referred to as "historical video").

[0060] The number of objects to be detected in a video can be single or multiple. In some real-world scenarios, the first or first few image frames of the video to be processed may not contain any objects. Electronic device 110 can automatically identify all objects in the first image frame in which objects appear in the video as objects to be detected, or receive a specification about objects (e.g., receive a user selection) and identify one or more of the multiple objects appearing in the image frame as objects to be detected.

[0061] In box 210, electronic device 110 determines candidate image frames from the video that potentially contain at least one object. Such image frames may contain an object and / or contain suspected objects. For example, in the case of smoke, a suspected object could be floating dust.

[0062] In some embodiments, the electronic device 110 may utilize a model deployed locally or on other devices (e.g., other electronic devices or third-party platforms) to extract candidate image frames from a video. For the purposes of discussion, such a model may be referred to as a "first machine learning model." For example, the first machine learning model may be a conventional machine learning model, such as K-Nearest Neighbors (KNN), Support Vector Machine (SVM), etc. As another example, the first machine learning model may also be a deep learning model, such as a Convolutional Neural Network (CNN), Vision Transformer (ViT), etc. The embodiments of this disclosure are not limited in this regard.

[0063] In some embodiments, the electronic device 110 may input each image frame in the video sequentially into the first machine learning model according to the time order of the image frames, or it may extract a certain number of image frames from the video and input them into the model to determine whether each image frame potentially contains an object, and determine the image frames that potentially contain objects as candidate image frames. In some embodiments, the electronic device 110 may recall image frames that do not contain objects, so that the image frames that do not contain objects can be used for training subsequent models, which will be described below.

[0064] In block 220, electronic device 110 determines at least one reference image frame associated with a candidate image frame from the video. The image frames in the video may be arranged in chronological order. In some embodiments, electronic device 110 may extract at least one image frame from the video as at least one reference image frame at time intervals (e.g., 3 seconds), using the candidate image frame as a reference. The at least one reference image frame may include at least one image frame preceding the candidate image frame and / or at least one image frame following the candidate image frame. In some embodiments, electronic device 110 may also extract at least one image frame from the video as at least one reference image frame at time intervals, using an image frame associated with the candidate image frame (e.g., a frame preceding or following the candidate image frame) as a reference. Alternatively or additionally, image frames closer to the candidate image frame may be preferentially selected as reference image frames.

[0065] Suppose that electronic device 110 determines the potential object contained in the image frame at time t from the video. At this time, the electronic device may, for example, extract the image frames at time t-3s and t+3s from the video as reference image frames at 3-second time intervals, using the image frame at time t as a reference; or extract the image frames at time t-3s and t-6s from the video as reference image frames; or extract the image frames at time t+3s and t+6s from the video as reference image frames.

[0066] In some embodiments, the electronic device 110 can also recall image frames that do not contain objects at the aforementioned time intervals. For example, it is determined from the video that the image frame at time t does not contain an object. In this case, the electronic device 110 can recall image frames at time t+3s, image frames at time t+6s, etc., for use in subsequent model training.

[0067] In box 230, electronic device 110 determines, based on the potential location of at least one object in a candidate image frame and in each of at least one reference image frames, multiple image segments that potentially contain at least one object.

[0068] The potential location of an object in a candidate image frame can be indicated by the object's coordinates within the candidate image frame or by the coordinates of the region where the object is located. Electronic device 110 can utilize models deployed locally or on other devices (e.g., other electronic devices or third-party platforms) to extract candidate image frames from the video. This model can be the first machine learning model described above. Of course, electronic device 110 can also utilize other models with similar functionality to determine the potential location of objects. Other models with similar functionality can be deployed locally on electronic device 110 or on other devices.

[0069] After determining the potential location of an object in a candidate image frame, the electronic device 110 can extract image segments corresponding to the potential location from the candidate image frame and at least one reference image frame as multiple image segments. An image segment refers to a local region cropped or extracted from an image frame. When determining image segments, the electronic device 110 can, for example, use an image processing tool to crop or extract images of local regions corresponding to the potential location of the object from each image frame in the candidate image frame and at least one reference image frame; that is, image segments potentially containing the object. In this way, background noise interference can be eliminated, retaining only the features of the foreground region.

[0070] In frame 240, electronic device 110 detects at least one object in a candidate image frame based on at least one differential image segment between multiple image segments.

[0071] In embodiments of this disclosure, at least one differential image segment includes differential image segments between each image segment in at least one reference image frame and an image segment in a candidate image frame. Here, a differential image segment may sometimes also be referred to as a "frame difference image segment." A differential image segment is an image obtained by performing a difference operation between each image segment in at least one reference image frame and an image segment in a candidate image frame. Differential image segments can reflect changes in the position and / or pixel values ​​of an object between two image frames. Since background noise interference has been removed from the image segments, differential image segments can accurately reflect changes in the position and / or pixel values ​​of an object between two image frames.

[0072] In some implementations, for an image segment in a candidate image frame and an image segment in a reference image frame, the electronic device 110 can perform a pixel-by-pixel difference operation on the two image segments, such as calculating the difference between the grayscale values ​​or color values ​​of corresponding pixels, thereby obtaining a difference image segment between the two image segments.

[0073] In some embodiments, the electronic device 110 may utilize a model deployed locally or on other devices (e.g., other electronic devices or third-party platforms) to detect objects in candidate image frames based on differential image segments. For the purposes of discussion, such a model may be referred to as a "second machine learning model." Similar to the first machine learning model, the second machine learning model may be a conventional machine learning model, such as K-Nearest Neighbors (KNN), Support Vector Machine (SVM), etc.; or it may be a deep learning model, such as Convolutional Neural Network (CNN), Vision Transformer (ViT), etc. The embodiments of this disclosure are not limited thereto.

[0074] When electronic device 110 detects objects in candidate image frames using a second machine learning model based on differential image segments, it can use the differential image segments as input to the second machine learning model to detect objects in the candidate image frames. However, when there are multiple types of objects or multiple similar objects, these objects may have overlapping features in the differential image segments. If all differential image segments are input into the second machine learning model for detection, feature confusion may occur, affecting the detection accuracy. In this case, the second machine sub-model can be split into multiple independent sub-models, each responsible for detecting one or more features, which can avoid feature confusion and improve detection accuracy.

[0075] In some embodiments, the second machine learning model may include a first sub-model and a second sub-model. The first sub-model is configured to detect a first object among at least one objects, and the second sub-model is configured to detect a second object among at least one objects. Here, the first object and the second object may be objects of different types.

[0076] The first and second objects can be two objects that overlap and occlude within the same image frame. For example, in a smoke detection scenario, the first object could be smoke, and the second object could be an open flame. When detecting smoke, smoke is usually accompanied by an open flame. Similarly, when detecting an open flame, the open flame is usually accompanied by smoke. Such two objects can lead to feature confusion.

[0077] At this point, if the same model is used to detect either the first or second object, the accuracy of detection will be affected due to feature confusion between the first and second objects. Therefore, two independent models can be used to detect the first and second objects separately. That is, the electronic device 110 can use at least one differential image segment as input to the first sub-model to detect the first object, and use at least one differential image segment as input to the second sub-model to detect the second object. This avoids feature confusion and improves the accuracy of detection.

[0078] In some embodiments, the video acquired by the electronic device 110 may be a compressed video. For example, the video may be from a video surveillance scene. Videos from video surveillance scenes often flicker due to factors such as video compression, motion noise, and background noise. Video flicker severely damages the frame difference image, not only directly destroying the frame difference features of the object but also introducing background semantic noise. By first identifying multiple image segments that potentially contain the object, and then detecting the object in the candidate image frame based on at least one difference image segment between the multiple image segments, the influence of the aforementioned factors on the detection results can be effectively avoided.

[0079] In some embodiments, the electronic device 110 may also trigger an alarm notification in response to detecting an object in a candidate image frame. For example, the alarm notification can be implemented through on-site alarms, reporting to a platform or server, or push notifications to users, thereby providing fire safety early warning. The alarm notification can be a visual alarm notification or an auditory alarm notification. The embodiments of this disclosure do not limit the specific form of the alarm. For example, visual alarm notifications may include flashing warnings on the monitoring screen, pop-up alarm prompts, etc.; auditory alarm notifications may include voice broadcasts, siren blasts, etc.

[0080] To more clearly explain the detection methods provided in some embodiments of this disclosure, the following description uses examples of smoke / open flame detection in the field of video surveillance. It is understood, of course, that the embodiments of this disclosure are not limited to this application scenario.

[0081] In the field of video surveillance, smoke and / or open flame detection differs from other video surveillance tasks. Due to the high requirements for recall and accuracy, it can be identified by integrating a multi-stage artificial intelligence (AI) smoke and fire recognition method that combines cloud, edge, and terminal. That is, a solution may include multiple modules such as terminal model, edge computing model, cloud model, and cloud strategy to achieve high-precision recognition.

[0082] As described above, smoke and / or open flames can be identified using multi-frame recognition or image sequence recognition. The process includes the following steps: foreground target extraction, detection of static and dynamic smoke features (frame difference threshold pixel statistics), and alarm strategies. For example, during smoke recognition, pixel statistics of the frame difference image or simple filtering rules can be used to perform secondary filtering on suspected smoke areas detected by the smoke recognition module. That is, when the frame difference pixel value of a single pixel or region reaches a certain threshold, it can be considered as caused by motion or a smoke event.

[0083] However, the videos used for smoke and fire detection are typically surveillance videos, which are usually compressed. In quantitative analysis of frame difference images under video compression scenarios, the maximum and average values ​​of frame difference pixel values ​​cannot distinguish between smoke and non-smoke areas. In other words, under video compression scenarios, relying on pixel value statistics and thresholds cannot distinguish between background areas (non-smoke and fire areas) and moving areas (smoke and fire areas).

[0084] In video compression scenarios, smoke and non-smoke areas often lack first-order (pixel value) separable features in the frame difference image. This means that simple pixel statistics and thresholding cannot accurately distinguish between smoke and non-smoke areas. Furthermore, the aforementioned methods generally suffer from drawbacks such as reliance on strong settings (significant pixel value changes and distribution characteristics in smoke areas), neglect of the realities of the video surveillance industry (high-ratio video compression, video flicker, long-tail distribution of smoke events, etc.), and computational complexity and inefficiency. Consequently, they cannot achieve accurate smoke and fire identification and are difficult to implement on a large scale.

[0085] Figure 3 A schematic diagram of a process 300 for detecting fireworks according to some embodiments of the present disclosure is shown. For example, process 300 can be implemented in... Figure 1 In electronic device 110. Next, in conjunction with... Figure 1 The process 300 is described in detail.

[0086] Electronic device 110 can acquire real-time video from image acquisition device 120, or acquire historical video from local or other storage devices. After acquiring the video, electronic device 110 can input image frames from the video into smoke and fire target detection model 303 to detect whether smoke and / or open flames exist in the image. Regardless of whether smoke and / or open flames are detected, image frames need to be recalled. The difference is that image frames for which smoke and / or open flames were not detected can be used for training smoke sequence determination model 304 and / or open flame sequence determination model 305.

[0087] If an image frame at time t+3s is detected to potentially contain smoke and / or open flame, these frames can be recalled for further detection. If an image frame at time t+3s is detected not to contain smoke and / or open flame, these frames can be recalled for model training.

[0088] Before further detection, the electronic device 110 can extract the image frame at time t from the video as a reference image frame. At the same time, based on the potential positions of smoke and / or open flame in the image frame at time t+3 output by the smoke and / or open flame target detection model 303, it uses image processing tools to extract image segments corresponding to the potential positions from the image frame at time t and the image frame at time t+3s, respectively, and performs differential operations on the extracted image segments to obtain differential image segments.

[0089] If the smoke target detection model 303 detects smoke in the image frame at time t+3s, the electronic device 110 can input the corresponding differential image segment into the smoke sequence determination model 304 for further detection. If the smoke target detection model 303 detects open flame in the image frame at time t+3s, the electronic device 110 can input the corresponding differential image segment into the open flame sequence determination model 305 for further detection.

[0090] If real smoke is detected by the smoke sequence determination model 304, the electronic device 110 can perform operations such as reporting, analysis, and early warning. If false smoke (e.g., floating dust) is detected, it can be ignored. If real fire is detected by the open flame sequence determination model 305, the electronic device 110 can perform operations such as reporting, analysis, and early warning. If false fire (e.g., an object with a similar color to real fire) is detected, it can also be ignored.

[0091] Because existing open-source datasets in the field of smoke and fire recognition are relatively small and mostly consist of images or videos, there is a lack of frame difference image datasets. Therefore, there is a shortage of training data when training the smoke sequence determination model 304 and the open flame sequence determination model 305. To address this, in some embodiments, a training dataset can be generated for training a second machine learning model (e.g., including sub-models such as the smoke sequence determination model 304 and the open flame sequence determination model 305).

[0092] Figure 4 A flowchart of a process 400 for generating a training dataset according to some embodiments of the present disclosure is shown. For example, process 400 may be implemented in Figure 1 In electronic device 110. Next, in conjunction with... Figure 1 A detailed description of process 400 is provided.

[0093] In box 410, electronic device 110 determines from the sample video a sample image frame containing at least one reference object associated with at least one object. The reference object may include the object to be detected itself, or an object with similar features to the object to be detected.

[0094] Take the construction of a training dataset for training the smoke sequence determination model 304 as an example. Figure 5 A schematic diagram of a process 500 for generating a training dataset for training a smoke sequence determination model 304, according to some embodiments of the present disclosure, is shown. Figure 5 As shown, electronic device 110 can acquire a fireworks video dataset 501, which can be open source. Electronic device 110 can input image frames of sample videos from the fireworks video dataset 501 into the fireworks target detection model 303 to detect whether the image frames input into the fireworks target detection model 303 contain real smoke. Simultaneously, the fireworks target detection model 303 can also output the coordinates of real smoke in the current image frame (e.g., denoted as the image frame at time t1). When real smoke is detected in an image frame, electronic device 110 can recall the current image frame (box 503) to form a sequence of image frames, which can be denoted as the real smoke sequence image frame at time t1.

[0095] Similarly, electronic device 110 can acquire a non-smoke video dataset 502 and input image frames of sample videos from the non-smoke video dataset 502 into the smoke target detection model 303 to detect whether the image frames input into the smoke target detection model 303 contain false smoke. When false smoke is detected in the current image frame (e.g., denoted as the image frame at time t2), electronic device 110 can also recall the current image frame (box 504) to form a sequence of image frames, which can be denoted as the false smoke sequence image frame at time t2. The real smoke sequence image frame at time t1 and the false smoke sequence image frame at time t2 can be used as sample image frames.

[0096] Continue to refer to Figure 4 In box 420, electronic device 110 determines from the video at least one reference sample image frame associated with the sample image frame (e.g., the real smoke sequence image frame at time t1 and the fake smoke sequence image frame at time t2).

[0097] In some embodiments, the electronic device 110 may determine a reference sample image frame based on the sample image frame, and determine at least one image frame from the sample video as at least one reference sample image frame according to a second time interval. The at least one reference sample image frame includes at least one image frame preceding the reference sample image frame and / or at least one image frame following the reference sample image frame.

[0098] Continue to refer to Figure 5 The electronic device 110 can use each image frame in the real smoke sequence image frame at time t1 as a reference sample image frame, and select reference sample image frames from the video at 3-second time intervals, thereby obtaining the corresponding real smoke sequence image frames at times t1-3s. Similarly, the fake smoke sequence image frames at times t2-3s can also be obtained. The real smoke sequence image frames at times t1-3s and the fake smoke sequence image frames at times t2-3s can be used as reference sample image frames.

[0099] In some embodiments, when determining the reference sample image frame, an image frame with a time offset from the sample video can also be determined as the reference sample image frame.

[0100] For example, electronic device 110 can further expand the number of image frames in the training dataset by adding a time offset to the video frame index. Figure 5 As shown, using i (which can indicate a time interval) as the time offset, the electronic device 110 can also determine the real smoke sequence image frame at time t1±i, the fake smoke sequence image frame at time t2±i, the real smoke sequence image frame at time t1-3s±i, and the fake smoke sequence image frame at time t2-3s±i from the video.

[0101] It should be noted that the real smoke sequence image frames at time t1±i can include real smoke sequence image frames at time t1+i and real smoke sequence image frames at time t1-i. The real smoke sequence image frames at time t1+i refer to the image frame sequence obtained by adding a time offset i to the index of each image frame in the real smoke sequence image frames at time t1. For example, if the real smoke sequence image frames at time t1 include frames 3, 5, and 7 in the sample video, adding a time offset i to the index of each image frame in the real smoke sequence image frames at time t1 yields frames 3+i, 5+i, and 7+i. Similarly, the real smoke sequence image frames at time t1-i refer to the image frame sequence obtained by subtracting a time offset i from the index of each image frame in the real smoke sequence image frames at time t1. For example, if the real smoke sequence image frames at time t1 include frames 3, 5, and 7 in the sample video, subtracting a time offset i to the index of each image frame in the real smoke sequence image frames at time t1 yields frames 3-i, 5-i, and 7-i. The image frames of the fake smoke sequence at time t2±i and time t2-3s±i are similar to the image frames of the real smoke sequence at time t1±i and time t1-3s±i, and will not be described again here.

[0102] In some embodiments, each image frame in the real smoke sequence image frame at time t1±i can be paired with the corresponding image frame in the real smoke sequence image frame at time t1-3s±i to facilitate subsequent difference operations. Similarly, each image frame in the fake smoke sequence image frame at time t2±i can also be paired with the corresponding image frame in the fake smoke sequence image frame at time t2-3s±i to facilitate subsequent difference operations.

[0103] For example, an image frame pair can be represented as: time t1 + i-th image frame, time t1+3 + i-th image frame, i∈{0,1,2,3,4,…,fps*t_max}, where fps is the video frame rate and t_max is the maximum time difference span.

[0104] See also Figure 4 In box 430, electronic device 110 determines, based on the position of at least one reference object in the sample image frame, multiple sample image segments in each image frame containing at least one reference object. In box 440, electronic device 110 determines that the training dataset includes multiple sample image segments.

[0105] like Figure 5As shown, the electronic device 110 can, based on the real smoke coordinates or fake smoke coordinates output by the smoke and fire target detection model 303 (boxes 505 and 506), use image processing tools in boxes 507 and 508 to process the aforementioned sequence of image frames (e.g., cropping or extracting images of local regions from the image frames) to obtain an image segment sequence. Then, it uses frame difference to process the image segment sequence pairs to obtain the real smoke frame difference image segment sequence at time t1±i (box 509) and the fake smoke frame difference image segment sequence at time t2±i (box 510), thereby obtaining the training dataset 511. In this way, only a small amount of data is needed to obtain a large amount of high-quality training data, allowing for rapid fine-tuning to adapt to various business scenarios and reducing the complexity and cycle time of AI applications. In some embodiments, the electronic device 110 can also use recalled image frames that do not contain smoke and / or open flames as training data for training the smoke and fire target detection model 303.

[0106] Figure 6 A schematic diagram of a process 600 for model training according to some embodiments of the present disclosure is shown. For example, process 600 may be implemented in... Figure 1 In electronic device 110. Next, in conjunction with... Figure 1 The process 600 is described in detail.

[0107] like Figure 6 As shown, the electronic device 110 can generate a real smoke-fake smoke / real fire-fake fire frame difference dataset 602 from real smoke-fake smoke / real fire-fake fire video 601, and the generation method can refer to the process 400 described above. When training the smoke sequence determination model 304 and the open flame sequence determination model 305, the training dataset 511 can be used for pre-training first (box 603), and then the real smoke-fake smoke / real fire-fake fire frame difference dataset 602 can be used to fine-tune the pre-trained smoke sequence determination model and the pre-trained open flame sequence determination model (box 604), thereby obtaining the smoke sequence determination model and the open flame sequence determination model to achieve higher recognition accuracy.

[0108] In this way, by using the smoke target detection model, the smoke sequence determination model, and the open flame sequence determination model, accurate identification of smoke and open flame can be achieved.

[0109] Compared to traditional methods, detecting smoke and fire using image fragments with frame differences offers stronger resistance to noise interference. It can also handle common problems in video surveillance scenarios such as video flicker, rain, snow, dust, and strong noise interference, achieving high-precision smoke and fire recognition even in low-bitrate video surveillance environments. Furthermore, through target detection and image classification, it can achieve smoke and fire recognition accuracy higher than traditional 3D video detection, while requiring lower computing power, memory, and bandwidth, making it suitable for various cloud-edge-device scenarios.

[0110] Figure 7 A block diagram of an apparatus 700 for detecting an object according to some embodiments of the present disclosure is shown. The apparatus 700 may be implemented as or included in... Figure 1 In the electronic device 110. The various modules / components in the device 700 can be implemented by hardware, software, firmware, or any combination thereof.

[0111] As shown in the figure, the apparatus 700 includes a first determining module 710 configured to determine candidate image frames potentially containing at least one object from a video. The apparatus 700 also includes a second determining module 720 configured to determine at least one reference image frame associated with the candidate image frames from the video. The apparatus 700 further includes a third determining module 730 configured to determine multiple image segments potentially containing at least one object in each of the candidate image frames and at least one reference image frame, based on the potential location of at least one object in the candidate image frames. The apparatus 700 also includes a detection module 740 configured to detect at least one object in the candidate image frames based on at least one differential image segment between the multiple image segments.

[0112] In some embodiments, the video may be a compressed video.

[0113] In some embodiments, the second determining module 620 may also be configured to determine at least one image frame from the video as at least one reference image frame at a time interval, based on the candidate image frame, wherein the at least one reference image frame includes at least one image frame located before the candidate image frame and / or at least one image frame located after the candidate image frame.

[0114] In some embodiments, at least one differential image segment may include a differential image segment between each image segment in at least one reference image frame and an image segment in a candidate image frame.

[0115] In some embodiments, the apparatus 600 may further include a fourth determining module configured to determine the potential location of an object in a candidate image frame using a first machine learning model.

[0116] In some embodiments, object detection in candidate image frames may be performed using a second machine learning model.

[0117] In some embodiments, the detection module 640 may also be configured to detect features of a first type of object based on at least one differential image segment using a first sub-model of the second machine learning model; and to detect features of a second type of object based on at least one differential image segment using a second sub-model of the second machine learning model.

[0118] In some embodiments, the second machine learning model may be trained based on a training dataset. In some embodiments, the apparatus 600 may also include a sample generation module configured to obtain the training dataset as follows: determine sample image frames from a sample video that contain at least one reference object associated with at least one object; determine at least one reference sample image frame associated with the sample image frame from the video; determine, based on the position of at least one reference object in the sample image frame, a plurality of sample image segments in each image frame of the sample image frame and the at least one reference sample image frame that contain at least one reference object; and determine that the training dataset includes a plurality of sample image segments.

[0119] In some embodiments, the sample generation module may also be configured to: determine a reference sample image frame based on the sample image frame; and determine at least one image frame from the sample video as at least one reference sample image frame according to a second time interval, wherein the at least one reference sample image frame includes at least one image frame located before the reference sample image frame and / or at least one image frame located after the reference sample image frame.

[0120] In some embodiments, the sample generation module may also be configured to: determine from the sample video an image frame that has a time offset from the sample image frame as a reference sample image frame.

[0121] In some embodiments, the apparatus 700 may further include an alarm module configured to trigger an alarm notification in response to the detection of an object in a candidate image frame.

[0122] Figure 8 A block diagram is shown illustrating a computing device 800 in which one or more embodiments of the present disclosure may be implemented. It should be understood that... Figure 8 The computing device 800 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 8 The computing device 800 shown can be used to implement Figure 1 Electronic devices 110.

[0123] like Figure 8 As shown, computing device 800 is in the form of a general-purpose computing device. Components of computing device 800 may include, but are not limited to, one or more processors or processing units 810, memory 820, storage devices 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. Processing unit 810 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 820. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 800.

[0124] Computing device 800 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to computing device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 820 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 830 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within computing device 800.

[0125] The computing device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 8 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 820 may include computer program product 825 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0126] The communication unit 840 enables communication with other computing devices via a communication medium. Additionally, the components of the computing device 800 can function as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 800 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0127] Input device 850 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 860 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 800 can also communicate as needed with one or more external devices (not shown) via communication unit 840. These external devices, such as storage devices, display devices, etc., can communicate with one or more devices that enable user interaction with computing device 800, or with any device (e.g., network card, modem, etc.) that enables computing device 800 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interfaces (not shown).

[0128] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0129] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0130] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0131] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0133] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for detecting an object, comprising: Identify candidate image frames from a video that potentially contain at least one object; Determine at least one reference image frame from the video that is associated with the candidate image frame; Based on the potential location of the at least one object in the candidate image frame, determine multiple image segments in the candidate image frame and each image frame of the at least one reference image frame that potentially contain the at least one object; as well as The at least one object is detected in the candidate image frame based on at least one differential image segment between the plurality of image segments.

2. The method according to claim 1, wherein the video is a compressed video.

3. The method of claim 1, wherein determining the at least one reference image frame comprises: Based on the candidate image frame, at least one image frame is determined from the video as the at least one reference image frame according to a first time interval, wherein the at least one reference image frame includes at least one image frame located before the candidate image frame and / or at least one image frame located after the candidate image frame.

4. The method of claim 1, wherein the at least one differential image segment comprises a differential image segment between each of the image segments in the at least one reference image frame and the image segments in the candidate image frame.

5. The method according to claim 1, further comprising: Using a first machine learning model, the potential location of the at least one object in the candidate image frame is determined.

6. The method of claim 1, wherein detecting the at least one object in the candidate image frame is performed using a second machine learning model.

7. The method of claim 6, wherein detecting the at least one object in the candidate image frame comprises: Using the first sub-model of the second machine learning model, a first object is detected based on the at least one differential image segment; as well as A second object is detected based on the at least one differential image fragment using a second sub-model of the second machine learning model.

8. The method of claim 6, wherein the second machine learning model is trained based on a training dataset, wherein the training dataset is obtained as follows: Determine sample image frames from the sample video that contain at least one reference object associated with the at least one object; Determine at least one reference sample image frame from the video that is associated with the sample image frame; Based on the position of the at least one reference object in the sample image frame, it is determined that the sample image frame and each image frame in the at least one reference sample image frame contain multiple sample image segments containing the at least one reference object; as well as The training dataset is determined to include the plurality of sample image fragments.

9. The method of claim 8, wherein determining at least one reference sample image frame associated with the sample image frame comprises: A baseline sample image frame is determined based on the aforementioned sample image frame; At a second time interval, at least one image frame is determined from the sample video as the at least one reference sample image frame, wherein the at least one reference sample image frame includes at least one image frame located before the reference sample image frame and / or at least one image frame located after the reference sample image frame.

10. The method of claim 9, wherein determining the reference sample image frame comprises: The reference sample image frame is determined from the sample video and has a time offset from the sample image frame.

11. The method according to claim 1, further comprising: An alarm notification is triggered in response to the detection of the object in the candidate image frame.

12. An apparatus for detecting an object, comprising: The first determining module is configured to determine candidate image frames from the video that potentially contain at least one object; The second determining module is configured to determine at least one reference image frame associated with the candidate image frame from the video; The third determining module is configured to determine, based on the potential location of the at least one object in the candidate image frame, and in each image frame of the at least one reference image frame, multiple image segments that potentially contain the at least one object; as well as The detection module is configured to detect the at least one object in the candidate image frame based on at least one differential image segment between the plurality of image segments.

13. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 11 when executed by the at least one processing unit.

14. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 11.

15. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 11.

Images