Image processing method and device and target detection method and device

Abstract

The invention provides an image processing method and device and a target detection method and device. The image processing method comprises the following steps: acquiring an input image; determining a first similarity between the input image and a preset first background template; determining a background template for the input image based on the first similarity; a foreground image is extracted from the input image based on a background template for the input image. The target detection method comprises the following steps: acquiring a video comprising a plurality of input images; sequentially executing the image processing method for the plurality of input images; and performing target detection on the foreground image obtained through the image processing method by using a target detection model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision, and more specifically, to an image processing method and device and a target detection method and device. Background Art

[0002] With the development of computer vision technology, image processing based on computer vision technology is increasingly used to detect and identify images. However, related image processing methods usually cannot meet the requirements of high accuracy and low latency when detecting and identifying target objects. Therefore, a technology that can extract foreground images to improve the performance and effect of image processing based on computer vision technology is needed.

[0003] The above information is presented as background information only to assist in understanding the present disclosure. No determination or assertion has been made as to whether any of the above contents may be applicable as related art with respect to the present disclosure. Summary of the invention

[0004] Embodiments of the present disclosure provide an image processing method and apparatus and an object detection method and apparatus to at least solve the above-mentioned problems and / or disadvantages.

[0005] According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, comprising: acquiring an input image; determining a first similarity between the input image and a preset first background template; determining a background template for the input image based on the first similarity; and extracting a foreground image from the input image based on the background template for the input image.

[0006] Optionally, the step of determining the background template for the input image based on the first similarity includes: determining the first background template as the background template for the input image based on the first similarity being greater than or equal to a first threshold; and determining the background template for the input image based on the input image and the first background template based on the first similarity being less than the first threshold.

[0007] Optionally, based on the first similarity being less than a first threshold, the step of determining a background template for the input image according to the input image and the first background template includes: aligning the input image with the first background template to obtain an aligned second background template; determining a second similarity between the input image and the second background template; based on the second similarity being greater than or equal to a second threshold, updating the first background template according to the input image, and determining the updated first background template as the background template for the input image; and based on the second similarity being less than a second threshold, generating a background template for the input image according to the input image.

[0008] Optionally, based on the second similarity being greater than or equal to a second threshold, the step of updating the first background template according to the input image includes: segmenting the input image by using a segmentation model to obtain a background area of ​​the input image; and filling a part of the area to be filled included in the second background template based on the background area to obtain an updated first background template.

[0009] Optionally, based on the background area, filling a part of the area to be filled in the second background template to obtain an updated first background template includes: determining the overlapping area between the background area and the area to be filled in the second background template; based on the overlapping area in the input image, filling the overlapping area in the second background template; and determining the second background template with the overlapping area filled as the updated first background template.

[0010] Optionally, based on the second similarity being less than a second threshold, the step of generating a background template for the input image according to the input image includes: segmenting the input image by using a segmentation model to obtain a non-background area of ​​the input image; setting the non-background area in the input image as an area to be filled; and determining the input image with the area to be filled as the background template for the input image.

[0011] Optionally, the step of extracting a foreground image from the input image based on a background template for the input image includes: obtaining a difference map of the input image based on the input image and the background template; obtaining a mask image based on the value of each pixel in the difference map; and obtaining the foreground image based on the input image and the mask image.

[0012] Optionally, the background template for the input image includes a to-be-filled area for extracting a foreground image, wherein image features of the to-be-filled area are set based on a frequency of occurrence of image features and are different from image features of areas other than the to-be-filled area in the background template.

[0013] Optionally, the image processing method further includes: detecting whether a predetermined object exists in the input image by using a target detection model; in response to detecting the predetermined object in the input image, stopping image processing of the input image and reacquiring the input image.

[0014] According to a second aspect of an embodiment of the present disclosure, a target detection method is provided, which includes: acquiring a video including multiple input images; executing the image processing method as described above in sequence for the multiple input images; and performing target detection on the foreground image obtained by the image processing method by using a target detection model.

[0015] According to a third aspect of an embodiment of the present disclosure, an image processing device is provided, comprising: an image acquisition unit configured to acquire an input image; a similarity determination unit configured to determine a first similarity between the input image and a preset first background template; a template determination unit configured to determine a background template for the input image based on the first similarity; and a foreground extraction unit configured to extract a foreground image from the input image based on the background template for the input image.

[0016] Optionally, the template determination unit is configured to determine the background template for the input image based on the first similarity by the following operations: based on the first similarity being greater than or equal to a first threshold, determining the first background template as the background template for the input image; and based on the first similarity being less than the first threshold, determining the background template for the input image based on the input image and the first background template.

[0017] Optionally, the template determination unit is configured to determine a background template for the input image based on the input image and the first background template based on the first similarity being less than a first threshold through the following operations: aligning the input image with the first background template to obtain an aligned second background template; determining a second similarity between the input image and the second background template; updating the first background template based on the input image based on the second similarity being greater than or equal to a second threshold, and determining the updated first background template as the background template for the input image; and generating a background template for the input image based on the input image based on the input image based on the second similarity being less than a second threshold.

[0018] Optionally, the template determination unit is configured to update the first background template according to the input image based on the second similarity being greater than or equal to a second threshold through the following operations: segmenting the input image by using a segmentation model to obtain a background area of ​​the input image; and filling a part of the area to be filled included in the second background template based on the background area to obtain an updated first background template.

[0019] Optionally, the template determination unit is configured to fill a part of the area to be filled in the second background template based on the background area through the following operations to obtain an updated first background template: determine the overlapping area between the background area and the area to be filled in the second background template; fill the overlapping area in the second background template based on the overlapping area in the input image; and determine the second background template with the overlapping area filled as the updated first background template.

[0020] Optionally, the template determination unit is configured to generate a background template for the input image based on the input image based on the second similarity being less than a second threshold through the following operations: segmenting the input image by using a segmentation model to obtain a non-background area of ​​the input image; setting the non-background area in the input image as a to-be-filled area; and determining the input image with the to-be-filled area set as the background template for the input image.

[0021] Optionally, the foreground extraction unit is configured to extract a foreground image from the input image based on a background template for the input image by the following operations: obtaining a difference map of the input image based on the input image and the background template; obtaining a mask image based on the value of each pixel in the difference map; and obtaining the foreground image based on the input image and the mask image.

[0022] Optionally, the background template for the input image includes a to-be-filled area for extracting a foreground image, wherein image features of the to-be-filled area are set based on a frequency of occurrence of image features and are different from image features of areas other than the to-be-filled area in the background template.

[0023] Optionally, the image processing device also includes: an object detection unit, configured to: detect whether a predetermined object exists in the input image by using a target detection model; in response to detecting the predetermined object in the input image, stop image processing of the input image, and reacquire the input image.

[0024] According to a fourth aspect of an embodiment of the present disclosure, a target detection device is provided, comprising: a video acquisition unit, configured to acquire a video comprising multiple input images; a foreground processing unit, configured to sequentially execute the image processing method as described above for the multiple input images; and a target detection unit, configured to perform target detection on the foreground image obtained by the image processing method by using a target detection model.

[0025] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein the computer executable instructions, when executed by the at least one processor, cause the at least one processor to execute the method described above.

[0026] According to a sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein, when instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the method described above.

[0027] According to the image processing method and the image processing device of the embodiment of the present disclosure, the foreground image that matches the input image can be flexibly extracted based on the input image and the preset background template, and the background template with higher accuracy can also be provided for the image processing of other images. For various scenes with diversity and complexity, the image processing method and the image processing device can generate and maintain the background template, reduce the interference of the real-time background on the foreground image extraction, and effectively improve the extraction accuracy.

[0028] The image processing method and the image processing device according to the embodiments of the present disclosure can also timely identify predetermined objects involving security issues, and avoid security issues caused by the predetermined objects in the input image. For example, the image processing method and the image processing device according to the embodiments of the present disclosure can be used for anti-sneak photography or security detection.

[0029] According to the target detection method and target detection device of the embodiment of the present disclosure, the image processing for accurately extracting the foreground image can be combined with the target detection, and the target detection with high recall rate and high precision can be performed while removing the background, which can achieve a significant improvement in performance on the basis of introducing less delay, and can also achieve dual detection of predetermined objects, and correspondingly improve safety performance. The image processing method for performing foreground image extraction in the target detection method and target detection device according to the embodiment of the present disclosure can serve as a bridge for the target detection method, so that the target detection method and target detection device have the detection capability of large-scale models.

[0030] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other aspects, features and elements of certain embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings, in which:

[0032] Figure 1 is a flowchart illustrating an image processing method according to an embodiment of the present disclosure;

[0033] Figure 2 is a flowchart illustrating an example process of an image processing method according to an embodiment of the present disclosure;

[0034] Figure 3 is a flowchart illustrating an example process of updating a background template according to an embodiment of the present disclosure;

[0035] Figure 4 is a flowchart illustrating an example process of generating a background template according to an embodiment of the present disclosure;

[0036] Figure 5is a flowchart illustrating an example process of extracting a foreground image according to an embodiment of the present disclosure;

[0037] Figure 6 is a flow chart showing a target detection method according to an embodiment of the present disclosure;

[0038] Figure 7 is a block diagram showing a structure of an image processing apparatus according to an embodiment of the present disclosure;

[0039] Figure 8 is a structural block diagram showing a target detection device according to an embodiment of the present disclosure; and

[0040] Fig. 9 is a structural block diagram showing an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0043] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step 1 and step 2" which means the following three parallel situations: (1) performing step 1; (2) performing step 2; (3) performing step 1 and step 2.

[0044] In the related art, in order to process a video or an image, a single general model is usually used to perform image processing operations on a target object. For example, a general target detection model is usually used to perform detection and recognition of a target object. However, for image processing using a single general model (such as a general target detection model) for multiple images, the high precision of the model calculation results and the low latency during the use of the model are usually not available at the same time. Specifically, when a large-scale model is used for image processing, although the detection results are more accurate, the technology of using a large-scale model to process multiple input images often has high latency and requires a lot of computing resources and storage space. When a lightweight model is used, although the technology using a lightweight model is more friendly to computing resources and storage space and can reduce latency, its detection or recognition results are usually less accurate.

[0045] Taking into account at least the above-mentioned problems existing in the related art, the present disclosure proposes such an image processing method and device: determine a background template suitable for the input video or image, and perform foreground extraction based on the background template to obtain a foreground image containing the target object. Such foreground extraction processing can achieve an optimal balance between the accuracy and delay of image processing with respect to the target object. In addition, the present disclosure also proposes a target detection method and device based on the above-mentioned image processing: since the foreground image containing the target object has been obtained based on the above-mentioned image processing method, only a lightweight target detection model can be used to perform target detection on the foreground image obtained based on the above-mentioned image processing, thereby obtaining accurate target detection results without using a large-scale model. The following will refer to Figures 1 to 9 An image processing method and apparatus and an object detection method and apparatus according to an embodiment of the present disclosure are described in detail.

[0046] First, some terms involved in this disclosure are explained in detail.

[0047] In this article, the foreground (also referred to as the foreground part or the foreground area) may include the part of the image associated with the target object, and the background (also referred to as the background part or the background area) may include the part of the image that is not related to the target object or the image analysis process.

[0048] Herein, the foreground image may refer to an image including a foreground and a region to be filled obtained by performing foreground extraction on an image. Here, foreground extraction may refer to a process of dividing the foreground from the background in an image or separating the foreground from the image.

[0049] In this article, the object included in the foreground can be moving or stationary, and accordingly, the object included in the background can also be moving or stationary. That is to say, the distinction between the foreground and the background is independent of the motion state of one or more objects in the image, and the foreground and the background are relatively independent. For example, in an indoor scene, when the target object is both a person and a camera, the foreground can be the part of the captured video or image that includes a moving person and a fixed camera. For another example, in a highway scene, when the target object is a car, the foreground can be the part of the captured video or image that includes a moving car and a stopped car.

[0050] In this article, large-scale (object detection) models include models that use relatively large-scale model parameters, and have the characteristics of using more computing resources and storage space, high latency and high accuracy.

[0051] In this paper, lightweight (target detection) models include models that use relatively small-scale model parameters, which have the characteristics of being more storage and computationally friendly, less latency, and lower precision.

[0052] According to the embodiments, although some terms of the present disclosure are defined in the above content, it should be understood that the above terms may also include any other applicable meanings commonly understood by ordinary technicians in the field.

[0053] The image processing method according to the embodiment of the present disclosure is described in detail below. Figure 1 is a flowchart illustrating an image processing method according to an embodiment of the present disclosure. Figure 2 is a flowchart illustrating an example process of an image processing method according to an embodiment of the present disclosure.

[0054] Reference Figure 1 In step S110, an input image is obtained, for example, referring to Figure 2 , obtain the input image containing characters and background in an indoor scene.

[0055] According to an embodiment of the present disclosure, methods of acquiring an input image may include receiving an input image (such as receiving image data), capturing an input image (capturing an input image using a video or image capture device (such as a camera)), reading an input image (such as reading an input image from a memory), etc., but are not limited thereto.

[0056] According to an embodiment of the present disclosure, when a video is acquired, at least one input image may be acquired from the video at a predetermined interval, for example, at least one video frame may be determined from a plurality of video frames of the video at intervals of a predetermined number of frames, and each of the at least one video frame may be determined as an input image. According to an embodiment of the present disclosure, when at least one image is acquired, each of the at least one image may be determined as an input image.

[0057] In step S120, a first similarity between the input image and a preset first background template is determined. Here, the input image and the first background template may have the same or substantially the same size.

[0058] In the following, the background template is first described. Here, the details described for the background template are applicable to other background templates (for example, the first background template, the second background template, etc.). That is to say, the background templates in the embodiments of the present disclosure have the same or similar functions or features, and the present disclosure is not limited to this.

[0059] According to an embodiment of the present disclosure, a preset background template (e.g., a first background template) for extracting a foreground image in an image processing situation may be preset. According to an embodiment, a background template determined based on experience or tests conducted in various scenarios may be preset as a preset background template.

[0060] For example, the scene to which one or more input images belong may be identified and a preset background template may be obtained based on the identified scene. Figure 2 , identifiable Figure 2 The scene of the input image is an indoor scene, and a preset scene template corresponding to the indoor scene can be determined.

[0061] According to an embodiment, an image including a predetermined image feature may be pre-set as a preset background template. According to an embodiment, the image feature may include at least one of a color feature, a texture feature, and a shape feature, but is not limited thereto.

[0062] For example, an image containing a predetermined color (e.g., a solid color image having only a single color) may be pre-set as a preset background template. For another example, an image containing a predetermined texture (e.g., a striped image having only regular stripes) may be pre-set as a preset background template. For another example, an image containing a predetermined shape outline (e.g., an image having a human outline) may be pre-set as a preset background template.

[0063] In addition, according to an embodiment of the present disclosure, the background template may include an area to be filled for extracting a foreground image, wherein image features of the area to be filled are set based on the frequency of occurrence of image features and are different from image features of areas other than the area to be filled in the background template.

[0064] For example, when the set image feature is a color feature, the image feature occurrence frequency can be the color occurrence frequency. Therefore, the color of the area to be filled in the background template may include a color determined based on the color occurrence frequency (such as a color that appears less frequently or infrequently in nature), and this color is different from the colors in the remaining areas of the background template.

[0065] For example, in nature or in common scenes, there are few objects with pure purple or colors close to purple. Therefore, errors in image processing caused by the presence of the same or similar color as the area to be filled in the input image and / or the same color as the rest of the area in the background template can be avoided by, for example, setting the area to be filled in the background template to a color with a low frequency of occurrence (such as purple).

[0066] Here, although the set color is determined by considering the frequency of occurrence of colors in nature or general scenes, the set color can also be a color determined based on other appropriate methods, as long as the color can distinguish the color of the area to be filled from the color in the area other than the area to be filled in the background template and / or the color in the input image. For example, the set color can be a color determined based on the colors included in one or more input images (for example, when the input image includes red, the set color should not include red or a color that is substantially close to red). For another example, the set color can be a color determined based on the scene of the identified input image, etc. (for example, when the scene of the input image is an outdoor scene at night, the set color should not include black or a color that is substantially close to black, and can be white, for example).

[0067] For another example, when the set image feature is a texture feature, the image feature occurrence frequency may be the texture occurrence frequency. Therefore, the texture of the area to be filled in the background template may include a texture determined based on the texture occurrence frequency (such as a texture that rarely appears or appears infrequently in nature (such as a regular linear texture, a regular point texture, etc.)), and the texture is different from the textures in the remaining areas of the background template.

[0068] Here, although the texture is determined by considering the frequency of occurrence of textures in nature or general scenes, the set texture can also be a color determined based on other appropriate methods, as long as the texture can distinguish the texture of the area to be filled from the texture in the area other than the area to be filled in the background template and / or the texture in the input image. For example, the set texture can be a texture determined based on the texture included in one or more input images (for example, when the input image includes a wallpaper pattern, the set texture should not include textures related to the wallpaper pattern). For another example, the set texture can be a texture determined based on the scene of the identified input image, etc. (for example, when the scene of the input image is an indoor scene, the set texture should not include textures related to floor patterns, wallpaper patterns, etc.).

[0069] For another example, when the set image feature is a shape feature, the frequency of occurrence of the image feature may be the frequency of occurrence of a shape outline. Therefore, the shape outline of the area to be filled in the background template may include a shape outline determined based on the frequency of occurrence of the shape outline (for example, usually more target detection processes use people as detection targets, therefore, the human shape outline may be determined as the set shape outline), and the shape outline is different from the shape outlines in the remaining areas of the background template.

[0070] Here, although the set shape contour is determined based on the frequency of occurrence of shape contours in general scenarios, the set shape contour can also be a shape contour determined based on other appropriate methods, as long as the shape contour can distinguish the shape contour of the area to be filled from the shape contours in the background template other than the area to be filled and / or the shape contours in the input image.

[0071] In addition, although the above description only describes in detail a single image feature, according to an embodiment of the present disclosure, the set image feature may include a color feature, a texture feature, or a shape feature, or a combination thereof. For example, the area to be filled in the background template may have a set shape outline and color, for example, referring to Figure 2 , the area to be filled in the first background template is shown as a human outline and gray.

[0072] The foreground image can be easily extracted from the input image by using a background template having a to-be-filled area with set image features. Figure 2 A detailed description is given on the use of background templates.

[0073] According to an embodiment of the present disclosure, the first similarity may represent the image similarity between the input image and the first background template. According to an embodiment, the metric used to measure image similarity may include, but is not limited to, structural similarity (SSIM), root mean squared error (RMSE), mean squared error (MSE), universal quality index (UQI), scale invariant feature transform (SIFT), deep learning algorithm, etc.

[0074] Reference Figure 2According to an embodiment of the present disclosure, a first similarity corresponding to SSIM may be determined, and the determined first similarity may be compared with a first threshold to determine a background template for the input image from the first background template. Here, the first similarity may also be a similarity determined in the above or any other measurement method. In order to avoid unnecessary details from obscuring the present disclosure, the specific details of the process of determining the similarities corresponding to various measurement methods are not described in detail. The process of determining the similarity of the present disclosure may be implemented in any appropriate manner.

[0075] Return to reference Figure 1 In step S130, a background template for the input image is determined based on the first similarity. Figure 2 , by comparing the first similarity with the first threshold, it is determined whether to directly use the first background template, or to update or generate the background template for the input image based on the input image and / or the first background template.

[0076] According to an embodiment of the present disclosure, the first threshold may be a preset empirical value, and when the similarity is greater than or equal to the first threshold, the similarity may be considered high, and when the similarity is less than the first threshold, the similarity may be considered low. For example, the first threshold may be set to 0.65.

[0077] According to an embodiment of the present disclosure, based on the first similarity being greater than or equal to the first threshold, the first background template is determined as the background template for the input image. Specifically, when the first similarity is high, it can be considered that the input image conforms to the first background template or the first background template is applicable to the input image, and therefore, the first background template can be directly used as the background template for the input image.

[0078] According to an embodiment of the present disclosure, based on the first similarity being less than the first threshold, a background template for the input image is determined based on the input image and the first background template. Specifically, when the first similarity is less than the first threshold, although the first background template may not be used directly to extract the foreground image, it is considered that there may be deviations or interference when acquiring the input image (such as capturing the image), which causes the current first background template to be unsuitable for the input image, while in fact the first background template may be applicable to the input image after minor adjustments. In this case, in order to save computing resources and time (for example, avoiding unnecessary new background template generation steps), it is further determined whether to update the first background template to determine the background template for the input image, or to generate a new background template for the input image.

[0079] For example, refer to Figure 2 , a slight adjustment can be achieved by alignment processing. According to an embodiment of the present disclosure, based on the first similarity being less than a first threshold, the input image is aligned with the first background template to obtain an aligned second background template.

[0080] According to an embodiment of the present disclosure, the step of aligning the input image with the first background template may include: aligning the input image with the first background template based on local features of the input image and local features of the first background template so that the local features of the two match.

[0081] According to an embodiment of the present disclosure, a local feature may represent a feature that can be detected at different scales and / or rotation angles, such as but not limited to corner points, edges, and any other features composed of feature descriptors of key points in an image and their related areas. For example, according to an embodiment, a plurality of corner points having a corresponding relationship may be determined in the input image and the first background template, respectively, and the input image and the first background template may be aligned so that the plurality of corner points having a corresponding relationship overlap, thereby achieving the same viewing angle for the input image and the aligned second background template. In addition, other local features may be aligned in a manner similar to that of aligning corner points, so that the first background template is corrected to be a corrected background template (i.e., the second background template) having the same viewing angle as the input image.

[0082] After the alignment process is performed, the determination step based on the comparison of the similarity with the threshold value needs to be used again. According to an embodiment of the present disclosure, a second similarity between the input image and the second background template is determined. Here, the process of determining the second similarity is the same as the details of the process of determining the first similarity described above, and is not described in detail here. In addition, the measurement method of the second similarity may be the same as or different from the measurement method of the first similarity, that is, the SSIM method may be used again to determine the second similarity corresponding to the SSIM, or other methods may be used to determine the second similarity different from the first similarity type.

[0083] According to an embodiment of the present disclosure, based on the second similarity being greater than or equal to the second threshold, the first background template is updated according to the input image, and the updated first background template is determined as the background template for the input image. That is, when the second similarity is high, it can be indicated that the background template alignment process is correct, and the background template needs to be updated and the updated background template is used. Figure 3 The process of updating the background template is described in detail.

[0084] Figure 3 is a flow chart illustrating an example process of updating a background template according to an embodiment of the present disclosure.

[0085] According to an embodiment of the present disclosure, the background region of the input image is obtained by segmenting the input image using a segmentation model (such as an instance segmentation model).

[0086] For example, an instance segmentation model may be used to determine from an input image the regions in the input image that correspond to a target object (e.g., Figure 3 The method further comprises dividing the target object region (or mask region) associated with the person in the input image in the image and the background region unrelated to the target object, and determining the corresponding mask image based on the divided target object region and background region, such as, Figure 3 The black area in the mask image represents the background area, and the white area represents the target object area.

[0087] According to an embodiment of the present disclosure, when a background area in an input image is determined, a portion of the area to be filled included in the second background template is filled based on the background area to obtain an updated first background template. That is, the step of updating the first background template includes aligning the first background template to the input image, and also includes further updating the first background template after alignment.

[0088] According to an embodiment of the present disclosure, the step of filling a part of the area to be filled included in the second background template based on the background area to obtain an updated first background template includes: determining the overlapping area between the background area and the area to be filled in the second background template; filling the overlapping area in the second background template based on the overlapping area in the background area; and determining the second background template with the overlapping area filled as the updated first background template.

[0089] For example, refer to Figure 3 , the background area of ​​the mask image and the current background template ( Figure 3 The area to be filled in the second background template determined based on the first background template) (ie, Figure 3 The overlapping area of ​​the background template in Figure 3 The overlapping area in the background template is filled based on the overlapping area in the input image, and an updated background template filled with the overlapping area can be obtained.

[0090] According to an embodiment of the present disclosure, the step of filling the overlapping area of ​​the first background template based on the overlapping area in the background area may include: filling the overlapping area in the second background template based on the image features (such as color features, texture features and / or shape features) of the overlapping area in the input image. For example, the overlapping area in the second background template is directly filled with the partial image corresponding to the overlapping area in the input image. For another example, the overlapping area in the second background template is filled based on the partial image features of the image corresponding to the overlapping area in the input image. For another example, the image of the partial image corresponding to the overlapping area in the input image that has been image processed (for example, image enhancement processing, image compression processing, etc.) is filled into the overlapping area in the second background template.

[0091] In addition, although the present disclosure shows that the update of the second background template is performed based on the second similarity being less than the second threshold, the present disclosure is not limited to this. The update of the first background template may also be performed directly based on the input image (for example, based on user input), or the update of the first background template and / or the second background template may be performed according to predetermined rules (for example, periodically), and the details of the processing for the update of the second background template as above may be applicable to the update of the background template in any case.

[0092] In addition, in the disclosed embodiment, the second threshold may be a preset empirical value, and when the similarity is greater than or equal to the second threshold, the similarity may be considered high, and when the similarity is less than the second threshold, the similarity may be considered low. For example, the second threshold may be set to 0.35.

[0093] Return to reference Figure 2 According to an embodiment of the present disclosure, based on the second similarity being less than the second threshold, a background template for the input image is generated according to the input image. That is, when the second similarity is low, it may indicate that the first background template is completely unavailable or the alignment process fails, and a new background template needs to be regenerated.

[0094] Figure 4 is a flow chart illustrating an example process of generating a background template according to an embodiment of the present disclosure.

[0095] According to an embodiment of the present disclosure, the background region of the input image is obtained by segmenting the input image using a segmentation model (such as an instance segmentation model).

[0096] For example, an instance segmentation model may be used to determine, from an input image, a target object region associated with the target object and a background region unrelated to the target object in the input image.

[0097] According to an embodiment of the present disclosure, when the target object area in the input image is determined, the target object area in the input image is set as the area to be filled (mask area), and the input image with the area to be filled is determined as the background template for the input image. Figure 4 , the target object area associated with the person in the input image is set as the to-be-filled area having the image features set as described above, thereby obtaining a background template.

[0098] According to an embodiment of the present disclosure, the obtained background template can be applied to the input image and / or other images associated with the input image in at least one image. In other words, since multiple input images included in a video may have the same or similar background, the same background template can be applied to a group of similar input images to save storage and computing resources.

[0099] Through the various operations described above, a background template with higher accuracy for the input image can be determined based on the input image and / or a preset background template, and then a foreground image can be accurately extracted from the input image by using the determined background template.

[0100] Return to reference Figure 1 In step S140, a foreground image is extracted from the input image based on a background template for the input image. Figure 5 The step S140 of extracting the foreground image is described in detail.

[0101] Figure 5 is a flowchart illustrating an example process of extracting a foreground image according to an embodiment of the present disclosure.

[0102] According to the embodiments of the present disclosure, referring to Figure 5 Based on the input image and the background template, a difference map of the input image can be obtained. Specifically, for example, the difference map can be obtained by subtracting the background template for the input image from the input image.

[0103] According to an embodiment, for each pixel in the input image and the background template, the difference of each pixel in the difference map can be obtained by subtracting the grayscale values ​​of the corresponding two pixels. According to an embodiment, for each pixel in the input image and the background template, whether the two pixels are similar can be determined by calculating the cosine similarity of the corresponding two pixels, and the similarity is converted into the difference of each pixel in the difference map. According to an embodiment, the method for measuring the difference between the input image and the background template may include but is not limited to other similarity calculation methods, image comparison methods, etc.

[0104] According to an embodiment of the present disclosure, after the difference map is obtained, a mask image is obtained based on the value of each pixel (ie, the difference) in the difference map.

[0105] Specifically, for each pixel in the difference map, a value is assigned to each pixel in the mask image based on a comparison result between the pixel value (ie, the difference) and the third threshold value, so as to obtain an initial mask image.

[0106] For example, based on the pixel value being greater than or equal to the third threshold, it is determined that the similarity between the pixel in the input image and the pixel at the same position in the background template is low, and the value of the pixel at the corresponding position in the mask image is set to 1 (i.e., the corresponding grayscale value is 255). Figure 5 The pixels of the white area in the mask image in are assigned a value of 1. According to an embodiment, the area where the pixel value is 1 may be considered as an area of ​​interest.

[0107] For example, based on the value of the pixel being less than the third threshold, it is determined that the similarity between the pixel in the input image and the pixel at the same position in the background template is high, and the value of the pixel at the corresponding position in the mask image (i.e., the grayscale value) is set to 0 (i.e., the corresponding grayscale value is 0), for example, Figure 5 The pixels in the black area of ​​the mask image are assigned a value of 0.

[0108] Therefore, by respectively assigning values ​​to a plurality of pixels of the mask image based on the difference values, an initial mask image can be obtained.

[0109] According to an embodiment of the present disclosure, the third threshold value may be a preset empirical value, and when the difference value is greater than or equal to the third threshold value, the similarity of the pixels may be considered to be low, and when the difference value is less than the third threshold value, the similarity of the pixels may be considered to be high. For example, the third threshold value may be set to 20.

[0110] However, the obtained initial mask image may have small holes due to inevitable minor errors during image processing. In this regard, according to an embodiment, morphological processing may also be performed on the initial mask image to obtain a mask image with the holes filled. Specifically, the holes in the initial mask image may be filled by corrosion and / or dilation processing in the morphological processing. Here, the details of the morphological processing are not described in detail, and other image quality optimization processing for the image may also be performed, but the present disclosure is not limited thereto.

[0111] Through morphological processing, some tiny holes in the mask image can be eliminated, avoiding tiny errors that deteriorate the image quality.

[0112] According to an embodiment of the present disclosure, after obtaining the mask image, referring to Figure 5 , based on the input image and the mask image, a foreground image is obtained. According to an embodiment, the foreground image can be obtained by multiplying the pixel matrix of the input image with the pixel matrix of the mask image. Specifically, since the value of the pixel in the region of interest in the mask image is 1, and the value of the pixel in other regions is 0, an image including only the region of interest, that is, the foreground image, can be obtained through the multiplication operation.

[0113] In addition, in order to achieve target detection, a large-scale model can also be used in the image processing method. According to an embodiment of the present disclosure, the image processing method may also include: detecting whether a predetermined object (e.g., another target object) is present in the input image by using a target detection model (such as a large-scale target detection model); in response to detecting a predetermined object in the input image, stopping image processing for the input image, and reacquiring the input image and executing the above-mentioned image processing method. That is, a large-scale model can be used to perform target detection for a predetermined object to detect whether a trigger condition for stopping image processing is met. For example, when a predetermined object (such as a camera) is detected, there may be a risk of leakage of current environmental information, and the image processing may be terminated and the user may be prompted to reacquire the input image. In this case, the user can choose to remove the predetermined object and re-execute the image processing method.

[0114] Here, for information security and to reduce the computational burden brought by large-scale models, before executing the image processing method as described above, target detection can be performed on the input image based on a predetermined rule (for example, every predetermined time interval, every predetermined number of input images) or in response to user input, without having to perform target detection on all input images. Specifically, in practical applications, computer vision technology can be used to determine whether a predetermined object exists in the environment, for example, whether there is a camera or other security-related object. Accurate detection of the predetermined object using a large-scale target detection model can ensure environmental security. Therefore, in the image processing method, a large-scale model is used to detect whether a predetermined object exists. Once a predetermined object involving security issues is detected, image processing can be stopped, which can accurately identify the predetermined object and stop processing in time to avoid privacy leakage risks and save computing resources.

[0115] Here, although target detection using a large-scale target detection model is described as being performed before the image processing method, the present disclosure is not limited thereto, and target detection may also be performed while executing the image processing method or before executing the image processing method.

[0116] The foreground image of the input image can be accurately obtained by the image processing method as described above. The image processing method as described above can also be applied to the target detection method.

[0117] Figure 6 is a flowchart illustrating a target detection method according to an embodiment of the present disclosure.

[0118] According to an embodiment of the present disclosure, in step S610, a video including a plurality of input images is acquired. Specifically, the method of acquiring the video is similar to the method of acquiring the video described above with reference to step S110, and will not be described repeatedly here.

[0119] In step S620, for a plurality of input images, reference Figures 1 to 5 That is, for each of the multiple input images, the image processing method described above is performed to obtain the foreground image of each input image. Figure 6 The right block diagram of describes step S620.

[0120] In step S621, an input image is acquired.

[0121] In step S622, a first similarity between the input image and a preset background template is determined.

[0122] In step S623, it is determined whether the first similarity is greater than or equal to a first threshold.

[0123] When the first similarity is greater than or equal to the first threshold, in step S628, a preset background template is directly obtained as the background template for the input image.

[0124] When the first similarity is less than the first threshold, in step S624, the input image is aligned with a preset background template to determine a second similarity between the input image and the aligned background template.

[0125] In step S625 , it is determined whether the second similarity is greater than or equal to a second threshold.

[0126] When the second similarity is greater than or equal to the second threshold, in step S626, the aligned background template is updated, and in step S628, the updated background template is obtained as the background template for the input image.

[0127] When the second similarity is less than the second threshold, in step S627, a new background template is generated, and in step S628, the generated background template is obtained as the background template for the input image.

[0128] Step S620 corresponds to step S110, step S120 and step S130 as a whole, wherein the specific details of step S110 are applicable to step S621, the specific details of step S120 are applicable to step S622, and the specific details of step S130 are applicable to steps S623 to S628, which are not repeated here.

[0129] When the background template is obtained, in step S629, the foreground image is extracted from the input image using the background template, wherein the foreground image can be used for subsequent target detection. Step S629 corresponds to step S140, and the specific details of step S140 are applicable to step S629, which will not be described repeatedly here.

[0130] When the foreground image is obtained in step S620, object detection may be performed on the extracted foreground image. The foreground image obtained using the image processing method may be used as an input to an object detection model in the object detection step.

[0131] In step S630, target detection is performed on the foreground image obtained by the image processing method in step S620 by using a target detection model (e.g., a lightweight target detection model). Specifically, target detection on the target object is performed in the input image using the target detection model. For example, in an embodiment of the present disclosure, the target object may be a person, and a foreground image substantially containing the person may be obtained by step S620. In step S630, target detection is performed on the person in the foreground image to determine the final target detection result.

[0132] According to an embodiment, the lightweight target detection model may include any type of target detection model, such as but not limited to, a faster region-based convolutional neural network (Faster Region-based Convolutional Neural Network, Faster-RCNN) model, a single shot multi-box detection (Single Shot MultiBox Detector, SSD) model, a RetinaNet model, a You Only Look Once (YOLO) model, and other target detection models.

[0133] Here, since the foreground image containing the target object has been obtained using image processing, the demand for the model capability for target detection of the foreground image is greatly reduced. Therefore, only a lightweight target detection model can be used to implement target detection during target detection processing. According to an embodiment of the present disclosure, since useless redundant background information has been removed, the expected detection accuracy can be obtained by performing target detection using a low-latency and resource-saving lightweight target detection model.

[0134] According to an embodiment of the present disclosure, in addition to detecting a predetermined object in the image processing step of step S620, the predetermined object (e.g., another target object) may be detected again in the target detection step of step S630, thereby achieving double detection of the triggering condition for stopping image processing (i.e., determining whether the predetermined object exists) and reducing security risks. For example, a lightweight target detection model may be used to perform target detection again on the predetermined object (i.e., another target object) in the input image to determine again whether to stop image processing.

[0135] For example, in an embodiment of the present disclosure, the predetermined object may be a camera, and the target object may be a person. When the camera is not detected through step S620 and a foreground image substantially containing the person is obtained, target detection is performed on the person in the foreground image in step S630 to determine the final target detection result, and a lightweight target detection model may also be used to perform target detection on the camera in the input image to determine again whether to stop image processing.

[0136] According to an embodiment, a large-scale model may be used in an image processing method to perform target detection of a predetermined object (i.e., another target object) to determine whether a trigger condition for stopping image processing is met (i.e., determining whether a predetermined object exists), and a lightweight model may be used again in a target detection method to perform target detection to obtain a target detection result of the target object. By combining such a target detection method of a large-scale model and a lightweight model, the accuracy of target detection can be greatly improved while introducing a small amount of delay.

[0137] Based on the results of testing the target detection method of the present invention, the detection accuracy of the target detection method according to the embodiment of the present invention can be close to the detection accuracy of the large-scale model in the related target detection technology, and the delay is close to the delay of the lightweight model in the related target detection technology. Therefore, the best balance between accuracy and delay can be achieved, which can greatly improve the accuracy while bringing less delay increase.

[0138] An image processing device that executes the above image processing method and an object detection device that executes the above object detection method will be described in detail below.

[0139] Figure 7 Detailed description is a block diagram showing the structure of an image processing apparatus according to an embodiment of the present disclosure.

[0140] According to an embodiment of the present disclosure, the image processing apparatus 700 may include an image acquiring unit 710 , a similarity determining unit 720 , a template determining unit 730 , and a foreground extracting unit 740 .

[0141] Specifically, the image acquisition unit 710 is configured to acquire an input image, the similarity determination unit 720 is configured to determine a first similarity between the input image and a preset first background template, the template determination unit 730 is configured to determine a background template for the input image based on the first similarity, and the foreground extraction unit 740 is configured to extract a foreground image from the input image based on the background template for the input image.

[0142] That is, the image acquisition unit 710 may perform the same Figure 1 The similarity determination unit 720 may perform the same operation as described above with reference to step S110 of the image processing method described above. Figure 1 and Figure 2 The template determination unit 730 may perform the operations corresponding to step S120 of the image processing method described above. Figures 1 to 4 The foreground extraction unit 740 may perform the same operation as described above with reference to step S130 of the image processing method described above. Figure 1 and Figure 5 The described image processing method corresponds to step S140.

[0143] According to an embodiment of the present disclosure, the template determination unit 730 may be configured to determine a background template for the input image based on the first similarity by the following operations: based on the first similarity being greater than or equal to a first threshold, determining the first background template as the background template for the input image; and based on the first similarity being less than the first threshold, determining the background template for the input image based on the input image and the first background template.

[0144] According to an embodiment of the present disclosure, the template determination unit 730 may be configured to determine a background template for the input image based on the input image and the first background template based on the first similarity being less than a first threshold through the following operations: aligning the input image with the first background template to obtain an aligned second background template; determining a second similarity between the input image and the second background template; updating the first background template based on the input image based on the second similarity being greater than or equal to a second threshold, and determining the updated first background template as the background template for the input image; and generating a background template for the input image based on the input image based on the input image based on the second similarity being less than a second threshold.

[0145] According to an embodiment of the present disclosure, the template determination unit 730 may be configured to update the first background template according to the input image based on the second similarity being greater than or equal to the second threshold through the following operations: segmenting the input image by using a segmentation model to obtain a background area of ​​the input image; and filling a portion of the area to be filled included in the second background template based on the background area to obtain an updated first background template.

[0146] According to an embodiment of the present disclosure, the template determination unit 730 may be configured to fill a portion of the area to be filled in the second background template based on the background area through the following operations to obtain an updated first background template: determine the overlapping area between the background area and the area to be filled in the second background template; fill the overlapping area in the second background template based on the overlapping area in the input image; and determine the second background template with the overlapping area filled as the updated first background template.

[0147] According to an embodiment of the present disclosure, the template determination unit 730 may be configured to generate a background template for the input image based on the input image, based on the second similarity being less than a second threshold, through the following operations: segmenting the input image by using a segmentation model to obtain a non-background area of ​​the input image; setting the non-background area in the input image as the area to be filled; and determining the input image with the area to be filled as the background template for the input image.

[0148] According to an embodiment of the present disclosure, the foreground extraction unit 740 may be configured to extract a foreground image from an input image based on a background template for the input image by the following operations: obtaining a difference map of the input image based on the input image and the background template; obtaining a mask image based on the value of each pixel in the difference map; and obtaining a foreground image based on the input image and the mask image.

[0149] According to an embodiment of the present disclosure, the background template for the input image includes an area to be filled for extracting a foreground image, wherein image features of the area to be filled are set based on the frequency of occurrence of image features and are different from image features of areas other than the area to be filled in the background template.

[0150] According to an embodiment of the present disclosure, the image processing device 700 may further include an object detection unit. The object detection unit may be configured to: detect whether a predetermined object exists in the input image by using a target detection model; in response to detecting the predetermined object in the input image, stop image processing for the input image, and reacquire the input image.

[0151] The specific manner in which the various units of the image processing device 700 in the above embodiment perform operations has been described in detail in the embodiment of the relevant image processing method, and will not be described in detail here.

[0152] Figure 8 Detailed description is a block diagram showing a structure of an object detection device according to an embodiment of the present disclosure.

[0153] According to an embodiment of the present disclosure, the object detection apparatus 800 may include a video acquisition unit 810 , a foreground processing unit 820 , and an object detection unit 830 .

[0154] Specifically, the video acquisition unit 810 can be configured to acquire a video including multiple input images, the foreground processing unit 820 can be configured to execute the image processing method described above in sequence for the multiple input images, and the target detection unit 830 can be configured to perform target detection on the foreground image obtained by the image processing method by using a target detection model.

[0155] That is, the video acquisition unit 810 can perform the same Figure 6The foreground processing unit 820 may perform the same operation as described above with reference to step S610 of the image processing method described above. Figure 6 The corresponding operation of step S620 of the image processing method described above or as referred to above Figure 1 and Figure 2 The target detection unit 830 may perform the operations corresponding to the image processing method described above. Figure 6 The described image processing method corresponds to step S630.

[0156] The specific manner in which the various units of the target detection device 800 in the above embodiment perform operations has been described in detail in the embodiments of the relevant target detection method, and will not be described in detail here.

[0157] In addition, it should be understood that the various units in the image processing device 700 and the target detection device 800 according to the embodiments of the present disclosure may be implemented as hardware components and / or software components. Those skilled in the art may implement the various units, for example, using a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), according to the processing or operation performed by the defined various units.

[0158] According to an embodiment of the present disclosure, an electronic device is also provided. Fig. 9 is a structural block diagram showing an electronic device according to an embodiment of the present disclosure.

[0159] According to an embodiment of the present disclosure, the electronic device 900 may include at least one processor 910 and at least one memory 920 storing computer executable instructions. When the computer executable instructions are executed by the at least one processor 910, the at least one processor 910 executes the image processing method or target detection method as described above.

[0160] According to an embodiment of the present disclosure, the electronic device 900 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-mentioned instruction set. Here, the electronic device 900 is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above-mentioned instructions (or instruction sets) individually or jointly. The electronic device 900 may also be part of an integrated control system or a system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0161] In the electronic device 900, the processor 910 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor 910 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0162] The processor 910 may execute instructions or codes stored in the memory 920, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0163] The memory 920 may be integrated with the processor 910, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 920 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 920 and the processor 910 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 910 can read files stored in the memory 920.

[0164] In addition, the electronic device 900 may further include a display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 900 may be connected to each other via a bus and / or a network.

[0165] According to an embodiment of the present disclosure, a computer-readable storage medium may also be provided, wherein when instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the image processing method or target detection method as described above.

[0166] According to an exemplary embodiment of the present disclosure, examples of computer-readable storage media include read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, nonvolatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in electronic devices such as clients, hosts, proxy devices, servers, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0167] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0168] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the claims.

Claims

1. An image processing method, comprising: Get the input image; Determining a first similarity between the input image and a preset first background template; Determining a background template for the input image based on the first similarity; A foreground image is extracted from the input image based on a background template for the input image.

2. The image processing method according to claim 1, wherein: The step of determining a background template for the input image based on the first similarity comprises: Based on the first similarity being greater than or equal to a first threshold, determining the first background template as the background template for the input image; and Based on the first similarity being less than a first threshold, a background template for the input image is determined according to the input image and the first background template.

3. The image processing method according to claim 2, wherein: Based on the first similarity being less than a first threshold, the step of determining a background template for the input image according to the input image and the first background template comprises: Aligning the input image with the first background template to obtain an aligned second background template; Determining a second similarity between the input image and a second background template; Based on the second similarity being greater than or equal to a second threshold, updating the first background template according to the input image, and determining the updated first background template as the background template for the input image; and Based on the second similarity being less than a second threshold, a background template for the input image is generated according to the input image.

4. The image processing method according to claim 3, wherein: Based on the second similarity being greater than or equal to a second threshold, the step of updating the first background template according to the input image comprises: Segmenting the input image by using a segmentation model to obtain a background area of ​​the input image; Based on the background area, a portion of the to-be-filled area included in the second background template is filled to obtain an updated first background template.

5. The image processing method according to claim 4, wherein: Based on the background area, the step of filling a part of the to-be-filled area included in the second background template to obtain an updated first background template includes: Determine an overlapping area between the background area and the area to be filled in the second background template; Filling the overlapping area in the second background template based on the overlapping area in the input image; The second background template that fills the overlapping area is determined as the updated first background template.

6. The image processing method according to claim 3, wherein: Based on the second similarity being less than a second threshold, the step of generating a background template for the input image according to the input image comprises: Segmenting the input image by using a segmentation model to obtain a non-background area of ​​the input image; Setting the non-background area in the input image as the area to be filled; The input image in which the area to be filled is set is determined as a background template for the input image.

7. The image processing method according to claim 1, wherein: The step of extracting a foreground image from the input image based on a background template for the input image comprises: Based on the input image and the background template, obtaining a difference map of the input image; Obtaining a mask image based on the value of each pixel in the difference map; The foreground image is obtained based on the input image and the mask image.

8. The image processing method according to claim 1, wherein: The background template for the input image includes a to-be-filled area for extracting a foreground image, wherein image features of the to-be-filled area are set based on image feature occurrence frequency and are different from image features of areas other than the to-be-filled area in the background template.

9. A target detection method, comprising: Obtaining a video comprising a plurality of input images; For the multiple input images, sequentially executing the image processing method according to any one of claims 1 to 8; By using the target detection model, target detection is performed on the foreground image obtained by the image processing method.

10. An image processing device, comprising: An image acquisition unit, configured to acquire an input image; a similarity determination unit, configured to determine a first similarity between the input image and a preset first background template; a template determining unit, configured to determine a background template for the input image based on a first similarity; The foreground extraction unit is configured to extract a foreground image from the input image based on a background template for the input image.

11. A target detection device, comprising: A video acquisition unit configured to acquire a video including a plurality of input images; a foreground processing unit, configured to sequentially execute the image processing method according to any one of claims 1 to 8 for the multiple input images; The target detection unit is configured to perform target detection on the foreground image obtained by the image processing method by using a target detection model.

12. An electronic device comprising: at least one processor; at least one memory storing computer executable instructions, When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to perform the method according to any one of claims 1 to 9.

13. A computer-readable storage medium, wherein: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the method according to any one of claims 1 to 9.

Images