A foreground extraction method and related equipment based on constant illumination transformation

Abstract

This invention discloses a foreground extraction method and related equipment based on constant illumination transformation. The method includes: for a video sequence, before background modeling, learning a consistent representation of its illumination invariance using a neural network; performing KNN background modeling on the original image and the image after constant illumination transformation; obtaining binary images with preliminary separation of foreground and background through KNN background modeling, and performing morphological processing on the binary images; after background modeling and morphological processing on the two images, performing a difference operation on the two foreground binary images to obtain accurate foreground extraction results for subsequent computer vision tasks. This invention eliminates interference caused by illumination changes in the image by considering constant illumination transformation for each image before background modeling, thereby reducing the negative impact of illumination changes on background modeling at the source, making the separation of background and foreground clearer, and effectively suppressing foreground noise.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and traffic technology, and in particular to a foreground extraction method and related equipment based on constant illumination transformation. Background Technology

[0002] In computer vision tasks, foreground extraction methods are widely used to filter moving targets. For example, in highway scenes, moving targets often include potential debris, such as detached vehicle tires, fragments, or discarded objects that could cause traffic accidents. Therefore, foreground extraction algorithms are used to filter moving targets, providing preprocessing for subsequent debris detection tasks. However, in practical applications, some complex scenes present numerous challenges to foreground extraction, such as changes in lighting and weather. These changes blur the difference between the static background and foreground targets in background modeling, causing changed areas to be mistakenly identified as foreground targets. This leads to more false positives in subsequent computer vision tasks, thus affecting the overall detection performance.

[0003] Currently, several background modeling methods aim to address the impact of illumination changes on foreground extraction. ZBS (Zero-Based Background Analysis) is a relatively new method that leverages phase information from illumination changes to improve the robustness of background modeling. However, this method requires significant computation for phase analysis, resulting in slow processing speeds and unsuitability for real-time monitoring scenes. Furthermore, it fails to effectively detect small moving targets, such as debris on highways, leading to missed detections. Other deep learning-based background modeling algorithms also exhibit some robustness to illumination changes, but their generalization ability is limited by training data constraints, preventing them from being applied to multiple highway scenes. Therefore, existing background modeling algorithms cannot meet the needs of practical applications. Summary of the Invention

[0004] In order to at least partially solve one of the technical problems existing in the prior art, the purpose of this invention is to provide a foreground extraction method and related equipment based on constant illumination conversion.

[0005] The first technical solution adopted in this invention is:

[0006] A foreground extraction method based on constant illumination transformation includes the following steps:

[0007] S1, Illumination-invariant transformation: For a video sequence, before background modeling, a neural network learns its consistent representation with respect to illumination invariance;

[0008] S2, KNN background modeling: Perform KNN background modeling on the original image and the image after constant illumination transformation to obtain preliminary foreground moving targets;

[0009] S3. Morphological processing: After obtaining a binary image with preliminary separation between the foreground and background through KNN background modeling, morphological processing is performed on the binary image.

[0010] S4. Difference operation: After performing background modeling and morphological processing on two images, in order to obtain a more accurate foreground target and maintain robustness to changes in illumination, the two binary foreground images are subjected to a difference operation to obtain accurate foreground extraction results for subsequent computer vision tasks.

[0011] Further, step S1 includes:

[0012] By using self-supervised learning, multiple sets of images of the same scene under different lighting conditions are used to learn an illumination-invariant image representation, thus eliminating the limitations of the physical model;

[0013] By combining the local attention mechanism of ELA, a PLAM model is designed to further enhance the network's attention to local information and improve its performance under varying illumination.

[0014] Furthermore, the method employs self-supervised learning to learn an illumination-invariant image representation of the same scene under different lighting conditions using multiple sets of images, thereby eliminating the limitations of the physical model. This includes:

[0015] First, let f int The intrinsic image representation represents the inherent features of a scene that remain unchanged regardless of changes in external conditions.

[0016] Let R θ R represents an image transformation with a set of parameters θ; θ It is used to simulate the changes in different image attributes caused by different conditions during the image acquisition process;

[0017] By analyzing f int Apply different transformations Obtain a series of different image instances f i , where θ i This represents specific parameters associated with the transformation; each image instance has different attributes, but they all represent the same scene, and the image transformation can be represented as:

[0018]

[0019] Design a transformation F(·) to be applied to image instances, such that all instances transformed by R... θ Acting on the intrinsic representation of the image f int The generated image instance f i All can be mapped to the same invariant representation space through F(·), and the resulting image is called f. inv ;

[0020] Let f = {f1, f2, ..., f N Let f be a set of N image instances, all of which are from the same scene. int However, they are affected by different environments; i Represents the i-th image instance; for another scene g int Also define an image set g = {g1, g2, ..., g} N};F(·) should satisfy different instances f in the same scene i and f j The similarity should be maintained as much as possible after the transformation:

[0021]

[0022] In the formula, D(·,·) represents a certain distance metric, and ∈ represents a small constant;

[0023] For image instances f from different scenes i and g j After the transformation, there should be a significant difference to ensure that the two can be effectively distinguished. F(·) should satisfy:

[0024]

[0025] In the formula, c represents a positive constant;

[0026] To obtain the transformation F(·) that meets the above requirements, a data-driven approach is adopted, and the Triplet network is trained according to equations (2) and (3); the network training set consists of M sets of data f m The data consists of M groups of different scenarios, where m = 1, ..., M, and each group contains N instances f. i m These examples represent the same scenario. After different The transformation obtained from it.

[0027] Furthermore, the KNN background modeling algorithm, also known as the k-nearest neighbor algorithm, is a non-parametric, variable bandwidth kernel density estimation method, which works as follows:

[0028] Let x1, x2, ..., x n Given an observed sample of a pixel in a time series, the value x of the sample at time t is... t The probability density is expressed as:

[0029]

[0030] The kernel function K adopts the normal distribution function N(0,δ), where δ is the bandwidth of the kernel function;

[0031] If P r (x) <T HP If the value of T is true, then the pixel x is considered foreground; otherwise, it is considered background. HP It is a predefined threshold.

[0032] Furthermore, the morphological processing includes expansion and etching operations;

[0033] The dilation operation fills gaps between foreground objects or connects broken target areas by expanding the foreground region in the image, thereby enhancing the shape of the foreground objects or restoring the connectivity of the target regions; for each pixel in the image, the dilation operation checks the neighboring pixels of the area covered by the surrounding structuring elements; if any of the neighboring pixels belongs to the foreground (i.e., has a value of 1), then the pixel is set as foreground;

[0034] The erosion operation is the opposite of the dilation operation. This operation checks the neighboring pixels of the area covered by the surrounding structuring elements of the pixel; if all pixels in the neighborhood are foreground (i.e., the value is 1), then the pixel remains foreground; otherwise, the pixel is set to background.

[0035] Further, step S3 includes:

[0036] One closing operation and one dilation operation are used to perform morphological processing on the binary images obtained by background modeling of two images to reduce noise and fill holes in the foreground.

[0037] The closing operation involves first performing an expansion operation and then an erosion operation, which can fill small holes in the foreground target or gaps between connected targets, while removing small background noise in the image.

[0038] Further, step S4 includes:

[0039] By using the difference operation, we can remove interference noise caused by changes in illumination while preserving the accuracy of the foreground object in the original image. The expression for the difference operation is as follows:

[0040] F final =|F org -F L_inv |

[0041] In the formula, F org and F L_inv These represent the original image and the foreground binary image obtained by morphological processing of the background model image after constant illumination transformation.

[0042] The second technical solution adopted in this invention is:

[0043] An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a foreground extraction method based on constant illumination transformation as described above.

[0044] The third technical solution adopted in this invention is:

[0045] A computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement a foreground extraction method based on constant illumination transformation as described above.

[0046] The fourth technical solution adopted in this invention is:

[0047] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned foreground extraction method based on constant illumination transition.

[0048] The beneficial effects of this invention are as follows: By performing a constant illumination transformation on each image before background modeling, this invention eliminates interference caused by illumination variations in the image. In this way, the negative impact of illumination variations on background modeling can be reduced at the source, resulting in clearer separation between the background and foreground, and effective suppression of foreground noise. Simultaneously, after performing the constant illumination transformation on the image, KNN background modeling and morphological processing are applied to remove minor noise while obtaining a relatively complete foreground target. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the steps of a foreground extraction method based on constant illumination transformation in an embodiment of the present invention. Detailed Implementation

[0051] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0052] The terminology used in the embodiments of this application is for the purpose of describing specific embodiments only and is not intended to limit the embodiments of this application. The singular forms "a," "described," and "the" used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. Furthermore, unless otherwise expressly limited, terms such as "set," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0053] In the description of this application, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0054] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0055] In the description of this application, "and / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0056] Terminology Explanation:

[0057] ELA (Efficient Local Attention Mechanism) is an attention mechanism designed to improve the performance of deep convolutional neural networks (CNNs).

[0058] PLAM: an abbreviation for Photo-consistent Local Attention Module, is a semi-parametric model that combines the flexibility of non-parametric models with the interpretability of parametric models.

[0059] To address the aforementioned problems, this invention provides a foreground extraction algorithm based on constant illumination transformation. By performing constant illumination transformation on each image before background modeling, interference caused by illumination variations in the image is eliminated. This approach reduces the negative impact of illumination changes on background modeling at the source, resulting in clearer separation between background and foreground and effective suppression of foreground noise. Simultaneously, after performing constant illumination transformation on the image, KNN background modeling and morphological processing are applied to remove minor noise while obtaining a more complete foreground target.

[0060] Example 1

[0061] like Figure 1 As shown, this embodiment provides a foreground extraction method based on constant illumination transformation, including the following steps:

[0062] S1. Illumination-Invariant Transformation: For a video sequence, before background modeling, a neural network learns and outputs a consistent representation of illumination invariant properties. For images of the same scene under different lighting conditions, the feature space of the illumination representation should be the same after passing through this neural network. Furthermore, substantial changes in the scene can be detected after the illumination-invariant transformation.

[0063] As an optional implementation, this embodiment employs an image-based consistency transformation method. This method directly generates an image map with constant illumination, thereby eliminating the impact of illumination variations on the image without relying on complex physical models. In this method, accurate estimation of object albedo and shadows is not required; instead, a simple image transformation technique is used to quickly generate an image representation that maintains consistency under different illumination conditions. This method is well-suited to the needs of computer vision tasks, especially those requiring real-time performance and accuracy.

[0064] Specifically, this embodiment references PhIT-Net and, through self-supervised learning, utilizes multiple sets of images of the same scene under different lighting conditions to learn an illumination-invariant image representation, thus eliminating the limitations of physical models. Based on this, we combined the ELA local attention mechanism to design the Photo-consistent Local Attention Module (PLAM), which further enhances the network's attention to local information and improves performance under varying lighting conditions.

[0065] For example, first let f int The intrinsic image representation represents the inherent features of a scene, remaining unchanged regardless of changing external conditions. However, due to variations in environmental conditions such as lighting changes, atmospheric interference, or noise interference during image acquisition, the image's properties may change. Let R... θ R represents an image transformation with a set of parameters θ. θ This is used to simulate changes in image attributes caused by different conditions during image acquisition. By analyzing f... int Apply different transformations A series of different image instances can be obtained f i , where θ i This represents specific parameters associated with the transformation. Each image instance has different attributes, but they all represent the same scene. The image transformation can be represented as:

[0066]

[0067] Since our goal is simply to keep image instances under different lighting or environmental conditions consistent in the light space, without attempting to estimate the image transformation parameters θ i Alternatively, we can obtain the intrinsic representation of the image. Therefore, we design a transform F(·) applied to image instances, and all instances transformed by R... θ Acting on the intrinsic representation of the image f int The generated image instance f i All can be mapped to the same invariant representation space through F(·), and the resulting image is called f. inv To ensure the validity of F(·), we require that for the intrinsic images f in both scenarios... int and g int If d(f) int ,g int If d(·,·) represents a metric function, then after the mapping F(·), it should also satisfy d(F(f)). i ),F(f(g i ))>δ. This indicates that the transformation F(·) can not only map different image instances of the same scene to the same invariant representation space, but also maintain the distinguishability between different scenes, which is necessary for subsequent foreground extraction tasks.

[0068] Let f = {f1, f2, ..., f N Let f be a set of N image instances, all of which are from the same scene. int However, they are affected by different environments. i Let represent the i-th image instance. Similarly, let's consider another scene g.int Also define an image set g = {g1, g2, ..., g} N The F(·) we define should satisfy the condition that for different instances f in the same scene... i and f j The similarity should be maintained as much as possible after the transformation:

[0069]

[0070] Where D(·,·) represents a distance metric, and ∈ represents a small constant. Furthermore, for image instances f from different scenes... i and g j After the transformation, there should be a significant difference to ensure that the two can be effectively distinguished. F(·) should satisfy:

[0071]

[0072] To obtain the transformation F(·) that satisfies the above requirements, we do not... θ Instead of modeling, a data-driven approach is used, employing the Triplet network trained according to equations (2) and (3). The network training set consists of M sets of data f m The data is composed of M groups of different scenarios, where m = 1, ..., M, and each group contains N instances f. i m These examples represent the same scenario. After different The transformation is obtained by this method. This method can ensure that images under different lighting conditions can obtain a consistent representation with invariant illumination. This illumination-constant image transformation can be applied to computer vision tasks and achieve good performance.

[0073] However, while PhIT-Net learns to obtain illumination-invariant representations of images under different lighting conditions within the same scene, in certain scenarios, the network may not adequately focus on some crucial regions when capturing image details, especially those with significant illumination changes. To address these issues, we introduce the ELA (Enhanced Local Attention) mechanism. This mechanism adaptively assigns higher weights to regions with significant brightness variations, thus more accurately capturing these changes and ensuring that areas with substantial illumination variations are not overlooked during the learning process. Simultaneously, the ELA attention mechanism also focuses on regions with significant structural differences, thereby enhancing the model's sensitivity to structural variations.

[0074] S2, KNN background modeling: Perform KNN background modeling on the original image and the image after constant illumination transformation.

[0075] After obtaining the illumination-invariant representation of the image through step S1, background modeling is then performed. This avoids the impact of illumination changes during foreground extraction, while still effectively detecting actual changes, i.e., moving targets. Furthermore, KNN background modeling is also performed on the original image (the one without illumination invariance processing) to obtain a more accurate foreground target region. Through this step, preliminary foreground moving targets are obtained.

[0076] In one embodiment, KNN background modeling is performed on both the original image and the image after constant illumination transformation. The KNN background modeling algorithm, also known as the k-nearest neighbor algorithm, is a nonparametric, variable-bandwidth kernel density estimation algorithm. If an example is similar to its K nearest neighbors in the feature space, the class of that example is determined by the majority class of those K nearest neighbors. Specifically, let x1, x2, ..., x... n Given an observed sample of a pixel in a time series, the value x of the sample at time t is... t The probability density can be expressed as:

[0077]

[0078] The kernel function K follows a normal distribution N(0,δ), where δ is the bandwidth of the kernel function. It calculates the value of a pixel x at a single time step. t With all its historical samples x i The distance between them is calculated, and the summation yields an estimate of the entire probability density. Based on this probability density estimate, if P... r (x) <T HP If the value of T is true, then the pixel x is considered foreground; otherwise, it is considered background. HP It is a predefined threshold, determined by minimizing the cost of misclassification. KNN enables foreground objects to blend into the background more slowly after they have come to rest, retains complete foreground pixels for a longer period of time, and can effectively identify small targets in changing regions.

[0079] S3. Morphological processing: A binary image is obtained by KNN background modeling to initially separate the foreground and background. Morphological processing is then performed on the binary image.

[0080] KNN background modeling yields a preliminary separation result between the foreground and background. However, the binary image still contains a significant amount of fine noise. To further improve the foreground image, reduce noise, and fill holes in the foreground, morphological processing can be performed on the binarized image. Optionally, this embodiment employs one closing operation and one dilation operation to perform morphological processing on the binary mask images obtained from the background modeling of the two images, thereby reducing noise and filling holes in the foreground.

[0081] Specifically, morphological processing is performed on the results of two background modeling operations. Morphological operations include dilation and erosion, which, based on structuring elements of rectangular or circular templates, enhance or reduce the morphological features of target regions in an image by manipulating the neighborhood relationships of pixels. Specifically, dilation fills gaps between foreground targets or breaks in the connection of target regions by expanding the foreground region in the image, thereby enhancing the shape of the foreground target or restoring the connectivity of the target region. For each pixel in the image, the dilation operation checks the neighboring pixels within the area covered by the surrounding structuring element. If any pixel in the neighborhood belongs to the foreground (value 1), the pixel is set as foreground. Erosion is the opposite of dilation; it checks the neighboring pixels within the area covered by the surrounding structuring element. If all pixels in the neighborhood belong to the foreground (value 1), the pixel remains foreground; otherwise, it is set as background. In practical applications, erosion and dilation operations are often used in combination for effective image denoising and enhancing the connectivity of target regions. For example, a closing operation, which involves first dilation and then erosion, can fill small holes in foreground targets or gaps between connected targets while removing small background noise from the image. The opening operation involves first performing an erosion operation, followed by a dilation operation. It is mainly used to remove small objects or discrete noise points in an image. This paper performs one closing operation and one dilation operation on a binary image. As shown in equation (5), K0 represents a kernel with a step size of 1 and a size of 3×3, and I represents the image to be operated on. · indicates an expansion operation, and · indicates a closing operation.

[0082]

[0083] S4. Difference operation: After performing background modeling and morphological processing on two images, the two foreground binary images are subjected to a difference operation to obtain accurate foreground extraction results for subsequent computer vision tasks.

[0084] After performing background modeling and morphological processing on two images, this embodiment performs a difference operation on the two foreground binary images to obtain more accurate foreground targets and maintain robustness to illumination changes. Background modeling of the image after constant illumination transformation effectively eliminates the impact of illumination changes, thereby reducing noise caused by illumination variations. However, the image after constant illumination transformation is not accurate enough in representing foreground targets and is prone to losing detail information. On the other hand, background modeling of the original image can better extract foreground targets and ensure accurate target positioning, but the original image contains more illumination noise, which may affect the final foreground extraction result. To compensate for the shortcomings of both, this embodiment uses a difference operation to combine the results of these two background modeling methods. That is, the final accurate foreground extraction result is input into subsequent computer vision tasks.

[0085] Specifically, to obtain a more accurate foreground target and maintain robustness to illumination changes, we perform a difference operation on the two foreground binary images. Through this difference operation, we can remove interference noise caused by illumination changes while preserving the accuracy of the foreground target in the original image. The difference operation is as follows:

[0086] F final =|F org -F L_inv | (6)

[0087] Where F org and F L_inv These represent the original image and the foreground binary image obtained by morphological processing of the background model image after constant illumination transformation.

[0088] In summary, compared with the prior art, the method of the present invention has at least the following advantages and beneficial effects:

[0089] (1) Reducing interference from illumination changes: This invention eliminates interference from illumination changes by performing a constant illumination transformation on the image before background modeling. This process effectively reduces the negative impact of illumination changes on foreground extraction and background modeling, making the separation of background and foreground clearer, thereby effectively suppressing foreground noise.

[0090] (2) Efficient foreground extraction: After performing constant illumination transformation, combined with KNN background modeling and morphological processing, the foreground target can be accurately extracted. Through morphological operations, small noises are reduced and holes in the foreground target are filled, further improving the quality of foreground extraction.

[0091] (3) Strong applicability: The method of this invention combines self-supervised learning with ELA local attention mechanism, which can adaptively process images under different lighting conditions, improve the model's attention to image details, and further enhance the ability to identify dynamically changing regions. It is more robust than traditional foreground extraction algorithms, and more applicable to deep learning-based background modeling algorithms, and can be generalized to multiple scenarios.

[0092] Example 2

[0093] This invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to achieve the following: Figure 1 This illustrates a foreground extraction method based on constant illumination transformation.

[0094] It is understood that the memory may include random access memory (RAM) or read-only memory. Optionally, the memory may include non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the stored data area may store data created according to the use of the server, etc.

[0095] A processor may include one or more processing cores. The processor connects to various parts of the server via various interfaces and lines, executing instructions, programs, code sets, or instruction sets stored in memory, and accessing data stored in memory to perform various server functions and process data. Optionally, the processor may be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor may integrate one or more of the following: Central Processing Unit (CPU) and Modem. The CPU primarily handles the operating system and applications; the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor.

[0096] Since this electronic device is an electronic device corresponding to the foreground extraction method based on constant illumination conversion in the embodiments of the present invention, and the principle of solving the problem by this electronic device is similar to that of the method, the implementation of this electronic device can refer to the implementation process of the above method embodiments, and the repeated parts will not be described again.

[0097] Example 3

[0098] This invention also provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to achieve the following: Figure 1 This illustrates a foreground extraction method based on constant illumination transformation.

[0099] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0100] Since this storage medium is the storage medium corresponding to a foreground extraction method based on constant illumination conversion in an embodiment of the present invention, and the principle of solving the problem by this storage medium is similar to that of this method, the implementation of this storage medium can refer to the implementation process of the above method embodiment, and repeated parts will not be described again.

[0101] Example 4

[0102] In some possible implementations, various aspects of the methods of the embodiments of the present invention can also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of a foreground extraction method based on constant illumination transformation according to various exemplary embodiments of the present application described above. The executable computer program code or "code" for performing the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0103] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0104] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0105] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A foreground extraction method based on illumination invariant transformation, characterized in that, Includes the following steps: S1, Illumination-invariant transformation: For a video sequence, before background modeling, a neural network learns its consistent representation with respect to illumination invariance; S2, KNN background modeling: Performing KNN background modeling on the original image and the image after constant illumination transformation; S3. Morphological processing: After obtaining a binary image with preliminary separation between the foreground and background through KNN background modeling, morphological processing is performed on the binary image. S4. Difference operation: After performing background modeling and morphological processing on two images, the two foreground binary images are subjected to a difference operation to obtain accurate foreground extraction results for subsequent computer vision tasks. Step S1 includes: By using self-supervised learning, multiple sets of images of the same scene under different lighting conditions are used to learn an illumination-invariant image representation, thus eliminating the limitations of the physical model; By combining the local attention mechanism of ELA, the network's attention to local information is further enhanced through the attention mechanism, thereby improving its performance under changes in lighting conditions. The method employs self-supervised learning, utilizing multiple sets of images of the same scene under different lighting conditions to learn an illumination-invariant image representation, thereby eliminating the limitations of physical models. This includes: First, let's denotes the intrinsic image representation, which represents the inherent characteristics of the scene, which remain unchanged regardless of changes in external conditions; Let denote an image transformation with a set of parameters ; be used to simulate changes in image properties due to different conditions during picture acquisition; Through the Apply different transformations To obtain a series of different image instances ,in This represents specific parameters associated with the transformation; each image instance has different attributes, but they all represent the same scene, and the image transformation is represented as: （1） Design a transformation to apply to image instances. All after transformation Action on image intrinsic representation Instances of generated images All can pass The image obtained by mapping to the same invariant representation space is called... ; set up For the reason A collection of image instances, all from the same scene. However, they are affected by different environments; Indicates the first One image instance; for another scene Also define an image set ; It should satisfy different instances in the same scenario and The similarity should be maintained as much as possible after the transformation: （2） In the formula, To represent a certain metric of distance, Represents a small constant; Image instances from different scenes and There should be a significant difference after the transformation to ensure that the two can be effectively distinguished. Should meet: （3） In the formula, Represent a positive integer; In order to obtain a transformation that meets the above requirements It is obtained by training the Triplet network in accordance with Equations (2) and (3) using a data-driven approach.

2. The foreground extraction method based on constant illumination transformation according to claim 1, characterized in that, The KNN background modeling algorithm, also known as the k-nearest neighbor algorithm, is a non-parametric, variable bandwidth kernel density estimation algorithm. It works as follows: set up For an observed sample of a pixel in a time series, the sample in time... value at The probability density is expressed as: The kernel function K adopts a normal distribution function. , The bandwidth of the kernel function; if Then the pixel point It is considered foreground, otherwise it is considered background, in which It is a predefined threshold.

3. The foreground extraction method based on constant illumination transformation according to claim 1, characterized in that, The morphological treatment includes expansion and etching operations; The dilation operation fills gaps between foreground objects or connects broken target areas by expanding the foreground region in the image, thereby enhancing the shape of the foreground objects or restoring the connectivity of the target regions; for each pixel in the image, the dilation operation checks the neighboring pixels of the area covered by the surrounding structuring elements; if any of the neighboring pixels belongs to the foreground (i.e., has a value of 1), then the pixel is set as foreground; The erosion operation is the opposite of the dilation operation. This operation checks the neighboring pixels of the area covered by the surrounding structuring elements of the pixel; if all pixels in the neighborhood are foreground (i.e., the value is 1), then the pixel remains foreground; otherwise, the pixel is set to background.

4. The foreground extraction method based on constant illumination transformation according to claim 1, characterized in that, Step S3 includes: One closing operation and one dilation operation are used to perform morphological processing on the binary images obtained by background modeling of two images to reduce noise and fill holes in the foreground. The closing operation involves first performing an expansion operation and then an erosion operation, which can fill small holes in the foreground target or gaps between connected targets, while removing small background noise in the image.

5. The foreground extraction method based on constant illumination transformation according to claim 1, characterized in that, Step S4 includes: By using the difference operation, we can remove interference noise caused by changes in illumination while preserving the accuracy of the foreground object in the original image. The expression for the difference operation is as follows: In the formula, and These represent the original image and the foreground binary image obtained by morphological processing of the background model image after constant illumination transformation.

6. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method as described in any one of claims 1 to 5.

8. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a processor, are used to perform the method as described in any one of claims 1 to 5.

Images