Foreground extraction method, device, storage medium and equipment

Abstract

One or more embodiments of the present invention provide a foreground extraction method, apparatus, storage medium and device, wherein the foreground extraction method comprises: when a target object does not appear in a framing area, collecting first point cloud data for the framing area; converting the first point cloud data to obtain a first depth image; when the target object appears in the framing area, collecting second point cloud data for the framing area; converting the second point cloud data to obtain a second depth image; comparing each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determining a foreground point cloud in the second point cloud data according to the comparison result, thereby improving the efficiency of foreground extraction.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a foreground extraction method, device, storage medium and equipment. Background Art

[0002] In target detection and extraction, when the background is static, any meaningful moving object is the foreground, and the foreground, i.e. the moving target, can be obtained through background segmentation. Currently, the background segmentation speed for 3D point clouds is slow and cannot distinguish between foreground and background. Summary of the invention

[0003] In view of this, one or more embodiments of the present invention provide a foreground extraction method, apparatus, storage medium and device, which improve the efficiency of foreground extraction.

[0004] One or more embodiments of the present invention also provide a foreground extraction method, comprising: when a target object does not appear in a framing area, collecting first point cloud data for the framing area; converting the first point cloud data to obtain a first depth image; when the target object appears in the framing area, collecting second point cloud data for the framing area; converting the second point cloud data to obtain a second depth image; comparing each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determining a foreground point cloud in the second point cloud data based on the comparison result.

[0005] Optionally, each pixel in the second depth image is compared with the corresponding pixel in the first depth image to obtain a comparison result, and the foreground point cloud in the second point cloud data is determined according to the comparison result, including: calculating the difference between the pixel value of each pixel in the second depth image and the pixel value of the corresponding pixel in the first depth image; determining that the pixel in the second depth image corresponding to the difference is a foreground point when the difference is greater than a threshold; determining that the pixel in the second depth image corresponding to the difference is a background point when the difference is not greater than the threshold; converting the foreground point into a point cloud to obtain a foreground point cloud in the second point cloud data.

[0006] Optionally, the method also includes: after obtaining the first depth image based on the conversion of the first point cloud data, establishing a mixed Gaussian model for each pixel in the first depth image to obtain a mixed Gaussian background model corresponding to each pixel; comparing each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determining a foreground point cloud in the second point cloud data based on the comparison result, including: matching each pixel in the second depth image with the mixed Gaussian background model of the corresponding pixel in the first depth image; determining the points in the second depth image that match the mixed Gaussian background model of the corresponding pixel in the first depth image as background points, and the points that do not match are foreground points; converting the foreground points into a point cloud to obtain a foreground point cloud in the second point cloud data.

[0007] Optionally, the method further includes: after determining the foreground point cloud in the second point cloud data, performing radius filtering on the foreground point cloud.

[0008] Optionally, the method further includes: after determining the foreground point cloud in the second point cloud data, clustering the point cloud to obtain at least two types of point clouds; and segmenting the at least two types of point clouds to obtain at least two independent point clouds.

[0009] Optionally, the method further includes: after obtaining at least two types of independent point clouds, calculating the center of mass and size information of the target corresponding to each independent point cloud.

[0010] Optionally, the first point cloud data includes multiple frames of point cloud data, and obtaining a first depth image according to the conversion of the first point cloud data includes: after collecting a frame of point cloud data, converting the frame of point cloud data into a third depth image; converting the collected new frame of point cloud data into a fourth depth image; comparing the first distance value of each point in the fourth depth image with the second distance value of each point in the third depth image, and if the first distance value is greater than the second distance value, using the first distance value to replace the second distance value in the third depth image;

[0011] Continue to collect a new frame of point cloud data until the preset number of frames of point cloud data have been collected.

[0012] According to one or more embodiments of the present invention, a foreground extraction device is provided, comprising: a first acquisition module, configured to acquire first point cloud data for a framing area when a target object does not appear in the framing area; a first conversion module, configured to obtain a first depth image based on the conversion of the first point cloud data; a second acquisition module, configured to acquire second point cloud data for the framing area when the target object appears in the framing area; a second conversion module, configured to obtain a second depth image based on the conversion of the second point cloud data; a determination module, configured to compare each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determine the foreground point cloud in the second point cloud data based on the comparison result.

[0013] According to one or more embodiments of the present invention, an electronic device is provided, comprising: a housing, a processor, a memory, a circuit board and a power supply circuit, wherein the circuit board is arranged inside a space enclosed by the housing, and the processor and the memory are arranged on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the electronic device; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute any one of the foreground extraction methods mentioned above.

[0014] According to one or more embodiments of the present invention, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable the computer to execute any one of the foreground extraction methods described above.

[0015] The foreground extraction method of one or more embodiments of the present invention first obtains a first depth map of the background based on first point cloud data of the background, and then obtains a second depth map based on second point cloud data containing the target object. The foreground points in the second depth map are identified by comparing pixels in the first depth map with pixels in the second depth map. By converting the three-dimensional point cloud data into a two-dimensional depth map for comparison, the processing speed can be increased, thereby improving the efficiency of foreground extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 The figure is a flow chart showing a foreground extraction method according to one or more embodiments of the present invention.

[0018] Figure 2 is a flow chart showing a foreground extraction method according to one or more embodiments of the present invention.

[0019] Figure 3 It is a schematic diagram of the structure of a foreground extraction device according to one or more embodiments of the present invention.

[0020] Figure 4 It is a schematic diagram showing the structure of an electronic device according to one or more embodiments of the present invention. DETAILED DESCRIPTION

[0021] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0022] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Figure 1 is a flow chart of a foreground extraction method according to one or more embodiments of the present invention. Figure 1 As shown, the method includes:

[0024] Step 101: When the target object does not appear in the framing area, first point cloud data is collected for the framing area;

[0025] For example, for a detected scene, the first point cloud data may be collected when the point clouds in the scene are static and unchanged, or the first point cloud data may be collected when no dynamic objects appear in the scene.

[0026] For example, a laser radar may be used as a sensor to collect the first point cloud data.

[0027] Step 102: Obtain a first depth image according to the first point cloud data conversion;

[0028] The first point cloud data may include, for example, multiple frames of point cloud, and each frame of point cloud in the first point cloud data may be converted into a depth map, that is, the background is modeled as a two-dimensional matrix, in which each element stores a distance value at a corresponding angle.

[0029] For example, according to the characteristics of the sensor that collects the first point cloud data, the horizontal field of view angle h_fov, the horizontal angle resolution h_res, the vertical field of view angle v_fov, and the vertical angle resolution v_res, the first point cloud data is converted into a depth image to obtain a first depth image. The parameters of the first depth image can be expressed as follows:

[0030] Number of rows in the depth image

[0031] Number of columns in the depth image

[0032] The formula for converting a point (x, y, z) to a depth image is as follows:

[0033]

[0034] r_idx=rows / 2-atan2(y,x)*180 / pi / v_res;

[0035]

[0036] depthmap[r_idx, c_idx]=dis tan ce;

[0037] Among them, distance represents the distance value of the depth map, r_idx represents the index of the row where the point cloud data is projected onto the depth image, and c_idx represents the index of the column where the point cloud data is projected onto the depth image.

[0038] Since multi-frame point cloud data is used in background modeling, the data of each frame needs to be compared with the distance value of the background depth image to determine whether to update the data at the corresponding position of the background depth image. The whole process of background modeling can be as follows: Figure 2 shown.

[0039] Step 103: When the target object appears in the framing area, collect second point cloud data for the framing area;

[0040] The target object may be, for example, a moving object that appears in the scene to be detected, that is, the second point cloud data is collected when the moving object appears in the scene to be detected.

[0041] Similarly, laser radar can also be used to collect the second point cloud data.

[0042] Step 104: Obtain a second depth image according to the second point cloud data conversion;

[0043] For example, the second depth image may be obtained by converting the second point cloud data in the same manner as the first depth image is obtained by converting the first point cloud data in step 102, which will not be described in detail herein.

[0044] Step 105: Compare each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determine a foreground point cloud in the second point cloud data according to the comparison result.

[0045] For example, pixels in the second depth image at the same position as those in the first depth image may be used as corresponding pixels in the first depth image.

[0046] In step 105, by comparing the differences between corresponding pixels in the two depth images, the pixel points belonging to the target object in the second depth image, i.e., the foreground points, can be accurately identified, and the identified foreground points are converted into point clouds to obtain the foreground point cloud in the second point cloud data.

[0047] The foreground extraction method of one or more embodiments of the present invention first obtains a first depth map of the background based on first point cloud data of the background, and then obtains a second depth map based on second point cloud data containing the target object. The foreground points in the second depth map are identified by comparing pixels in the first depth map with pixels in the second depth map. By converting the three-dimensional point cloud data into a two-dimensional depth map for comparison, the processing speed can be increased, thereby improving the efficiency of foreground extraction.

[0048] In one or more embodiments of the present invention, each pixel in the second depth image is compared with each corresponding pixel in the first depth image to obtain a comparison result, and determining the foreground point cloud in the second point cloud data according to the comparison result may include: calculating the difference between the pixel value of each pixel in the second depth image and the pixel value of each corresponding pixel in the first depth image; determining that the pixel in the second depth image corresponding to the difference is a foreground point when the difference is greater than a threshold; determining that the pixel in the second depth image corresponding to the difference is a background point when the difference is not greater than the threshold; converting the foreground point into a point cloud to obtain the foreground point cloud in the second point cloud data. For example, assuming that the pixel value of pixel A in the second depth image is fabs|Depthmap[i,j], the pixel value of the pixel corresponding to the pixel A in the first depth image is Background[i,j], and the threshold is assumed to be tolerance. If it is determined by comparison that fabs|Depthmap[i,j]-Background[i,j]|>tolerance, the pixel A is regarded as a foreground point, otherwise the pixel A is regarded as a background point. After each foreground point is determined, the foreground point cloud can be output.

[0049] In one or more embodiments of the present invention, the foreground extraction method may further include: after determining the foreground point cloud in the second point cloud data, performing radius filtering on the foreground point cloud to eliminate noise and abnormal points. When determining the foreground point and background point in the second depth image by comparing the pixel values ​​of the pixel points in the first depth image and the second depth image, some background points may be mistakenly identified as foreground points due to factors such as sensor acquisition noise. However, the number of such misjudged points is relatively small and relatively discrete compared to the actual foreground points. In order to remove these misjudged foreground points, a radius filtering method may be used to remove points with fewer points within a certain spatial range of the determined foreground point cloud. For example, points with fewer points within a certain spatial range than a preset value may be removed. Radius filtering is a filtering operation based on three-dimensional space. Since the number of points to be filtered out is relatively small, performing radius filtering at this time will be much faster than performing radius filtering on the original second point cloud data, and can meet real-time requirements.

[0050] In one or more embodiments of the present invention, the collected second point cloud data may include multiple target objects. After the above processing, the point clouds of multiple target objects in the second point cloud data can be extracted together, but these point cloud data are not segmented according to different objects. Therefore, in one or more embodiments of the present invention, the foreground extraction method may also include: after determining the foreground point cloud in the second point cloud data, clustering the point cloud to obtain at least two types of point clouds; segmenting the at least two types of point clouds to obtain at least two types of independent point clouds. For example, the determined foreground point cloud can be segmented into several independent point clouds according to different objects by using distance-based Euclidean clustering, so as to facilitate the subsequent independent analysis of each independent point cloud.

[0051] In one or more embodiments of the present invention, the foreground extraction method may further include: after obtaining at least two types of independent point clouds, calculating the centroid and size information of the target corresponding to each independent point cloud. For example, the coordinates of the centroid and attribute information such as length, width, and height of each independent point cloud may be calculated, or the position and volume of each target object may be calculated.

[0052] In one or more embodiments of the present invention, since the background distance value in the depth image may not be in a single-peak state, it may be in a multi-peak state. For example, in a scene where wind blows the leaves, the distance values ​​detected by the sensor are different, but the detected distance values ​​in this scene cannot be regarded as foreground targets. For this kind of scene, a mixed Gaussian model can be used to process the depth map represented by the distance, and a mixed Gaussian model can be established for the collected multiple frames of first point cloud data. This model is used to characterize the background point cloud data, so it can be called a mixed Gaussian background model. Based on this, the foreground extraction method may also include: after obtaining the first depth image according to the conversion of the first point cloud data, a mixed Gaussian model is established based on each pixel in the first depth image to obtain a mixed Gaussian background model corresponding to each pixel;

[0053] For example, for each distance pixel x in the first depth image above i The established mixed Gaussian model is as follows:

[0054]

[0055] Among them, λ i，k Represents x i The weight of the kth Gaussian component of the mixed Gaussian model; μ i，k Represents x i The mean of the kth Gaussian component of the Gaussian mixture model; ∑ i，k Represents x i The variance of the kth Gaussian component of the mixed Gaussian model;

[0056] Parameter initialization:

[0057] A mixed Gaussian model of 5 Gaussian components with k=5 is used. The first Gaussian component is initialized in the first frame depth image, with the mean being the value of the current pixel, the variance being a larger value, and the weight being 1. The mean, variance, and weight of the other Gaussian components except one Gaussian component can all be 0.

[0058] Parameter update:

[0059] The first frame of depth image data performs basic parameter initialization, and the subsequent depth images update the parameters of each component. The update process is as follows:

[0060] Step a: Pixel x of each frame i Calculate the model mean μ of each Gaussian component i，k If the distance is less than 2.5 times The current pixel is considered to match the kth Gaussian component, otherwise it is considered not to match.

[0061] Update weight coefficients:

[0062] λ i，k =(1-α)λ i，k +aM i，k ;

[0063] Among them, α is the learning rate, and M i，k is 1, otherwise it is 0;

[0064] After the match is confirmed, proceed to step b below. If there is no match, k=k+1, and execute step a again. If all components have been tried to match, proceed to step c.

[0065] Step b: Update Norm[μ i，k ,∑ i，k ] parameters, after the parameters are completed, go to step a;

[0066] u i，k =(1-α)u i，k +αx i

[0067] ∑ i，k =(1-α)∑ i，k +α(x i -u i，k ) T (x i -u i，k );

[0068] Step c: Normalize the sum of the weights of the k Gaussian components;

[0069]

[0070] Step d: If none of the Gaussian components in step a have been matched, then a new Gaussian component is added to replace the smallest component among the k Gaussian components;

[0071] Step e: After all the depth images used to establish the mixed Gaussian background model have completed the above steps a, b, c, and d, select B Gaussian components with the largest weights from the k Gaussian components of each pixel as the mixed Gaussian components of the true background model.

[0072] Comparing each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determining a foreground point cloud in the second point cloud data according to the comparison result, including:

[0073] Matching each pixel in the second depth image with a mixed Gaussian background model of a corresponding pixel in the first depth image;

[0074] Determine points in the second depth image that match the mixed Gaussian background model of corresponding pixels in the first depth image as background points, and determine unmatched points as foreground points;

[0075] For example, the pixel point S in the second depth image and the pixel point S' in the first depth image are corresponding pixels, and the mixed Gaussian background model corresponding to the pixel point S' is M. When it is determined that the pixel point B matches the model M, the pixel point S is determined to be a background point, otherwise the pixel point S is determined to be a foreground point.

[0076] When determining whether each pixel in the second depth image is a foreground point or a background point, each pixel may be matched with a mixed Gaussian background model (B components) of each pixel in the first depth image in turn. For example, for pixel x i , calculate its model mean μ with each Gaussian component i，k If the distance is less than 2.5 times Then the pixel x i The kth Gaussian component is matched, and the pixel x i is a background point, otherwise determine the pixel x i It is a foreground spot.

[0077] The foreground point is converted into a point cloud to obtain the foreground point in the second point cloud data.

[0078] In one or more embodiments of the present invention, the first point cloud data includes multiple frames of point cloud data, and converting the first point cloud data to obtain a first depth image may include: after collecting a frame of point cloud data, converting the frame of point cloud data into a third depth image; converting a new frame of point cloud data collected into a fourth depth image; comparing the first distance value of each point in the fourth depth image with the second distance value of each point in the third depth image, and if the first distance value is greater than the second distance value, using the first distance value to replace the second distance value in the third depth image; and continuing to collect a new frame of point cloud data until a preset number of frames of point cloud data have been collected. Figure 2 Taking the figure as an example, the process of converting the first point cloud data into the first depth image is described. Figure 2As shown, after the background modeling starts, step 201 is executed: point cloud data is collected; step 202: the collected point cloud data is converted into a depth image; step 203: it is determined whether the current frame is the first frame. If it is the first frame, step 204 is executed: the converted depth image is directly used as the background image, and the process returns to step 201. If the current frame is not the first frame, step 205 is executed: the distance value of each point in the current converted depth image (such as the fourth depth image) is compared with the distance value of each point in the background image (such as the third depth image), wherein the distance value of each point in the current depth image can be expressed as Depthmap[r_idx, c_idx], and the distance value of each point in the background image is The distance value can be expressed as Background[r_idx, c_idx]. If Depthmap[r_idx, c_idx]>Background[r_idx, c_idx], execute step 206: make Background[r_idx, c_idx]=Depthmap[r_idx, c_idx], that is, use the distance value in the current depth image to replace the distance value in the background image at the corresponding position. Step 207: Determine whether the number of frames of the collected point cloud data is sufficient. If not, return to step 201. If it is sufficient, for example, when the number of frames of the collected point cloud data reaches a preset number of frames, the background modeling is completed.

[0079] Figure 3 is a schematic diagram of a foreground extraction device according to one or more embodiments of the present invention. Figure 3 As shown, the device 30 comprises:

[0080] The first acquisition module 31 is configured to acquire first point cloud data for the framing area when the target object does not appear in the framing area;

[0081] A first conversion module 32 is configured to convert the first point cloud data to obtain a first depth image;

[0082] A second acquisition module 33 is configured to acquire second point cloud data for the framing area when the target object appears in the framing area;

[0083] A second conversion module 34 is configured to convert the second point cloud data to obtain a second depth image;

[0084] The determination module 35 is configured to compare each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determine the foreground point cloud in the second point cloud data according to the comparison result.

[0085] In one or more embodiments of the present invention, the determination module can be specifically configured to: calculate the difference between the pixel value of each pixel in the second depth image and the pixel value of the corresponding pixel in the first depth image; determine that the pixel in the second depth image corresponding to the difference is a foreground point when the difference is greater than a threshold; determine that the pixel in the second depth image corresponding to the difference is a background point when the difference is not greater than the threshold; convert the foreground point into a point cloud to obtain a foreground point cloud in the second point cloud data.

[0086] In one or more embodiments of the present invention, the foreground extraction device may further include:

[0087] The establishment module is configured to establish a mixed Gaussian model for each pixel in the first depth image after obtaining the first depth image through conversion according to the first point cloud data, so as to obtain a mixed Gaussian background model corresponding to each pixel; the determination module is specifically configured to: match each pixel in the second depth image with the mixed Gaussian background model of the corresponding pixel in the first depth image; determine the points in the second depth image that match the mixed Gaussian background model of the corresponding pixel in the first depth image as background points, and the unmatched points as foreground points; convert the foreground points into a point cloud to obtain the foreground points in the second point cloud data.

[0088] In one or more embodiments of the present invention, the foreground extraction device may further include: a filtering module configured to: after determining the foreground point cloud in the second point cloud data, perform radius filtering on the foreground point cloud.

[0089] In one or more embodiments of the present invention, the foreground extraction device may further include: a clustering module, configured to: after determining the foreground point cloud in the second point cloud data, cluster the point cloud to obtain at least two types of point clouds; a segmentation module, configured to segment the at least two types of point clouds to obtain at least two independent point clouds.

[0090] In one or more embodiments of the present invention, the foreground extraction device may further include: a calculation module configured to: after obtaining at least two types of independent point clouds, calculate the center of mass and size information of the target corresponding to each independent point cloud.

[0091] In one or more embodiments of the present invention, the foreground extraction device may further include: the first point cloud data includes multiple frames of point cloud data, and the conversion module is configured to: after collecting a frame of point cloud data, convert the frame of point cloud data into a third depth image; convert the collected new frame of point cloud data into a fourth depth image; compare the first distance value of each point in the fourth depth image with the second distance value of each point in the third depth image, and if the first distance value is greater than the second distance value, use the first distance value to replace the second distance value in the third depth map; continue to collect a new frame of point cloud data until a preset number of frames of point cloud data have been collected.

[0092] One or more embodiments of the present invention also provide an electronic device, comprising: a housing, a processor, a memory, a circuit board and a power supply circuit, wherein the circuit board is placed inside the space enclosed by the housing, and the processor and the memory are arranged on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the electronic device; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute any one of the foreground extraction methods mentioned above.

[0093] One or more embodiments of the present invention further provide a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable the computer to execute any one of the foreground extraction methods described above.

[0094] Correspondingly, such as Figure 4 As shown, one or more embodiments of the present invention further provide an electronic device, which may include: a shell 41, a processor 42, a memory 43, a circuit board 44 and a power supply circuit 45, wherein the circuit board 44 is arranged inside the space enclosed by the shell 41, and the processor 42 and the memory 43 are arranged on the circuit board 44; the power supply circuit 45 is used to supply power to various circuits or devices of the server; the memory 43 is used to store executable program codes; the processor 42 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 43, so as to execute any foreground extraction method provided in the aforementioned embodiments.

[0095] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0096] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0097] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0098] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0099] In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0100] For the convenience of description, the above device is described by dividing the functions into various units / modules. Of course, when implementing the present invention, the functions of each unit / module can be implemented in the same or multiple software and / or hardware.

[0101] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0102] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A foreground extraction method, characterized in that: include: When the target object does not appear in the framing area, collecting first point cloud data for the framing area; A first depth image is obtained by converting the first point cloud data, and a mixed Gaussian model is established for each pixel in the first depth image to obtain a mixed Gaussian background model corresponding to each pixel: P\left ( {{\chi}_{i}} \right )={{\sum{}}^{K}_{k=1}}{\lambda}_{i,k}Norm\left [ {{\mu}_{i,k},{\Sigma}_{i,k}} \right ] ; Among them, λ i,k Represents the weight of the kth Gaussian component of the mixed Gaussian model of xi; μ i,k represents the mean of the kth Gaussian component of the mixed Gaussian model of xi; ∑ i,k Represents the variance of the kth Gaussian component of the mixed Gaussian model of xi; When the target object appears in the framing area, collect second point cloud data for the framing area; obtain a second depth image according to the conversion of the second point cloud data; compare each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determine the foreground point cloud in the second point cloud data according to the comparison result; match each pixel in the second depth image with a mixed Gaussian background model of the corresponding pixel in the first depth image; determine that the points in the second depth image that match the mixed Gaussian background model of the corresponding pixel in the first depth image are background points, and the points that do not match are foreground points; convert the foreground points into point clouds to obtain foreground points in the second point cloud data; after determining the foreground point clouds in the second point cloud data, cluster the point clouds to obtain at least two types of point clouds; segment the at least two types of point clouds to obtain at least two independent point clouds; after obtaining at least two independent point clouds, calculate the center of mass and size information of the target corresponding to each independent point cloud.

2. The method according to claim 1, characterized in that Compare each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determine the foreground point cloud in the second point cloud data according to the comparison result, including: calculating the difference between the pixel value of each pixel in the second depth image and the pixel value of each corresponding pixel in the first depth image; determining that the pixel in the second depth image corresponding to the difference is a foreground point when the difference is greater than a threshold; determining that the pixel in the second depth image corresponding to the difference is a background point when the difference is not greater than the threshold; converting the foreground point into a point cloud to obtain a foreground point cloud in the second point cloud data.

3. The method according to claim 1, characterized in that The method further includes: after determining the foreground point cloud in the second point cloud data, performing radius filtering on the foreground point cloud.

4. The method according to any one of claims 1 to 3, characterized in that: The first point cloud data includes multiple frames of point cloud data, and a first depth image is obtained according to the conversion of the first point cloud data, including: after collecting a frame of point cloud data, converting the frame of point cloud data into a third depth image; converting a new frame of point cloud data collected into a fourth depth image; comparing a first distance value of each point in the fourth depth image with a second distance value of each point in the third depth image, if the first distance value is greater than the second distance value, using the first distance value to replace the second distance value in the third depth image; and continuing to collect a new frame of point cloud data until a preset number of frames of point cloud data have been collected.

5. An electronic device, characterized in that: The electronic device comprises: a housing, a processor, a memory, a circuit board and a power supply circuit, wherein the circuit board is placed inside the space enclosed by the housing, and the processor and the memory are arranged on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the electronic device; the memory is used to store executable program codes; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the foreground extraction method described in any one of claims 1 to 4.

6. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the foreground extraction method according to any one of claims 1 to 4.

Images