Connected domain detection method, image segmentation method, device, equipment and storage medium

Abstract

The present invention provides a connected domain detection method, image segmentation method, apparatus, device, and computer storage medium. The connected domain detection method includes the following steps: determining a background image; acquiring a current image and determining a foreground region within the current image based on the background image; determining whether each pixel within the foreground image is a valid point in a predetermined order, and determining a connected domain based on the valid points. The method of the present invention is used for connected domain detection in pedestrian detection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more specifically, to a connected domain detection method, an image segmentation method, an apparatus, a device, and a computer storage medium. Background Art

[0002] Pedestrian detection has long been a hot topic in computer vision research. A key challenge in pedestrian detection is identifying people and obtaining the corresponding connected domains. Currently, when using images captured by RGB color cameras or infrared cameras to identify people and construct connected domains, color thresholds or HOG features are used. This results in highly complex algorithms and a significant computational effort, further limiting other aspects of pedestrian detection.

[0003] In view of this, this invention is proposed. Summary of the Invention

[0004] The purpose of the present invention is to provide a connected domain detection method, device, equipment and storage medium. The method provided by the present invention is based on depth images and is used for connected domain detection in pedestrian detection. Based on the connected domain detection method, the present invention also provides an image segmentation method.

[0005] According to a first aspect of the present invention, a method for detecting connected domains is provided. The method is based on a depth image and comprises the following steps:

[0006] Determine the background image;

[0007] Get the current image and determine the foreground area in the current image based on the background image;

[0008] Determine whether each pixel in the foreground image is a valid point in a preset order, and determine the connected domain based on the valid points.

[0009] Furthermore, the step of determining whether each pixel in the foreground image is a valid pixel comprises the following steps:

[0010] Determine whether the distance between pixels is 0;

[0011] If not, the pixel is a valid point; if so, the pixel is not a valid point.

[0012] Furthermore, the step of determining whether each pixel point in the foreground image is a valid point in a preset order and determining a connected region based on the valid points includes the following steps:

[0013] Determine the first effective point;

[0014] Traversing each pixel point on the edge of the first valid point, determining the pixel points that meet the first preset condition as the in-domain points of the first valid point, marking the in-domain points of the first valid point, the first valid point and its in-domain points form a connected domain, and determining the central pixel point of the connected domain;

[0015] Traversing each pixel point on the edge of the in-domain point of the first valid point, determining the pixel points that meet the first preset condition as the in-domain points of the first valid point, marking the in-domain points of the first valid point, the first valid point and its in-domain points form a connected domain, and determining the central pixel point of the connected domain;

[0016] Determine a new first valid point according to the preset order and repeat the above steps.

[0017] Furthermore, the first preset condition includes the following steps:

[0018] The distance between pixels is not zero;

[0019] And the pixels are not marked;

[0020] And the Euclidean distance between the pixel point and the central pixel point of the connected domain is less than a preset first threshold.

[0021] Furthermore, determining the foreground area in the current image according to the background image includes the following steps:

[0022] Determine whether the distance between pixels in the current image is less than the distance between corresponding pixels in the background image, and determine whether the distance between pixels in the current image is greater than a preset second threshold;

[0023] If so, the pixel is determined to be the foreground area.

[0024] Furthermore, the step of determining the foreground area in the current image based on the background image further includes the following steps:

[0025] If not, the pixel is determined to be a background area, and the pixel corresponding to the pixel in the background image is updated.

[0026] According to a second aspect of the present invention, there is provided an image segmentation method, wherein the method is based on a depth image and comprises the following steps:

[0027] Acquire a connected domain in an image according to any method of the first aspect of the present invention, and determine whether the connected domain is a valid connected domain, if so, segment the connected domain, otherwise, do not segment the connected domain;

[0028] Get the upward concave hull on the outer contour of the valid connected domain to form a concave hull set, and determine whether there is a valid concave hull in the concave hull set. If so, connect the farthest point in the valid concave hull with the ground point corresponding to the farthest point, and split the valid connected domain into subdomains. Otherwise, do not split the connected domain.

[0029] According to a third aspect of the present invention, there is provided an apparatus comprising:

[0030] Memory, for storing computer instructions;

[0031] A processor is configured to call the computer instructions from the memory to execute any one of the methods described in the first and second aspects of the present invention.

[0032] According to a fourth aspect of the present invention, there is provided a device comprising:

[0033] The device according to the third aspect of the present invention.

[0034] According to a fifth aspect of the present invention, a computer-readable storage medium is provided.

[0035] A computer program is stored thereon, and when the computer program is executed by a processor, the method described in any one of the first and second aspects of the present invention is implemented.

[0036] In the connected domain detection method provided by the present invention, the foreground area is a relatively close and moving object in the depth image. Usually, the moving objects include people, animals, and cars. On the one hand, when these objects are close, the length and width dimensions differ greatly. On the other hand, although the length and width dimensions of these objects differ greatly when they are close, the length and width dimensions and their ratios are relatively fixed. Therefore, in pedestrian detection, the subsequent steps after connected domain detection (such as image segmentation) can distinguish people, animals, or cars only by size. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0038] Figure 1 is a block diagram of a hardware configuration provided by an embodiment of the present invention;

[0039] Figure 2 is a flow chart of a connected domain detection method provided by an embodiment of the present invention;

[0040] Figure 3 is an example of connected domain detection provided by an embodiment of the present invention;

[0041] Figure 4 is a flow chart of an image segmentation method provided by an embodiment of the present invention;

[0042] Figure 5 This is an example of image segmentation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention.

[0044] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0045] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0046] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0047] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0048] <Hardware Configuration Example>

[0049] Figure 1 This is a block diagram of the hardware configuration of an electronic device for implementing a connected domain detection method provided in an embodiment of the present application.

[0050] The electronic device 1000 may generally be a smart phone, a tablet computer, a desktop computer, etc.

[0051] The electronic device may include a processor 1100 , a memory 1200 , an interface device 1300 , a communication device 1400 , a display device 1500 , an input device 1600 , a speaker 1700 , and a microphone 1800 , among others.

[0052] The processor 1100 may be a central processing unit (CPU), a microprocessor (MCU), or the like. The memory 1200 may include, for example, a ROM (read-only memory), a RAM (random access memory), or a non-volatile memory such as a hard disk. The interface device 1300 may include, for example, a USB interface, a headphone jack, or the like. The communication device 1400 may be capable of wired or wireless communication. The display device 1500 may be, for example, a liquid crystal display (LCD) or a touch screen display. The input device 1600 may include, for example, a touch screen or a keyboard. A user may input / output voice information via the speaker 1700 and the microphone 1800.

[0053] Despite Figure 1 Multiple devices are shown for the electronic device 1000, but the present application may only involve some of the devices, for example, the electronic device 1000 only involves the memory 1200 and the processor 1100.

[0054] In the embodiment of the present application, the memory 1200 of the electronic device 1000 is used to store instructions, which are used to control the processor 1100 to execute the device monitoring method provided by the embodiment of the present application.

[0055] In the above description, a person skilled in the art can design instructions according to the solution disclosed in this application. How instructions control the operation of a processor is well known in the art and will not be described in detail here.

[0056] <Connected Domain Detection Method Example>

[0057] The embodiment of the present invention provides a method for detecting a connected domain. The execution subject of the method can be Figure 1 The hardware shown in FIG. 1 may also be any other hardware capable of completing the steps in the method. Figure 1 The hardware shown is used as an execution body of the connected domain detection method to illustrate the method provided by the embodiment of the present invention.

[0058] like Figure 2 As shown, an embodiment of the present invention provides a connected domain detection method, the method is based on a depth image, and the method includes the following S10 to S30:

[0059] S10: Determine the background image.

[0060] In this embodiment, multiple frames of depth images can be obtained when the depth camera is turned on, and the background image is determined by a time filtering algorithm. The background image is a distant and fixed object in the depth image (such as buildings, trees, mountains, etc.). The distance here refers to the distance between the object in the image and the depth camera in the real physical space. It should be understood that determining the background image is an initialization operation in this embodiment.

[0061] S20: Acquire the current image and determine the foreground area of ​​the current image according to the background image.

[0062] In this embodiment, the current image refers to the image frame obtained after S10 determines the background image, that is, the current image refers to the image frame obtained after the initialization operation is completed. For example, the first six depth image frames after the depth camera is turned on are used to determine the background image, that is, the first six depth image frames are used for the initialization operation, and the seventh depth image and subsequent depth image frames are the current images, which are used for connected domain detection.

[0063] In this embodiment, the foreground area is an object that is relatively close and moving in the depth image, so the background image determined by the initialization operation can determine the foreground area in the current image.

[0064] In this embodiment, the foreground area is a relatively close and moving object in the depth image. Usually, the moving objects include people, animals, and cars. On the one hand, when these objects are relatively close, the length and width dimensions differ greatly. On the other hand, although the length and width dimensions differ greatly when these objects are relatively close, the length and width dimensions and their ratios are relatively fixed. Therefore, in pedestrian detection, the steps after connected domain detection (such as image segmentation) only need to use size to distinguish between people, animals, or cars. In other words, the connected domain of people can be selected by size.

[0065] S30: judging whether each pixel point in the foreground image is a valid point according to a preset order, and determining a connected domain based on the valid points.

[0066] In this embodiment, the preset order can be from top to bottom column by column or from left to right row by row.

[0067] In this embodiment, there are two situations for the pixel points in the depth image. The first situation is that the depth image has a detection range (for example, a distance of 0.5 to 3 meters). In the real physical space, for objects that exceed this range, the distance detected by the depth image may be 0; the second situation is the edge of the object, and the distance detected by the depth image may be 0. The pixel points in the depth image corresponding to these two situations are not valid points and should not be considered during connected domain detection. That is to say, the connected domain detection is only carried out based on the valid points in the depth image whose distance is not 0.

[0068] As an optional embodiment, the determining whether each pixel point in the foreground image is a valid point includes the following steps:

[0069] S31: Determine whether the distance between the pixels is 0.

[0070] S32: If no, the pixel is a valid point; if yes, the pixel is not a valid point.

[0071] As an optional embodiment, the step of determining whether each pixel point in the foreground image is a valid point in a preset order and determining a connected region based on the valid points includes the following steps:

[0072] S33: Determine the first valid point.

[0073] In this embodiment, the first valid point refers to the valid point based on which the connected domain needs to be determined. Figure 3 As shown, Figure 3 Each square in the figure represents a pixel. For the convenience of explanation, it is assumed that the judgment starts from the first pixel in the upper left corner of the depth image in a preset order, and it is determined that the first pixel in the upper left corner is a valid point. At this time, it is necessary to determine the connected domain based on the first pixel in the upper left corner. That is to say, the first pixel in the upper left corner is the first valid point.

[0074] S34: traverse each pixel point on the edge of the first valid point, determine the pixel points that meet the first preset condition as the in-domain points of the first valid point, and mark the in-domain points of the first valid point. The first valid point and its in-domain points form a connected domain, and determine the central pixel point of the connected domain.

[0075] In this embodiment, the pixel points at the edge of the first effective point refer to the pixel points in a circle around the first effective point. Figure 3 In the example shown, there are 3 pixel points on the edge of the first effective point. Of course, when the first effective point is located in the middle of the image, the number of points on the edge of the first effective point is 8. When the first effective point is located at other positions in the image, the number of points on the edge of the first effective point can also be other numbers.

[0076] S35: Traverse the pixel points on the edge of the in-domain points of the first valid point, determine the pixel points that meet the first preset condition as the in-domain points of the first valid point, and mark the in-domain points of the first valid point. The first valid point and its in-domain points form a connected domain, and determine the central pixel point of the connected domain.

[0077] In this embodiment, Figure 3 In the example shown, assuming that the pixel point below the first valid point is determined as the in-domain point of the first valid point in S34, then in S35, it is necessary to judge the pixel points on the edge of the in-domain point according to the first preset condition to determine whether there is an in-domain point of the first valid point among the pixel points on the edge of the in-domain point. In order to simplify the calculation, the pixel points on the edge of the in-domain point and the first valid point that have been judged in S34 do not need to be judged again in S35.

[0078] S36: Determine a new first valid point according to a preset order, and repeat the above steps S34 and S35.

[0079] In this embodiment, S35 is a repetitive step. As long as a certain pixel point is determined to be an in-domain point of the first valid point, S35 needs to be executed until a new in-domain point of the first valid point cannot be determined. At this time, the connected domain of the first valid point is constructed and S36 is started.

[0080] In this embodiment, Figure 3 In the example shown, assuming that it is determined through S35 that there is only one in-domain point for the first valid point, that is, the pixel point below the first valid point is the in-domain point of the first valid point, then in S36, starting from the third pixel in the first column, or starting from the second pixel in the first row, continue to judge and determine the new first valid point in accordance with the preset order.

[0081] As an optional embodiment, the first preset condition includes the following steps:

[0082] The distance between pixels is not zero;

[0083] And the pixels are not marked;

[0084] And the Euclidean distance between the pixel point and the central pixel point of the connected domain is less than a preset first threshold.

[0085] In this embodiment, the distance of a pixel point refers to the distance between a point corresponding to the pixel point in real physical space and the depth camera.

[0086] In this embodiment, the pixel point is not marked, which means that the pixel point is not a point within the domain of other first valid points.

[0087] As an optional embodiment, determining the foreground area in the current image according to the background image includes the following steps:

[0088] S21: Determine whether the distance between pixels in the current image is smaller than the distance between corresponding pixels in the background image, and determine whether the distance between pixels in the current image is greater than a preset second threshold.

[0089] S22: If yes, the pixel is determined to be a foreground area.

[0090] As an optional embodiment, determining the foreground area in the current image according to the background image includes the following steps:

[0091] S23: If not, the pixel point is determined to be a background area, and the pixel point corresponding to the pixel point in the background image is updated.

[0092] <Image Segmentation Method Example>

[0093] The embodiment of the present invention provides an image segmentation method. The execution subject of the method can be Figure 1The hardware shown in FIG. 1 may also be any other hardware capable of completing the steps in the method. Figure 1 The hardware shown is used as an execution body of the image segmentation method to illustrate the method provided by the embodiment of the present invention.

[0094] like Figure 4 As shown, an embodiment of the present invention provides an image segmentation method, wherein the method is based on a depth image and includes the following S40 to S50:

[0095] S40: Obtain a connected domain in the image and determine whether the connected domain is a valid connected domain. If so, segment the connected domain; otherwise, do not segment the connected domain.

[0096] In this embodiment, the method described in <Connected Component Detection Method Embodiment> can be used to obtain the connected components in the image.

[0097] In this embodiment, a valid connected domain refers to a situation where the connected domain includes multiple pedestrians that are close together or relatively close to each other. In other words, when multiple pedestrians that are close together or relatively close to each other in the depth image are detected as the same connected domain, the connected domain is considered valid. In this case, the valid connected domain needs to be segmented to identify different pedestrians within the valid connected domain, which can then be used for pedestrian detection. Specifically, whether a connected domain is valid can be determined based on the height and width range of a single person.

[0098] S50: Obtain the upward concave hull on the outer contour of the valid connected domain to form a concave hull set, and determine whether there is a valid concave hull in the concave hull set. If so, connect the farthest point in the valid concave hull with the ground point corresponding to the farthest point, and divide the valid connected domain into subdomains. Otherwise, do not divide the connected domain.

[0099] In this embodiment, if Figure 5 As shown in the figure, when the connected domain includes multiple pedestrians, there is usually a concave hull between pedestrians. Therefore, a valid concave hull (a valid concave hull refers to a concave hull that meets the size between pedestrians) can be selected to divide the effective connected domain into several subdomains. Different subdomains represent different pedestrians in the effective connected domain. In other words, by dividing the effective connected domain through the effective concave hull, different pedestrians in the effective connected domain can be divided.

[0100] In this embodiment, the ground point corresponding to the farthest point refers to a point on the ground directly below the farthest point in the real physical space in the depth image.

[0101] As an optional embodiment, the determining whether the connected domain is a valid connected domain includes the following steps:

[0102] S41: Determine whether the height of the connected domain is greater than a preset third threshold, and determine whether the width of the connected domain is greater than a preset fourth threshold.

[0103] S42: If yes, the connected domain is a valid connected domain; otherwise, the connected domain is not a valid connected domain.

[0104] In this embodiment, the height of the connected domain refers to the maximum value of the height of the connected domain, and the width of the connected domain refers to the maximum value of the width of the connected domain.

[0105] As an optional embodiment, obtaining the upward concave hull on the outer contour of the valid connected domain to form a concave hull set includes the following steps:

[0106] S51: Obtain the concave hull on the outer contour of the valid connected domain.

[0107] In this embodiment, the findContours function in opencv can be used to find the outer contour of the connected domain, and the convexHull function and convexityDefect function in opencv can be used to obtain the concave hull on the outer contour of the connected domain.

[0108] S52: Classify the concave hulls on the outer contour of the effective connected domain that meet the second preset condition into a concave hull set.

[0109] The second preset condition is:

[0110] (starting point.x<farthest point.x &&farthest point.x<ending point.x)||(ending point.x<farthest point.x &&farthest point.x<starting point.x);

[0111] And (starting point.y<farthest point.y&&ending point.y<farthest point.y).

[0112] In this embodiment, if Figure 5 As shown, Figure 5 The effective connected area 7 is included. Usually, the vertex in the upper left corner of the depth image is the origin of the coordinate system, the horizontal direction in the depth image is the x direction, and the vertical direction in the depth image is the y direction.

[0113] In this embodiment, if Figure 5 As shown, the concave hull usually includes four features, the starting point 4, the ending point 5, the farthest point 6 and the depth, where the starting point 4 and the ending point 5 refer to the two end points of the concave hull, the farthest point 6 refers to the point at the corner of the concave hull, and the depth refers to the distance from the farthest point 6 to the line connecting the starting point 4 and the ending point 5.

[0114] In this embodiment, starting point.x and y refer to the x and y values ​​of the starting point in the coordinate system of the depth image, ending point.x and y refer to the x and y values ​​of the ending point in the coordinate system of the depth image, and farthest point.x and y refer to the x and y values ​​of the farthest point in the coordinate system of the depth image.

[0115] In this embodiment, the upward concave hull usually has the following two characteristics: the first characteristic is that the x values ​​of the starting point, the farthest point, and the end point increase or decrease in sequence; the second characteristic is that the y value of the farthest point is greater than the y values ​​of the starting point and the end point. Therefore, the concave hull set can be determined and formed through S52.

[0116] As an optional embodiment, determining whether there is a valid concave hull in the concave hull set includes the following steps:

[0117] S53: Acquire the concave hulls in the concave hull set that meet the third preset condition to form a preselected concave hull set.

[0118] S54: Select a valid concave hull from the pre-selected concave hull set.

[0119] Among them, the third preset condition is:

[0120] The Euclidean distance between the starting point and the ending point of the concave hull is greater than a preset fifth threshold;

[0121] and the Euclidean distance between the starting point and the farthest point of the concave hull is greater than a preset sixth threshold;

[0122] And the Euclidean distance between the end point of the concave hull and the farthest point is greater than the preset seventh threshold.

[0123] In this embodiment, smaller concave hulls in the concave hull set are deleted in advance according to the third preset condition to form a preselected concave hull set, and then valid concave hulls are selected from the preselected concave hull set to segment the valid connected domain into several subdomains.

[0124] As an optional embodiment, the selecting of a valid concave hull from the preselected concave hull set includes the following steps:

[0125] S541: Select the concave hull with a depth greater than a preset eighth threshold as a valid concave hull.

[0126] As an optional embodiment, the selecting of a valid concave hull from the preselected concave hull set includes the following steps:

[0127] S542: Select the concave hull with the largest depth as the valid concave hull.

[0128] The method further comprises the steps of:

[0129] S60: Determine whether the subdomain is a valid connected domain. If so, segment the subdomain. If not, do not segment the subdomain.

[0130] In this embodiment, the valid concave hull selected in S542 can be used to split the valid connected domain into two subdomains. The subdomains are then further evaluated. If a subdomain is still a valid connected domain, segmentation is continued until the last segmented subdomain is no longer a valid connected domain. Thus, in this embodiment, the final result is achieved through multiple segmentations, rather than a single complete segmentation of the valid connected domain determined in S40. Therefore, compared to the previous embodiment, this embodiment does not require the eighth threshold value, resulting in a higher segmentation accuracy.

[0131] As an optional embodiment, the determining whether the subdomain is a valid connected domain includes the following steps:

[0132] S61: Determine whether the width of the subdomain is greater than a fourth threshold;

[0133] S62: If yes, the subdomain is a valid connected domain; otherwise, the subdomain is not a valid connected domain.

[0134] <Device Example>

[0135] An embodiment of the present invention provides a device comprising: a memory and a processor,

[0136] The memory is used to store computer instructions;

[0137] The processor is configured to call the computer instructions from the memory to execute the method as described in any one of the above method embodiments.

[0138] <Equipment Example>

[0139] An embodiment of the present invention provides a device including:

[0140] The device described in the above device embodiment.

[0141] <Storage Medium Embodiment>

[0142] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in any one of the above method embodiments is implemented.

[0143] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0144] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0145] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0146] The computer program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), is personalized by utilizing the state information of the computer readable program instructions, and the electronic circuit can execute the computer readable program instructions, thereby realizing various aspects of the present invention.

[0147] Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0148] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0149] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0150] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of an instruction, and the module, program segment or part of the instruction contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.

[0151] While various embodiments of the present invention have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A connected domain detection method, characterized in that: The method is based on a depth image and comprises the following steps: Determine the background image; Get the current image and determine the foreground area in the current image based on the background image; Determine whether each pixel in the foreground image is a valid point in a preset order, and determine the connected domain based on the valid points; The process of determining whether each pixel in the foreground image is a valid point in a preset order and determining a connected region based on the valid points includes the following steps: Determine the first effective point; Traversing each pixel point on the edge of the first valid point, determining the pixel points that meet the first preset condition as the in-domain points of the first valid point, marking the in-domain points of the first valid point, the first valid point and its in-domain points form a connected domain, and determining the central pixel point of the connected domain; Traversing each pixel point on the edge of the in-domain point of the first valid point, determining the pixel points that meet the first preset condition as the in-domain points of the first valid point, marking the in-domain points of the first valid point, the first valid point and its in-domain points form a connected domain, and determining the central pixel point of the connected domain; Determine a new first valid point according to the preset order and repeat the above steps; The first preset condition includes the following steps: The distance between pixels is not zero; And the pixels are not marked; And the Euclidean distance between the pixel point and the central pixel point of the connected domain is less than a preset first threshold.

2. The method according to claim 1, characterized in that The method of determining whether each pixel in the foreground image is a valid pixel comprises the following steps: Determine whether the distance between pixels is 0; If not, the pixel is a valid point; if so, the pixel is not a valid point.

3. The method according to claim 1, characterized in that Determining the foreground area in the current image according to the background image comprises the following steps: Determine whether the distance between pixels in the current image is less than the distance between corresponding pixels in the background image, and determine whether the distance between pixels in the current image is greater than a preset second threshold; If so, the pixel is determined to be the foreground area.

4. The method according to claim 3, characterized in that Determining the foreground area in the current image according to the background image further comprises the following steps: If not, the pixel is determined to be a background area, and the pixel corresponding to the pixel in the background image is updated.

5. An image segmentation method, characterized in that: The method is based on a depth image. The following steps are involved: Acquire a connected domain in an image according to the method of any one of claims 1 to 4, and determine whether the connected domain is a valid connected domain, if so, segment the connected domain, otherwise, do not segment the connected domain; Get the upward concave hull on the outer contour of the valid connected domain to form a concave hull set, and determine whether there is a valid concave hull in the concave hull set. If so, connect the farthest point in the valid concave hull with the ground point corresponding to the farthest point, and split the valid connected domain into subdomains. Otherwise, do not split the connected domain.

6. A device, characterized in that include: Memory, for storing computer instructions; A processor, configured to call the computer instructions from the memory to execute the method according to any one of claims 1 to 4 or 5.

7. A device, characterized in that include: The device of claim 6.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 or 5 is implemented.

Images