Method, device and storage medium for detecting a type of a door

Abstract

The present disclosure provides a method, device and computer readable storage medium for detecting the type of a door. The method comprises: obtaining a plurality of door images of a door in different states; extracting door features associated with the door from the plurality of door images respectively; and calculating the difference between the door features extracted from the plurality of door images respectively, and determining the type of the door based on the difference. According to the door type detection method provided by the present disclosure, by analyzing a plurality of door images taken in different states of the door to calculate the difference between the corresponding door features in these door images, the type of each door in a house can be automatically identified, and the detection result of the door type can be applied to the generation of a house plan, thereby avoiding the high labor cost caused by manually drawing door icons and improving the efficiency of house plan generation.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more specifically, to methods, apparatus, and computer-readable storage media for detecting the type of a door based on image processing. Background Technology

[0002] In recent years, online real estate, digital home decoration and design services have experienced significant growth. To enable virtual real estate tours, it's typically necessary to generate a complete floor plan (or unit layout) of the entire house so customers can filter units online. For example, a house may include various types of doors, such as hinged doors, sliding doors, and folding doors. Customers can view each door online, noting its location and using icons to understand its opening direction and method, thus gaining a better understanding of the room's structure and function. However, current methods for generating floor plans rely on manually drawn door icons, which is insufficient to meet the growing demands of the online real estate business.

[0003] On the other hand, image processing technology has wide applications in object detection; for example, it can analyze images of doors to detect them. However, most existing detection methods aim to detect whether an object in an image is a door or another object, without addressing the distinction between door types. Overall, there is currently a lack of door type detection technology applicable to the generation of house floor plans.

[0004] Therefore, there is a need for a door type detection technology based on image processing to automatically identify the type of door and apply it to the generation of house floor plans. Summary of the Invention

[0005] According to one aspect of this disclosure, a method for detecting the type of a door is provided. The method includes: acquiring multiple door images of doors in different states; extracting door features associated with the door from each of the multiple door images; and calculating differences between the door features extracted from the multiple door images, and determining the type of the door based on the differences.

[0006] According to another aspect of this disclosure, an apparatus for detecting the type of a door is provided. The apparatus includes a processor and a memory. The memory stores computer program instructions, wherein, when executed by the processor, the processor performs the following steps: acquiring multiple door images of doors in different states; extracting door features associated with the door from each of the multiple door images; and calculating differences between the door features extracted from the multiple door images, and determining the type of the door based on the differences.

[0007] According to another aspect of this disclosure, an apparatus for detecting the type of a door is provided. The apparatus includes: an image acquisition unit configured to acquire multiple door images of the door in different states; a feature extraction unit configured to extract door features associated with the door from the multiple door images; and a door type determination unit configured to calculate differences between the door features extracted from the multiple door images and determine the type of the door based on the differences.

[0008] According to another aspect of this disclosure, a computer-readable storage medium is provided having computer program instructions stored thereon, wherein the computer program instructions, when executed, perform the following steps: acquiring multiple door images of doors in different states; extracting door features associated with the doors from the multiple door images respectively; and calculating the differences between the door features extracted from the multiple door images respectively, and determining the type of the door based on the differences.

[0009] According to the door type detection technology provided in the above aspects of this disclosure, by analyzing multiple door images taken in different states of the door in order to calculate the differences in the corresponding door features between these door images, the type of each door in the house can be automatically identified, and the detection results of the door type can be applied to the generation of house floor plans, thereby avoiding the high labor costs caused by manually drawing door icons and improving the efficiency of house floor plan generation. Attached Figure Description

[0010] These and / or other aspects and advantages of this disclosure will become clearer and more readily understood from the following detailed description of embodiments of this disclosure taken in conjunction with the accompanying drawings, wherein:

[0011] Figure 1 A schematic diagram of an exemplary house floor plan is shown.

[0012] Figure 2 A flowchart of an exemplary method for detecting the type of a room door according to an embodiment of this disclosure is shown.

[0013] Figure 3 A schematic diagram is shown of multiple door images in different states acquired in a method for detecting the type of a door according to an embodiment of the present disclosure.

[0014] Figure 4 The diagram illustrates an exemplary process in which door features associated with a door are extracted from multiple door images in a method for detecting the type of a door according to an embodiment of the present disclosure.

[0015] Figure 5AThe illustration shows an exemplary process in a method for detecting the type of a door according to an embodiment of the present disclosure, in which multiple door images are compared for image differences and thresholded to obtain a thresholded fused image.

[0016] Figure 5B A schematic diagram of an exemplary process for segmenting a door image based on a thresholded fusion image is shown in a method for detecting the type of a door according to an embodiment of the present disclosure.

[0017] Figure 6 A schematic diagram illustrates an exemplary process for edge feature extraction from door open and door closed images in a method for detecting the type of a room door according to an embodiment of the present disclosure.

[0018] Figure 7 The illustration shows an exemplary process for extracting door features from an input image from each channel of a plurality of door images in a method for detecting the type of a door according to an embodiment of the present disclosure.

[0019] Figure 8 The illustration shows a schematic process for determining the type of a door based on differences between door features extracted from multiple door images in a method for detecting the type of a door according to an embodiment of the present disclosure.

[0020] Figure 9 A schematic hardware block diagram of a device for detecting the type of a room door according to an embodiment of the present disclosure is shown.

[0021] Figure 10 A schematic structural block diagram of a device for detecting the type of a room door according to an embodiment of the present disclosure is shown. Detailed Implementation

[0022] To enable those skilled in the art to better understand this disclosure, the disclosure will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] First, a brief overview of the basic idea of ​​the door type detection technology disclosed herein is provided. Considering that doors within a house can be in various states during opening / closing, and that the same feature of a door can exhibit differences between these states, these differences are typically closely related to and unique to that door type. Therefore, this disclosure analyzes multiple images of doors in different states, extracting and calculating the differences in the same door feature between these images. This allows for the determination of the door type based on the correlation between the observed differences and the specific door type. According to the embodiments of this disclosure, the door type detection technology, by analyzing multiple door images taken in different states to calculate the differences in corresponding door features between these images, can automatically identify the types of doors within a house. Furthermore, the detection results can be applied to the generation of house floor plans, thereby avoiding the high labor costs associated with manually drawing door icons and improving the efficiency of house floor plan generation.

[0024] For ease of explanation, the door type detection technique of this disclosure is described herein using the scenario of generating house floor plans in the online real estate field as an example. For example, Figure 1 A schematic diagram of an exemplary house floor plan is shown, illustrating that this single house includes nearly ten doors, and these doors may be of various different types. For example, Figure 1 The left side shows one of the sliding doors in the house floor plan. In this disclosure, sliding doors are defined from the perspectives of the sliding direction of the door leaf, the left-right / inside-outside positional relationship of the sliding door leaf relative to the fixed door leaf, etc. (e.g., left inward sliding door, right inward sliding door, left outward sliding door, right outward sliding door, etc.). Figure 1 (The left side is shown from top to bottom). For example, a left-inward sliding door can refer to a door type where, when the door is closed, the sliding door leaf is to the left of the fixed door leaf, and as the sliding door leaf slides to the right, it slides behind the fixed door leaf (therefore, the sliding door leaf is on the inside relative to the fixed door leaf). As another example, Figure 1 The right side shows one of the hinged doors in the house floor plan. In this disclosure, hinged doors are defined from the perspectives of the opening direction, the direction of rotation of the door leaf around the hinge, and the pushing and pulling method, etc., as: inward left-turning hinged door, inward right-turning hinged door, outward left-turning hinged door, and outward left-turning hinged door, etc. (e.g.) Figure 1 (As shown from left to right on the right side). For example, an inward-turning left-hinged door can refer to a type of door that needs to be opened by pushing inwards and whose leaf rotates to the left around the hinge during opening.

[0025] As discussed above, for a small number of doors or houses, it's possible to manually draw door icons for each door to reflect its specific door type. However, for the ever-growing online real estate business, such as the need to generate floor plans for hundreds of thousands of houses, this manual door icon drawing method is clearly inefficient and labor-intensive. The door type detection technology disclosed herein can automatically determine the door type of each door by analyzing a given set of images taken under different door conditions. This allows for the automatic insertion of the corresponding door type icon when generating the house floor plan. For example, it can automatically draw... Figure 1 The illustration shows one or more of the four types of sliding door icons and four types of hinged door icons. It should be noted that the sliding doors and hinged doors discussed above are merely examples of door types that the door type detection technology of this disclosure can detect, and are not intended to limit the door type detection technology of this disclosure. Furthermore, the examples of the four specific door types of sliding doors and hinged doors in this disclosure are merely illustrative examples; sliding doors and hinged doors can be given other names or more detailed classifications from many other perspectives, and this disclosure is not limited thereto.

[0026] It should be noted that the application of the door type detection technology disclosed herein in online real estate business is merely an example and not a limitation of this disclosure. For instance, it has a wide range of applications, such as digital home decoration and design, virtual tours, urban planning, and various other fields. Accordingly, the door type detection technology disclosed herein can be used to automatically identify the door type in images captured in corresponding scenarios, thereby applying the automatic identification results to any desired scenario. The following references... Figures 2-8 This invention describes a method for detecting the type of a room door according to embodiments of the present disclosure.

[0027] Figure 2 A flowchart illustrating an exemplary method for detecting the type of a room door according to an embodiment of this disclosure is shown. Figure 2 As shown, in step S101, multiple door images in different states are acquired. In this step S101, multiple door images can be obtained by taking pictures of the doors in different states within the house. For example, the state of the door can refer to the state of the door being open / closed. For instance, as the door is opened or closed, it can be in a fully open state, a fully closed state, a slightly open state, a half-open state, a nearly fully open state, etc. By analyzing multiple door images covering various door states, it is helpful to understand how a certain door feature will show differences during the opening / closing process, thereby helping to determine the door type based on the manifestation of these differences.

[0028] In the embodiments of this disclosure, doors can be photographed in any suitable manner, as long as the doors have different states in the images captured. For example, each door can be photographed as a single object, with individual images captured when it is closed and open. The same processing can be applied to the remaining doors in the house to obtain individual images of those doors. Alternatively, to avoid differences in camera placement, shooting angle, and shooting distance that might arise when photographing each door individually, all doors in the house can be photographed. A camera can be placed at a certain location within the house, and its shooting angle adjusted. A panoramic image can be captured when all doors are open, and another panoramic image can be captured when all doors are closed. Panoramic photography reduces changes in shooting angle during the two shooting sessions, thus avoiding the need for subsequent processing such as image alignment required when photographing each door individually. This reduces the complexity of image processing in the door detection method and lowers the manpower cost during photography. The following is combined with... Figure 3 Examples of different door images taken in different states.

[0029] Figure 3 This diagram illustrates multiple door images in different states acquired during a method for detecting the type of a door according to an embodiment of the present disclosure. Figure 3 As shown in the image above, it is possible to capture an image of the door opening even when the door is in the open position, and as... Figure 3 As shown in the image below, an image of a closed door can be captured. In this example, panoramic images are captured in both door states. It is understandable that, although... Figure 3 The illustration shows three doors in each of the open and closed door images. However, the number of doors mentioned above is merely illustrative; the open and closed door images in this disclosure may include more or fewer doors, for example, each image may include only one door. It can be seen that the same door appears to be of substantially the same size and located in substantially the same position in different panoramic door images, thus avoiding the need for image alignment, rotation, and scaling operations. It is understood that although the following description primarily uses open and closed door states as examples to illustrate multiple different door images acquired when the door is in different states, the door images in this disclosure may also include more door images captured when the door is in other states, such as slightly open, half-open, nearly fully open, etc.

[0030] return Figure 2In step S102, door features associated with the door are extracted from the multiple door images. As discussed above, the door to be detected is in different states (e.g., open / closed) in the multiple door images. As the door is opened or closed, certain features of the door will exhibit different shapes, positions, sizes, etc., in different door images. The specific manifestation of the differences in these features between different door images can be unique to that type of door. Extracting these features will help in determining the door type. Therefore, in step S102, the extracted door features can be features related to the shape, position, size, etc., of the door leaf or door hardware. These features will exhibit a certain degree of difference due to the opening or closing action of the door. By reflecting the door's unique characteristics such as opening direction and opening method through these differences, the type of door can be inferred. For example, for an inward-turning left-hinged door, as it is opened, the door handle will show a leftward shift in position between the closed and open images; similarly, for an inward-turning right-hinged door, the door handle will show a rightward shift in position between the closed and open images. This positional difference related to the direction of the door handle shift is closely related to the door type. Therefore, extracting the door handle features and their positional differences across multiple door images is helpful in inferring the door type. It is understood that the door handles mentioned above are merely illustrative examples of various features extracted in this disclosure, and this disclosure is not limited thereto. In the embodiments of this disclosure, various methods can be used to extract door features associated with the door and helpful in determining the door type. The following, in conjunction with... Figure 4 This section describes an example of a schematic process for extracting door features from a door image.

[0031] Figure 4 This diagram illustrates an exemplary process in a method for detecting the type of a room door according to an embodiment of the present disclosure, in which door features associated with the room door are extracted from multiple room door images. Figure 4 As shown, firstly, the door region can be segmented from the door-open and door-closed images respectively, thereby determining the area where the door may be located. This reduces the computational load and resource consumption of subsequent image processing. For example, three door regions can be extracted from the original panoramic image. Then, door features associated with the door can be extracted from each door region in the door-open and door-closed images, thereby determining the door type based on these door features and their differences between different door states. Alternatively, door features associated with the door can be extracted directly from the entire door image without prior image segmentation processing; this disclosure does not limit this process.

[0032] In the embodiments of this disclosure, various methods can be used to segment the door region from door-open and door-closed images. As an illustrative example, the door region can be segmented from the entire image as a region of interest based on features such as grayscale, color, spatial texture, and geometric shape. Furthermore, the inventors of this disclosure recognize that the purpose of door segmentation is primarily to exclude content other than the door from the entire image. After obtaining multiple door images in different states, the door portion can be naturally highlighted by comparing the differences between these images. Segmentation can then be performed using the comparison results to obtain the region of interest including the door. Therefore, this disclosure proposes a door segmentation process based on image difference comparison and multiple thresholding. The segmentation of the door region is based on the pixel distribution in the thresholding fusion result, thereby effectively obtaining each door region including the door to be detected. For the completeness of this disclosure, the following description is combined with… Figure 5A and Figure 5B This section describes an illustrative example of door segmentation based on image difference comparison and multiple thresholding techniques.

[0033] Figure 5A This illustration shows an exemplary process in a method for detecting the type of a room door according to an embodiment of the present disclosure, which involves comparing image differences and thresholding multiple room door images to obtain a thresholded fused image. The following description uses an example of multiple room door images including door-open and door-closed images. Figure 5A As shown, firstly, the image differences between the open and closed door images are compared to calculate the door difference image between them. For example, the door difference image can be obtained by calculating the pixel differences between the open and closed door images pixel by pixel. Optionally, before calculating the difference image between the open and closed door images, both images can be downsampled to reduce computation. Figure 5A As can be seen in the door difference image shown, the door portion is highlighted in the door difference image, thus distinguishing it from features outside the door.

[0034] Then, the door difference image is subjected to fixed thresholding and adaptive thresholding. Specifically, the thresholding process involves comparing each pixel value of the image to be processed with a pixel threshold to obtain a binarized image. Fixed thresholding is the process of binarizing the entire image using the same fixed threshold, while adaptive thresholding is the process of obtaining local pixel thresholds based on the brightness of different regions of the image and binarizing different regions using local thresholds. The inventors of this disclosure have noted that by using local thresholding for binarization, even for house images with uneven lighting, adaptive thresholding can obtain relatively accurate binarization results. At the same time, compared to fixed thresholding, adaptive thresholding can also obtain more accurate binarization results for local detail image information, thereby providing an accurate distinction between the target door area of ​​interest and the background image as a whole. However, in actual shooting, considering the difference between taking two shots with the door open and closed, the camera's shooting parameters, such as exposure and white balance, will change to some extent. Therefore, the two door images will have a certain degree of color difference. Adaptive thresholding takes into account local brightness and uses multiple local thresholds, reflecting the color differences caused by these changes in camera shooting parameters in its thresholding result. For example... Figure 5A The dashed box shows the result of adaptive thresholding, which includes unwanted image information from the NOT gate. Therefore, to leverage the good performance of adaptive thresholding under uneven illumination conditions and accurate binarization of local thinning features, while removing noise introduced by color differences as described above, this disclosure employs the idea of ​​multiple thresholding to obtain a relatively accurate overall binarization result, as specifically described below.

[0035] Finally, the results of fixed thresholding and adaptive thresholding are fused to generate a thresholded fused image. As an illustrative example, for instance... Figure 5A The coded implementation of fusing multiple thresholding results illustrates that the result of adaptive thresholding can be assigned to most pixels in the thresholded fused image. Only when some pixels appear as background in the result of fixed thresholding are the results of adaptive thresholding not applied (i.e., indicating that adaptive thresholding introduces color difference noise at these pixel locations). Instead, the result of fixed thresholding is applied to these pixels to remove color difference noise. Through the fusing of multiple thresholding results in this embodiment, the thresholded fused result can be made to contain, or minimize, content other than doors, making it simpler to detect and segment doors from multiple door images.

[0036] Figure 5B A schematic diagram illustrates an exemplary process of segmenting a door image based on a thresholded fusion image in a method for detecting the type of a door according to an embodiment of this disclosure. Figure 5B As shown, after obtaining a thresholded fused image excluding content other than the door, the door region can be segmented from the door-open and door-closed images based on the pixel distribution of the thresholded fused image. For example, by analyzing the pixel distribution of the thresholded fused image, the horizontal and vertical coordinate ranges that may correspond to the door can be roughly determined. For the completeness of this disclosure, embodiments of this disclosure may employ horizontal and vertical projection methods to segment one or more door regions that may cover the door from multiple door images. Figure 5B As shown, horizontal projection is used to separate all doors from the ceiling and floor, and vertical projection is used to separate all doors from the walls, thus detecting three door zones that may cover the doors through a combination of horizontal and vertical projection.

[0037] Then, after roughly identifying three door regions from the thresholded fused image, segmentation can be performed at the corresponding regions in the closed and open door images to obtain three corresponding door regions for subsequent feature extraction. The inventors also noted that corresponding door features can be extracted from the door difference image, and these extracted features can be used to assist in determining the door type. Therefore, optionally, in this embodiment, the three corresponding door regions can also be segmented from the door difference image based on the pixel distribution of the thresholded fused image. It should be noted that the door region segmentation in this embodiment is not intended to accurately segment the region of each door from the original door image, but rather to determine the region that covers both closed and open doors; therefore, each segmented door region is typically larger than the door frame region.

[0038] Those skilled in the art will understand that after segmenting the corresponding door regions from the corresponding door images, various feature extraction methods can be used to extract door-related features from each door region, such as color, texture, and edge feature extraction methods. However, the inventors of this disclosure have found that edge features perform well in terms of both the ease of feature extraction and the effectiveness of features in distinguishing door types. Furthermore, edge features are well-suited for detecting objects with regular shapes, such as doors. Therefore, in the embodiments of this disclosure, edge features can be extracted from the door regions of both the door-open and door-closed images to construct corresponding input images containing edge features for door classification. For example, edge extraction algorithms such as the Canny operator, Sobel operator, and Laplacian operator can be used to process the door regions to extract door edge features from each region, thereby constructing various input images for detecting door types. In addition to the various edge extraction algorithms mentioned above, the embodiments of this disclosure also propose an edge feature extraction method based on fuzzing processing. Specifically, for either the door open image or the door closed image, the corresponding segmented door region can be blurred, and the edge of each door region can be obtained by calculating the difference between the original segmented door region and the blurred door region.

[0039] The inventors also noted that since walls and furniture inside a house also have some color texture, when extracting edge features from the door area in open and closed door images, these color textures are sometimes also detected as edges. This results in noise in the edge detection of the door area in open and closed door images, which is detrimental to subsequent detection of door features such as door leaf and / or door hardware, as well as the identification of door type. In view of this, in a preferred embodiment of this disclosure, denoising processing can be performed based on the edge detection results of the door difference image to eliminate noise related to the color texture of walls and furniture from the edge detection results of open and closed door images.

[0040] Figure 6 A schematic diagram illustrates an exemplary process for edge feature extraction from door-open and door-closed images in a method for detecting the type of a room door according to an embodiment of this disclosure. Figure 6 As shown, firstly, in accordance with the above references Figure 5A and Figure 5B Similarly, as described, a door difference image can be calculated between an open door image and a closed door image, and then the door region can be segmented from the door difference image, such as... Figure 6 As shown, the three door regions can be segmented from the door difference image in the same way. Then, as... Figure 6As shown in the solid-line box, edge detection can be performed separately in the three door regions of both the door-open and door-closed images. Figure 6 As shown in the dashed box, edge detection can also be performed from the three door regions of the door difference image to denoise the edge detection results of the other two.

[0041] Then, based on the edge detection results of the door region in the door difference image, noise can be denoised from the edge detection results of the door open image and the door closed image. In this disclosure, considering that the door difference image is obtained by comparing image differences between the door open image and the door closed image (e.g., image subtraction), color textures related to walls and furniture inside the house are also eliminated in the subtraction operation. Therefore, features that are mistakenly detected as edges but are actually furniture textures can be removed from the original door open image and door closed image. For example, as... Figure 6 As shown, the edge detection results of the three door regions in the door difference image can be used as a reference to remove noise from the edge detection results of the three corresponding door regions in the door open and door closed images. For example, the edge detection results of the three door regions in the door difference image can be used as a filter for denoising. As an illustrative example, for instance... Figure 6 The coded implementation of the filter illustrates that, for any one of the edge detection results of the three door regions in both the open and closed door images, the original edge detection result of the door region can be retained at most pixel locations. However, if some pixels appear as non-edges in the edge detection result of the door difference image but as edges in the open and / or closed door images (i.e., these pixels may correspond to the color texture of walls and furniture inside the house), then the original edge detection results of these pixels should be reclassified as non-edge pixels, thereby removing noise from the edge detection results of the door regions in both the open and / or closed door images.

[0042] Finally, after denoising the edge detection results of the door regions in both the open and closed door images, the denoised edge detection results of the door regions in both images can be used to construct the input image for the corresponding channel, so as to extract features associated with the door. For example, as... Figure 6As shown, the denoised edge detection results of the three door regions in the closed door image can be used as the input image for the first channel, and the denoised edge detection results of the three door regions in the open door image can be used as the input image for the second channel. This allows the input images for the corresponding channels to be provided to a door type classifier for door type identification. Considering that the door difference image also contains three door regions and various door-related features, the original edge detection results of the three door regions in the door difference image can optionally be used as the input image for the third channel to extract door features for door type identification. Furthermore, the input images for each channel can optionally be normalized, such as by normalizing pixel values ​​and image size, to facilitate subsequent feature extraction and difference comparison calculations. It should be noted that this disclosure does not limit the number of channels for the input image. For example, in the embodiments of this disclosure, more types of door images can be captured when the door is in a slightly open, half-open, or nearly fully open state, and corresponding channels can be constructed for each state, such as slightly open, half-open, or nearly fully open.

[0043] return Figure 2 In step S103, the differences between the door features extracted from multiple door images are calculated, and the door type is determined based on these differences. As discussed above, as a door is opened or closed, certain features of the door will exhibit different shapes, positions, sizes, etc., in different door images. The specific manifestations of these differences between different door images can be unique to that type of door. These differences can reflect the unique characteristics of the door, such as its opening direction and method, thus aiding in the determination of the door type. The following combines... Figure 7 and Figure 8 The specific description provides an example of an illustrative process for determining the type of a door based on the differences between door features extracted from multiple door images.

[0044] Figure 7 This diagram illustrates an exemplary process for extracting door features from input images from respective channels of multiple door images in a method for detecting the type of a door according to an embodiment of this disclosure. Figure 7 As shown, after obtaining input images from multiple channels associated with different door states, door features associated with each door can be extracted from the input images of the corresponding channels, such as... Figure 7 The candidate features are shown in the boxes. It can be understood that the door features extracted from the edge detection results of the respective door regions in the input images of each channel can include door hardware features, door leaf bounding box features, and image content features, etc. It should be noted that... Figure 7The candidate features can include a wider variety of door features that can reflect aspects such as how the door opens, and this disclosure is not limited thereto.

[0045] It is understood that, in the embodiments of this disclosure, various door feature extraction methods can be used to extract features related to the door leaf or door hardware as described above. These methods include analyzing texture and shape, analyzing the positional / directional relationships between candidate features, and using machine models such as neural networks. Various discrimination rules are employed to extract features corresponding to the door from the input images containing edge features in each channel, thereby calculating the differences between different door images and determining the door type accordingly. It should be noted that various suitable methods can be used to extract features related to the door leaf or hardware from the edge features of the input images in each channel. This disclosure does not limit the specific feature extraction method. For the completeness of this disclosure, the following description, in conjunction with... Figure 7 Furthermore, the features of door hardware, door leaf boundary frame, and image content are used as examples for illustrative explanation.

[0046] On the one hand, common hardware in houses mainly includes door handles, knobs, hinges, bolts, door locks, etc. As the door is opened and closed, these hardware components undergo variations in displacement and deformation, and the degree of displacement and deformation differs between different types of doors. Therefore, extracting the position and shape features of these door hardware components can help identify the type of door. For example... Figure 7 As shown, for the door area in a multi-channel input image, door hardware features such as the door handle (as marked "△"), door hinge (as marked "○") of one hinged door, and the door track (as marked "◇") of one sliding door can be detected. Of course, Figure 7The hardware shown is merely an illustrative example. This disclosure can also detect other types of door hardware and determine the door type accordingly. Specifically, taking door handles as an example, considering the differences in offset and deformation that occur between the open and closed states of a door handle, and the different door types leading to varying degrees of offset and deformation, for example: for hinged doors, as they are opened, the door handle will change from a front view to a side view (sometimes even both door handles on both sides of the door leaf may be visible, or only one door handle may be visible), and the door handle will shift to the left or right; while for sliding doors, as they are opened, the door handle usually does not change in view and usually does not have both door handles visible, and the door handle will only shift to the left or right. Similarly, for hinged doors, as the door is opened, the hinge will undergo changes in view and transitions between being obscured and revealed, or other detectable changes in position and / or shape; for sliding doors, the track will exhibit changes in the degree of obscuration and changes in the shape of the obscured / revealed portion. Based on the above analysis, in this embodiment of the present disclosure, the position, shape, and size characteristics of door hardware such as door handles, knobs, hinges, and tracks can be detected for use in subsequent difference calculations and door type determination.

[0047] On the other hand, doors typically consist of a door frame, door leaf, hardware, and other accessories. The door frame connects the door to the wall, and the door leaf is usually the movable part. Applying external force to the movable door leaf allows the door to open and close. As the door is opened and closed, the movable door leaf shifts, and the width of its bounding box in the input image varies. Furthermore, the degree of shift and variation in the movable door leaf differs across different door types. Therefore, extracting features such as the position and width of the movable door leaf's bounding box helps in identifying the door type. Figure 7 As shown, for a specific door area in a multi-channel input image, the bounding box of one hinged door leaf in the input image can be detected. This is an exemplary implementation, similar to the above reference. Figure 5A and Figure 5BSimilarly, as described, the bounding box of a door leaf can be determined based on pixel distribution by horizontally and vertically projecting each door region in an input image containing edge features. The inventors also note that since the top and bottom sides of the door leaf's bounding box typically do not shift during door opening and closing, but rather the left and right sides undergo translation and rotation, the left and right positions of the bounding box may contain relatively more useful information. Therefore, optionally, in embodiments of this disclosure, for the bounding box features of a door leaf, only its left and right positions and the width between them (i.e., the width of the door leaf presented to the viewer in the left-right direction in the input image) can be detected. For example, the left-right position and / or width of the door leaf's bounding box can be determined by vertically projecting only the room region containing edge features. For example, with a sliding door, as it is opened, the boundary frame of its door panel will shift. For instance, the boundary frame of the left inner sliding door will shift to the right, and during this shift, it will gradually be obscured by the boundary frame of the fixed door panel. Therefore, the width of the sliding door panel (the distance from left to right within the boundary frame of the door panel) will change from the original door panel width to zero. Similarly, the boundary frame of the right outer sliding door will shift to the left, and during this shift, it will gradually obscure the boundary frame of the fixed door panel. Consequently, the width of the fixed door panel will also change from the original door panel width to zero. Furthermore, for hinged doors, as they are opened, the door leaf boundary frame undergoes rotation and displacement. For example, the boundary frame of an inward-turning left-hinged door will shift to the left because the door leaf rotates inward around the hinge. Additionally, because the door leaf is not presented in a frontal view in the input image when the door is open but at an angle, the width of the door leaf boundary frame will change from the original door leaf width (i.e., 100% of the original door leaf width in a fully frontal view when the door is closed) to a predetermined percentage of the original width. Similarly, the boundary frame of an outward-turning right-hinged door will shift to the right because the door leaf rotates outward around the hinge. Likewise, the door leaf width will change from the original door leaf width to a predetermined percentage of the original width. Based on the above analysis, in this embodiment, the position and / or length characteristics of the door leaf boundary frame can be detected for subsequent difference calculations and door type determination.

[0048] On the other hand, as the door is opened and closed, more interior decorations are obscured or revealed, or light shines from inside the room. Consequently, the edges of these decorations are also detected, or, due to differences in light intensity, more or fewer edges are detected. Therefore, compared to a closed door image with relatively simple edges, the corresponding area of ​​the door extracted from an open door image may have more complex edges, thus presenting different image content. Through experimental testing and investigation on various door types, it has been found that the image content presented in the input images containing edge features for both the internal area and the surrounding area of ​​each door differs between open and closed door images. Furthermore, the degree of difference in image content varies between different door types and different door states. Therefore, extracting features such as the image content of the area corresponding to the door (e.g., the internal area of ​​the door, the surrounding area of ​​the door, or both the internal and surrounding areas of the door, etc.) also helps in identifying the type of door. As an example implementation, the image content of an input image containing edge features can be measured from the perspective of the total pixel value of the area corresponding to the door. Generally speaking, more complex image content corresponds to a higher total pixel value, while simpler image content corresponds to a lower total pixel value. For example, as... Figure 7 As shown, for a specific door area in a multi-channel input image, the sum of the pixel values ​​of the area surrounding the door and / or the area inside the door can be calculated. Furthermore, based on historical research data on various door types, it is known that the difference in the sum of pixel values ​​for each specific door type across different door states falls within a predetermined difference range (for example, one type of door may correspond to a larger difference, while another type may correspond to a smaller difference). Thus, the door type can be inferred based on the specific difference in the sum of pixel values. Based on the above analysis, in this embodiment, the sum of the pixel values ​​of the corresponding area of ​​the door extracted from the door-open image and the door-closed image can be calculated for subsequent difference calculations and door type determination. It is understood that, in addition to the method described above of calculating the sum of the pixel values, other suitable methods can be used to reflect the different image content of the door area across different door states.

[0049] Figure 8 The illustration shows a schematic diagram of a method for detecting the type of a room door according to an embodiment of the present disclosure, illustrating the process of determining the door type based on differences between door features extracted from multiple door images. It can be understood that, in conjunction with the above... Figure 7After extracting various door features associated with each door, the door type can be determined based on the differences between the door features extracted from multiple door images, according to a predetermined discrimination rule. For example, after extracting corresponding door hardware features from open and closed door images respectively, the positional and / or shape differences of the corresponding door hardware between different door images can be calculated, and the door type can be determined based on these differences. As another example, after extracting door leaf bounding box features from open and closed door images respectively, the positional and / or width differences of the corresponding door leaf bounding box between different door images can be calculated, and the door type can be determined based on these differences. Yet another example, after obtaining the image content of the corresponding door region extracted from open and closed door images respectively, the differences in image content between different door images can be calculated, and the door type can be determined based on these differences. It should be noted that although the above describes a method for determining door type based on the differences between different door images using a single door feature, it is not limited to this. To improve the confidence of the door type determination result, embodiments of this disclosure can also use a combination of multiple door features to comprehensively determine the door type based on multiple factors. For example, embodiments of this disclosure can comprehensively determine the door type based on a weighted combination of one or more of the following: differences in position and / or shape between corresponding door hardware, differences in position and / or width between door leaf boundaries, and differences in image content, as discussed above.

[0050] For example, combining Figure 8 The extracted door features shown differ between the edge detection results of the open and closed door images. From the extracted door hardware features, it can be seen that as the door is closed, the door handle shifts accordingly, and a change occurs from a side view to a front view. The door hinge also exhibits a different shape (e.g., at least because different parts of the hinge are exposed). From the extracted door leaf bounding box, it can be seen that as the door is closed, the door leaf bounding box shifts to the right, and this rightward shift is due to the door leaf rotating inward around the hinge. Additionally, the width of the door leaf bounding box also differs. Furthermore, the differences in image content between the open and closed door images can be calculated, for example, by calculating the difference between the sum of the pixel values ​​in the regions and determining which specific difference interval this difference falls into. By combining one or more of the above factors in a weighted manner, the door type to be inspected can be determined to be an outward-turning left hinge door from various door types such as left inward sliding door, right inward sliding door, left outward sliding door, right outward sliding door, inward-turning left hinge door, inward-turning right hinge door, outward-turning left hinge door, and outward-turning left hinge door.

[0051] It should be noted that the above combination Figure 7 and Figure 8 This primarily describes an embodiment for determining the type of a door by considering the differences between extracted door features in an open image and a closed image. However, it is not limited to this; as mentioned above, it is also possible to extract corresponding door features from the door difference image, and accordingly, the extracted door features can be used to determine the door type. The inventors noted that since the door difference image is obtained by comparing the image differences between an open image and a closed image, it can avoid most interference from content unrelated to the door, thus making it easier to identify features such as the door frame or door leaf frame and obtain more accurate results. Therefore, optionally, such as Figure 7 and Figure 8 As shown in dashed lines, embodiments of this disclosure can further determine the type of door based on door features extracted from door difference images. As an illustrative example, door features can be extracted from the edge detection results of the door open image, door closed image, and door difference image, such as door hardware features, door leaf bounding box features, and image content features. Furthermore, the changes in door features among the three images are considered, thereby determining the type of door based on... Figure 7 and Figure 8 Similar discrimination rules are described to determine door types. By comparing the differences in door features among three images—door open, door closed, and door difference images—the door type can be determined more accurately, improving the confidence level of the discrimination results.

[0052] return Figure 2 Optionally, after step S103, the method for detecting the type of a room door according to embodiments of this disclosure may further include drawing a floor plan of the house based on the type of the room door. For example, the room door icon corresponding to the determined room door type can be automatically drawn to the corresponding room door location, thereby saving labor costs and improving the efficiency of floor plan generation.

[0053] The method for detecting the type of a room door according to embodiments of the present disclosure has been described above with reference to the accompanying drawings. According to the room door type detection method provided by the present disclosure, by analyzing multiple room door images taken in different states to calculate the differences in corresponding room door features between these images, the type of each room door in a house can be automatically identified. Furthermore, the detection results of the room door types can be applied to the generation of a house floor plan, thereby avoiding the high labor costs caused by manually drawing room door icons and improving the efficiency of house floor plan generation.

[0054] Door type detection equipment

[0055] According to another aspect of this disclosure, a device for detecting the type of a room door is provided, which is described below in conjunction with... Figure 9 Detailed description of device 900 for detecting the type of room door.

[0056] Figure 9 A hardware block diagram of a device for detecting the type of a room door according to an embodiment of this disclosure is shown. Figure 9 As shown, the device 900 includes a processor U901 and a memory U902.

[0057] The processor U901 can be any processing-capable device capable of implementing the functions of the various embodiments of this disclosure. For example, it can be a general-purpose processor, digital signal processor (DSP), ASIC, field programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein.

[0058] The memory U902 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, or other removable / non-removable, volatile / non-volatile computer system memory, such as hard disk drives, floppy disks, CD-ROMs, DVD-ROMs, or other optical storage media.

[0059] In this embodiment, the memory U902 stores computer program instructions, and the processor U901 can execute the instructions stored in the memory U902. When the computer program instructions are executed by the processor, the processor performs the method for detecting the type of a door according to this embodiment of the present disclosure. The method for detecting the type of a door is similar to that described above. Figures 2-8 The descriptions are basically the same, so to avoid repetition, I will not repeat them.

[0060] According to another aspect of this disclosure, a device for detecting the type of a room door is provided, in conjunction with the following... Figure 10 Detailed description of device 1000 for detecting the type of room door.

[0061] Figure 10 A structural block diagram of a device for detecting the type of a room door according to an embodiment of the present disclosure is shown. Figure 10 As shown, the device 1000 includes an image acquisition unit U1001, a feature extraction unit U1002, and a door type determination unit U1003. Each component can perform the functions described above. Figure 2-8 The various steps / functions of the door type detection method described herein are omitted here for the purpose of avoiding repetition. Only a brief description of the device is given below, while detailed descriptions of the same details are omitted.

[0062] The image acquisition unit U1001 can acquire multiple images of doors in different states. For example, the image acquisition unit U1001 can take pictures of doors in different states within a house to obtain multiple corresponding door images. In one example, the image acquisition unit U1001 can take each door as the subject and take individual images of that door in both its closed and open states, and perform the same process to obtain individual images of the remaining doors in the house. In another example, the image acquisition unit U1001 can take all the doors in the house as the subject, place the camera at a certain location within the house and adjust its shooting angle, then take a panoramic image when all doors are in their open states, and another panoramic image when all doors are in their closed states.

[0063] The feature extraction unit U1002 can extract door features associated with the door from the multiple door images. The door features extracted by the feature extraction unit U1002 can be features related to the shape, position, and size of the door leaf or door hardware. These features will exhibit certain differences due to the opening or closing of the door, thereby reflecting unique door characteristics such as the opening direction and method, and inferring the door type. In embodiments of this disclosure, the feature extraction unit U1002 can use various methods to extract door features associated with the door and helpful for door type determination. For example, the feature extraction unit U1002 can first segment the door region from the door-open image and the door-closed image respectively. Then, it can extract door features associated with the door from the respective door regions in the door-open and door-closed images. Alternatively, the feature extraction unit U1002 can directly extract door features associated with the door from the entire door image without prior image segmentation processing; this disclosure does not limit this.

[0064] Specifically, the feature extraction unit U1002 can segment the door region from the door open and door closed images using various methods, such as based on features like grayscale, color, spatial texture, and geometric shape. In addition, the feature extraction unit U1002 can perform door segmentation based on image difference comparison and multiple thresholding, segmenting the door region according to the pixel distribution in the thresholding fusion result, thereby effectively obtaining each door region including the door to be detected. For example, the feature extraction unit U1002 first compares the image differences between the door open and door closed images to calculate the door difference image between them. Then, it performs fixed thresholding and adaptive thresholding on the door difference image. Finally, it fuses the results of the fixed thresholding and adaptive thresholding to generate a thresholded fused image. After obtaining the thresholded fused image excluding the door, the feature extraction unit U1002 can segment the door region from the door open and door closed images respectively based on the pixel distribution of this thresholded fused image. For example, the feature extraction unit U1002 can use horizontal and vertical projection to segment one or more door regions that may cover the door from multiple door images. Then, after roughly locking down three door regions from the thresholded fused image, segmentation can be performed at the corresponding regions in the closed and open door images to obtain three corresponding door regions for feature extraction.

[0065] Then, based on segmenting the corresponding door regions from the corresponding door images, the feature extraction unit U1002 can extract door-related features from each door region using various feature extraction methods, such as color, texture, and edge feature extraction methods. In addition, the feature extraction unit U1002 can extract edge features from the door regions of both the open and closed door images, thereby constructing corresponding input images containing edge features for door classification. For example, the feature extraction unit U1002 can use edge extraction algorithms such as the Canny operator, Sobel operator, and Laplacian operator to process the door regions to extract door edge features from their respective regions. Alternatively, the feature extraction unit U1002 can blur the corresponding segmented door regions of either the open or closed door image, and obtain the edges in the door regions by calculating the difference between the original door region and the blurred door region.

[0066] Preferably, the feature extraction unit U1002 can perform denoising processing based on the edge detection results of the door difference image to eliminate noise related to the color texture of the walls and furniture from the edge detection results of the door open image and the door closed image. For example, the feature extraction unit U1002 can calculate the door difference image between the door open image and the door closed image, and then segment the door region from the door difference image. Then, edge detection can be performed separately in the three door regions of each of the door open image and the door closed image, and edge detection can also be performed in the three door regions of the door difference image for denoising the edge detection results of the other two. Finally, after denoising the edge detection results of the door regions of each of the door open image and the door closed image, the denoised edge detection results of the door regions of each of the door open image and the door closed image can be constructed as the input image of the corresponding channel to extract features associated with the door.

[0067] The door type determination unit U1003 can calculate the differences between door features extracted from multiple door images and determine the door type based on these differences. Specifically, after obtaining input images of multiple channels associated with different door states, door features associated with each door can be extracted from the input images of the corresponding channels, such as door hardware features, door leaf bounding box features, and image content features.

[0068] In one example, considering that common hardware in a house mainly includes door handles, knobs, hinges, bolts, locks, etc., and that these hardware components will experience differences in offset and deformation as the door is opened / closed, with different degrees of offset and deformation for different types of doors, extracting the position and shape features of these door hardware components can help determine the type of door. To this end, the door type determination unit U1003 can detect the position, shape, and size features of door handles, knobs, hinges, tracks, etc., for use in subsequent difference calculations and door type determination.

[0069] In another example, considering that a door typically consists of a door frame, door leaf, hardware, and other accessories, and that the movable door leaf shifts as the door is opened / closed, resulting in variations in the width of its bounding box in the input image, and that the degree of shift and variation of the movable door leaf differs for different types of doors, extracting features such as the position and width of the movable door leaf's bounding box is helpful in determining the door type. To this end, the door type determination unit U1003 can determine the position and / or length features of the door leaf's bounding box for subsequent difference calculations and door type determination.

[0070] In another example, considering that the opening / closing of a door can cause more interior decorations to be obscured or revealed, or light to shine out of the room, and based on experimental testing and investigation of various door types, it has been found that the image content of both the internal and surrounding areas of each door differs between the input images containing edge features in the door-open and door-closed images. Furthermore, the degree of difference in image content varies between different door types and door states. Therefore, extracting the image content features of the area corresponding to the door can help determine the door type. As an exemplary implementation, the image content can be determined based on the sum of pixel values ​​of the area corresponding to the door. For example, the door type determination unit U1003 can calculate the sum of pixel values ​​of the area corresponding to the door extracted from the door-open and door-closed images, respectively, for use in subsequent calculations of the difference in the sum of pixel values ​​and in determining the door type.

[0071] Then, the door type determination unit U1003 can determine the door type based on the differences between the door features extracted from multiple door images after extracting various door features associated with each door. For example, after extracting the corresponding door hardware features from the door open image and the door closed image respectively, the door type determination unit U1003 can calculate the position and / or shape differences of the corresponding door hardware between different door images, and determine the door type based on the position and / or shape differences between the corresponding door hardware. As another example, after extracting the door leaf bounding box features from the door open image and the door closed image respectively, the door type determination unit U1003 can calculate the position and / or width differences of the corresponding door leaf bounding box between different door images, and determine the door type based on the position and / or width differences between the door leaf bounding boxes. As yet another example, after obtaining the image content of the corresponding area of ​​the door extracted from the door open image and the door closed image respectively, the door type determination unit U1003 can calculate the differences in the image content between different door images, and determine the door type based on the differences in the image content. In addition, the door type determination unit U1003 can also comprehensively determine the type of door based on a weighted combination of one or more of the following: differences in position and / or shape between corresponding door hardware, differences in position and / or width between door leaf boundary frames, and differences in image content. For example, the type of the door to be detected can be determined as one of various door types such as left inward sliding door, right inward sliding door, left outward sliding door, right outward sliding door, inward left-turning hinge door, inward right-turning hinge door, outward left-turning hinge door, and outward left-turning hinge door.

[0072] In addition, the door type determination unit U1003 can further determine the door type based on door features extracted from the door difference image. For example, when the feature extraction unit U1002 extracts door features (such as door hardware features, door leaf bounding box features, and image content features) from the edge detection results of the door open image, door closed image, and door difference image, the door type determination unit U1003 can consider the changes in door features among the three images to determine the door type. By comparing the differences in door features among the three images, the door closed image, and door difference image, the door type can be determined more accurately, improving the confidence of the discrimination result.

[0073] Optionally, the device 1000 may also include a house floor plan generation unit (not shown). After the door type detection unit U1004 determines the door types within the house, the house floor plan generation unit can draw a house floor plan based on the detected door types. For example, the door icons corresponding to the determined door types can be automatically drawn to the corresponding door locations, thereby saving labor costs and improving the efficiency of floor plan generation.

[0074] The apparatus for detecting the type of a room door according to embodiments of the present disclosure has been described above with reference to the accompanying drawings. The described apparatus for detecting the type of a room door analyzes multiple images of the room door taken in different states to calculate the differences in corresponding door features between these images. It can automatically identify the type of each room door in a house and apply the detection results to the generation of a house floor plan, thereby avoiding the high labor costs caused by manually drawing door icons and improving the efficiency of house floor plan generation.

[0075] Computer-readable storage media

[0076] The method / apparatus for detecting the type of a door according to this disclosure can also be implemented by providing a computer program product containing program code implementing the method or apparatus, or by any storage medium storing such a computer program product.

[0077] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the embodiments of this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0078] The block diagrams of devices, apparatuses, devices, and systems involved in the embodiments of this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0079] Additionally, as used herein, the “or” used in a list of items beginning with “at least one” indicates a separate list, such that a list of, for example, “at least one of A, B, or C” means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word “exemplary” does not imply that the described example is preferred or better than other examples.

[0080] It should also be noted that in the apparatus and method of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0081] It will be understood by those skilled in the art that all or any part of the methods and apparatus of this disclosure can be implemented in hardware, firmware, software, or a combination thereof in any computing device (including processors, storage media, etc.) or network of computing devices. The hardware may be a general-purpose processor, digital signal processor (DSP), ASIC, field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, it may be any commercially available processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration. The software may reside in any form of computer-readable tangible storage medium. By way of example and not limitation, such computer-readable tangible storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other tangible medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible by a computer. If used herein, the disks include compact discs (CDs), laser discs, optical discs, digital universal discs (DVDs), floppy disks, and Blu-ray discs.

[0082] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0083] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0084] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for detecting the type of a room door, comprising: Acquire multiple images of doors in different states; Extract the door features associated with the door from the multiple door images respectively; as well as Calculate a first difference between door features extracted from the plurality of door images, and determine the type of door based on the first difference and a second difference between door features in different states corresponding to different types of doors.

2. The method according to claim 1, wherein, Acquiring multiple images of doors in different states includes: Capture an image of the door opening while the door is in the open state; and Take an image of the door closing while it is in the closed state.

3. The method according to claim 2, wherein, Extracting door features associated with the door from the multiple door images includes: The door area was segmented from the door open image and the door closed image, respectively; and In the respective door regions of the door open image and the door closed image, door features associated with the door are extracted.

4. The method according to claim 3, wherein, Segmenting the door area from the door open image and the door closed image respectively includes: Calculate the door difference image between the door open image and the door closed image, and perform fixed thresholding and adaptive thresholding on it; The results of the fixed thresholding process and the adaptive thresholding process are fused to generate a thresholded fused image; and Based on the pixel distribution of the thresholded fused image, the door area is segmented from the door open image and the door closed image, respectively.

5. The method according to claim 3, wherein, In the respective door regions of the door-open image and the door-closed image, the door features associated with the door are extracted, including: Calculate the door difference image between the door open image and the door closed image, and segment the door area from the door difference image; Edge detection is performed in the respective door regions of the door open image, the door closed image, and the door difference image; Based on the edge detection results of the door region in the door difference image, denoise the edge detection results of the door region in the door open image and the door closed image; and In the denoised edge detection results of the door region in both the door-open image and the door-closed image, door features associated with the door are extracted respectively.

6. The method according to any one of claims 2-5, wherein, The door features associated with the door include one or more of the following: door hardware features, door leaf bounding box features, and image content features, wherein... Calculating a first difference between door features extracted from the plurality of door images, and determining the type of door based on the first difference and a second difference between door features in different states corresponding to different types of doors, includes: Calculate the position and / or shape differences between the corresponding door hardware extracted from the door open image and the door closed image, respectively; Calculate the position and / or width differences between the door leaf bounding boxes extracted from the door open image and the door closed image, respectively; Calculate the differences between the image content of the region corresponding to the door extracted from the door-open image and the door-closed image, respectively; and The type of the room door is determined based on one or more of the positional and / or shape differences between the corresponding door hardware, the positional and / or width differences between the door leaf boundary frames, and the differences between the image content, as well as the second difference.

7. The method according to claim 6, further comprising: The type of door is further determined based on the door features extracted from the door difference image between the door open image and the door closed image.

8. A device for detecting the type of a room door, comprising: processor; as well as Memory, which stores computer program instructions. When the computer program instructions are executed by the processor, the processor performs the following steps: Acquire multiple images of doors in different states; Extract door features associated with the door from the multiple door images; and Calculate a first difference between door features extracted from the plurality of door images, and determine the type of door based on the first difference and a second difference between door features in different states corresponding to different types of doors.

9. A device for detecting the type of a room door, comprising: The image acquisition unit is configured to acquire multiple images of doors in different states; The feature extraction unit is configured to extract door features associated with the door from the plurality of door images, respectively; as well as The door type determination unit is configured to calculate a first difference between door features extracted from the plurality of door images, and to determine the type of the door based on the first difference and a second difference between door features in different states corresponding to different types of doors.

10. A computer-readable storage medium having stored thereon computer program instructions, wherein, When the computer program instructions are executed, they perform the following steps: Acquire multiple images of doors in different states; Extract door features associated with the door from the multiple door images respectively; and Calculate a first difference between door features extracted from the plurality of door images, and determine the type of door based on the first difference and a second difference between door features in different states corresponding to different types of doors.

Images