Method for detecting state of opening and closing device, computer device and storage device

Abstract

The application discloses a state detection method of an opening and closing device, a computer device and a storage device. The method comprises the following steps: acquiring multiple preset state images of the opening and closing device, wherein the opening and closing device in each preset state image is at a different opening and closing degree; using the multiple preset state images, at least two state image features corresponding to at least two opening and closing degrees of the opening and closing device are extracted; and based on the comparison result of the detection image feature of a to-be-detected image and the at least two state image features, the opening and closing state of the opening and closing device is determined. The above scheme can improve the accuracy of state detection of multiple opening and closing devices.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a state detection method of an opening and closing device, a computer device and a storage device. BACKGROUND

[0002] The opening and closing device refers to a device having two states of opening and closing, for example, the opening and closing device is a door, a window or the like, and the opening and closing device is widely used in shopping malls, office areas, residential areas and the like, and is an indispensable part of a building and closely related to our daily life.

[0003] Taking the opening and closing device including a door as an example, different doors have different special functions, and can have functions of anti-theft, fireproof, heat insulation, sound insulation and the like. Generally, the doors at places such as fireproof doors, factories and warehouses need to be kept closed in normal times, and can only be opened when necessary, due to the large number of doors, it is difficult to rely on manpower to patrol, and a large amount of human resources needs to be spent for detection.

[0004] At present, the opening and closing state of the opening and closing device is usually detected by installing a sensor, but a large number of existing opening and closing devices need to be replaced or modified, the detection cost is large, and only a single opening and closing device can be detected, and the adaptability is poor. SUMMARY

[0005] The technical problem solved by the present application is to provide a state detection method of an opening and closing device, a computer device and a storage device, which can improve the accuracy of state detection of a plurality of opening and closing devices.

[0006] In order to solve the above problems, the first aspect of the present application provides a state detection method of an opening and closing device, which comprises: acquiring a plurality of preset state images of the opening and closing device, the opening and closing device in each preset state image being in different opening and closing degrees; using the plurality of preset state images, at least two state image features corresponding to at least two opening and closing degrees of the opening and closing device are extracted; and based on the comparison result of the detection image feature of the to-be-detected image and the at least two state image features, the opening and closing state of the opening and closing device is determined.

[0007] In order to solve the above problems, the second aspect of the present application provides a computer device, which comprises a memory and a processor coupled to each other, the memory stores program data, and the processor is used to execute the program data to realize any step of the above-mentioned state detection method of the opening and closing device.

[0008] In order to solve the above problems, the third aspect of the present application provides a storage device, which stores program data capable of being run by a processor, and the program data is used to realize any step of the above-mentioned state detection method of the opening and closing device.

[0009] The scheme is characterized in that: the opening and closing device is in different opening and closing degrees in each preset state image, at least two state image features corresponding to at least two opening and closing degrees of the opening and closing device are extracted by using the plurality of preset state images, and the opening and closing state of the opening and closing device is determined based on a comparison result of a detection image feature of the to-be-detected image and the at least two state image features. Since the preset state images of the opening and closing device in different opening and closing degrees are compared with the to-be-detected image to determine the opening and closing state of the opening and closing device, the opening and closing device does not need to be improved, the state detection of various opening and closing devices can be adapted, and the accuracy of the state detection of various opening and closing devices can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings. Among them:

[0011] Figure 1 is a flowchart of a first embodiment of the state detection method of the opening and closing device of the present application;

[0012] Figure 2 is a flowchart of a first embodiment of the state detection method of the opening and closing device of the present application; Figure 1

[0013] Figure 3 is a flowchart of a first embodiment of the state detection method of the opening and closing device of the present application; Figure 2

[0014] Figure 4 is a flowchart of a second embodiment of the state detection method of the opening and closing device of the present application;

[0015] Figure 5 is a flowchart of a third embodiment of the state detection method of the opening and closing device of the present application;

[0016] Figure 6 is a flowchart of a fourth embodiment of the state detection method of the opening and closing device of the present application;

[0017] Figure 7 is a flowchart of a fifth embodiment of the state detection method of the opening and closing device of the present application;

[0018] Figure 8 is a structural diagram of an embodiment of the state detection device of the opening and closing device of the present application;

[0019] Figure 9 is a structural diagram of an embodiment of the computer device of the present application;​​

[0020] Figure 10 Figure 1 is a schematic diagram of an embodiment of a storage device of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0022] The terms "first", "second", etc. in the present application are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second", etc. can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0023] In the present application, the phrase "embodiment" means that the specific features, structures or properties described in conjunction with the embodiment can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily refer to the same embodiment, nor is it mutually exclusive or alternative to other embodiments. A person of ordinary skill in the art explicitly and implicitly understands that the embodiments described herein can be combined with other embodiments.

[0024] The term "and / or" herein is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship. In addition, "multiple" in this paper means two or more than two. In addition, the term "at least one" in this paper means any one of the plurality or any combination of at least two of the plurality, for example, including at least one of A, B and C can mean including any one or more elements selected from the set consisting of A, B and C.

[0025] The present application provides the following embodiments, which will be specifically described below.

[0026] Referring to Figure 1 , Figure 1 is a flowchart of a first embodiment of a state detection method of the opening and closing device. The method can include the following steps:

[0027] S11: Obtain multiple preset state images of the opening and closing device, and each opening and closing device in each preset state image is at a different opening and closing degree.

[0028] The opening and closing device includes doors, windows, etc. For example, the door can include various types of doors such as a normal sliding door, a double-leaf door, a sliding door, a roller shutter door, a cabinet door, etc., and the window can include various types of windows such as a sliding window, a double-leaf window, etc. The opening and closing device of the present application includes a device having two states of opening and closing, and the present application does not limit this.

[0029] In some embodiments, a camera device can be arranged in a scene where the opening and closing device is located, and the opening and closing device is located within the visual angle range of the camera device, so that the camera device can capture a picture image containing the opening and closing device at multiple different opening and closing degrees.

[0030] In some embodiments, the image captured by the same camera device can contain images of multiple opening and closing devices, i.e., multiple opening and closing devices are located within the visual angle range of the same camera device.

[0031] In some embodiments, after determining the installation position of the camera device, the picture image of the opening and closing device at different opening and closing degrees is collected by using the camera device, the opening and closing device in the image can be target detected to obtain the position of the opening and closing device in the picture image; or the position of the opening and closing device in the picture image input by the user is received. The position described herein can be represented as a coordinate box in the picture image, and there can be multiple coordinate boxes of the opening and closing device in one picture image.

[0032] In some embodiments, the picture image is cropped, and the image of each coordinate box, i.e., the image of each opening and closing device, can be used as a preset state image of the opening and closing device at an opening and closing degree.

[0033] In some embodiments, the multiple preset state images can include at least one preset state image of the opening and closing device at a closed state and at least two preset state images of the opening and closing device at different open states.

[0034] In some embodiments, the types of the opening and closing device at different opening and closing degrees can be represented as 0 (completely closed), 15%-30%, 30%-70%, greater than 70%, etc. Different opening and closing degrees can be set according to application scenarios, types of opening and closing devices, etc., and the present application does not limit this.

[0035] In this way, the plurality of preset state images of each opening and closing device can be obtained, where the plurality of preset state images refers to two or more, and each preset state image is of a different opening and closing degree.

[0036] S12: At least two state image features corresponding to at least two opening and closing degrees of the opening and closing device are extracted using the plurality of preset state images.

[0037] The plurality of preset state images are input into a preset neural network for feature extraction, and the state image features corresponding to at least two opening and closing degrees of the opening and closing device can be extracted. The preset neural network can be ResNet (Deep Residual Network), VGG (Visual Geometry Group), etc., which is not limited in the present application.

[0038] In some embodiments, feature extraction can be performed on each preset state image to obtain state image features corresponding to a plurality of opening and closing degrees of the opening and closing device.

[0039] In some embodiments, the at least two state image features include at least one first state image feature and / or a plurality of second state image features.

[0040] The first state image feature is obtained based on a target difference region of at least one preset state image, and the target difference region can represent the opening and closing direction of the opening and closing device. The second state image feature is obtained by feature extraction based on each preset state image.

[0041] S13: The opening and closing state of the opening and closing device is determined based on the comparison result of the detection image feature of the to-be-detected image and the at least two state image features.

[0042] Before detecting the opening and closing state of the opening and closing device, a camera device is used to capture a captured image of the opening and closing device to obtain a to-be-detected image of the opening and closing device.

[0043] In some embodiments, the captured image can include a plurality of opening and closing devices, and each opening and closing device in the captured image can be cropped to obtain a corresponding to-be-detected image.

[0044] In some embodiments, feature extraction is performed on the to-be-detected image of the opening and closing device to obtain a detection image feature, and the feature extraction can use a preset neural network such as ResNet or VGG, which is not limited in the present application.

[0045] The detection image features are compared with the at least two state image features to obtain corresponding comparison results. The comparison can obtain similarity information of the detection image features and the at least two state image features, and the similarity information is used to determine the comparison results.

[0046] In some embodiments, the at least two state image features include at least one first state image feature and / or a plurality of second state image features. Similarity information of the detection image features and the at least one first state image feature can be obtained respectively, and / or similarity information of the detection image features and the plurality of second state image features can be obtained respectively. Thus, the similarity information is used to obtain the comparison results, and the comparison results are used to determine the opening and closing state of the opening and closing device.

[0047] In some embodiments, the comparison results of the detection image features and the plurality of second state image features can be obtained respectively, and the opening and closing state of the opening and closing device is determined based on the comparison results. If the opening and closing state of the opening and closing device cannot be determined, the comparison results of the detection image features and the at least one first state image feature can be obtained respectively to determine the opening and closing state of the opening and closing device.

[0048] In this embodiment, a plurality of preset state images of the opening and closing device are obtained, the opening and closing device in each preset state image is in a different opening and closing degree, at least two state image features corresponding to at least two opening and closing degrees of the opening and closing device are extracted using the plurality of preset state images, and the opening and closing state of the opening and closing device is determined based on the comparison results of the detection image features of the to-be-detected image and the at least two state image features. Since the preset state images of the opening and closing device in different opening and closing degrees are compared with the to-be-detected image to determine the opening and closing state of the opening and closing device, the opening and closing device does not need to be improved, can adapt to state detection of various opening and closing devices, and can improve the accuracy of state detection of various opening and closing devices.

[0049] In some embodiments, please refer to Figure 2 The step S12 of the above embodiment can be further expanded. At least two state image features corresponding to at least two opening and closing degrees of the opening and closing device are extracted using a plurality of preset state images. The embodiment can include at least one of the following steps:

[0050] S121: Feature extraction is performed on each preset state image respectively to obtain second state image features corresponding to each preset state image.

[0051] Specifically, each preset state image can be feature-extracted using a preset neural network to obtain second state image features corresponding to each preset state image, that is, second state image features corresponding to different opening and closing degrees of the opening and closing device.

[0052] For example, the plurality of preset state images include a preset state image 1 with an opening and closing degree of 0, a preset state image 2 with an opening and closing degree of 15%-30%, a preset state image 3 with an opening and closing degree of 30%-70%, and a preset state image 4 with an opening and closing degree greater than 70%. i ={f1, f2, f3, f4}, i is an integer.

[0053] S122: Obtain a target difference region between the plurality of preset state images, and perform feature extraction on the target difference region of at least one preset state image to obtain at least one first state image feature.

[0054] The target difference region can be determined by using the plurality of preset state images, so that based on the position of the target difference region, the target difference region corresponding to at least one preset state image is determined, and feature extraction is performed on the target difference region corresponding to at least one preset state image to obtain the first state image feature corresponding to at least one preset state image.

[0055] For example, the plurality of preset state images include a preset state image 1 with an opening and closing degree of 0, a preset state image 2 with an opening and closing degree of 15%-30%, a preset state image 3 with an opening and closing degree of 30%-70%, and a preset state image 4 with an opening and closing degree greater than 70%.

[0056] The region outside the target difference region can be shielded, and the region outside the target difference region is assigned a preset pixel, such as a black pixel. The target difference region corresponding to the preset state image 1 and / or the preset state image 2 can be subjected to feature extraction to obtain the first state image feature fd1 corresponding to the preset state image 1 and the first state image feature fd2 corresponding to the preset state image 2.

[0057] In some embodiments, the above-mentioned first state image features and second state image features and the positions of the corresponding opening and closing devices are stored.

[0058] In some embodiments, referring to Figure 3 The step S122 of the above-mentioned embodiments can be further extended. The target difference region between the plurality of preset state images is obtained, and the present embodiment can include the following steps:

[0059] S1221: Obtain a first difference region between each pair of preset state images in the plurality of preset state images.

[0060] In the plurality of preset state images, each pair of preset state images is taken as a group of image pairs, so that two images in the plurality of preset state images form a group of image pairs.

[0061] The one of the preset state images in the image pair is divided into a plurality of first regions, and the other of the preset state images in the image pair is divided into a plurality of second regions, wherein the region division manners of the image pairs are the same, for example, the same grid division method is adopted to divide the one of the preset state images in the image pair into a plurality of first regions and to divide the other of the preset state images into a plurality of second regions.

[0062] In some embodiments, the plurality of first regions and the plurality of second regions are one-to-one corresponding, and each first region and the corresponding second region form a region pair. For example, it can be understood that the first region and the second region at the same position in the image pair correspond, and the first region and the second region at the same position form a region pair.

[0063] The region similarity of each region pair in the image pair is used, wherein a similarity algorithm such as SSIM (Structural Similarity), SIFT (Scale-invariant feature transform), perceptual hashing algorithm, etc. can be used. Using the region similarity of each region pair in the image pair, the first difference region between the image pairs can be determined.

[0064] In some embodiments, among the region pairs contained in the image pair, at least one target region pair in which the region similarity in the image pair meets the difference region requirement is selected; wherein the difference region requirement includes that the region similarity is less than a region similarity threshold. That is, the region pair in which the region similarity in the image pair is less than the region similarity threshold is found as the target region pair.

[0065] The image positions corresponding to the at least one target region pair in the image pair are combined or spliced to obtain the first difference region between the image pairs, for example, which can be denoted as d ij =SSIM(s i ,s j ), which can represent the first difference region between the preset state image i and the preset state image j, i and j are integers greater than or equal to 1.

[0066] In this way, the first difference region between each image pair can be obtained.

[0067] S1222: Obtain a target difference region by using the first difference region between each two preset state images.

[0068] In some embodiments, after obtaining the first difference region between each pair of images, the intersection of the first difference region of each pair of images can be obtained to determine the minimum difference region. The minimum difference region can be determined by performing an intersection operation on the first difference region between each pair of preset state images, so that the minimum difference region is used as the target difference region between each pair of preset state images.

[0069] In some embodiments, the target difference region between n preset state images can be represented as d = ∩d ij (1≤i,j≤n,i≠j), n is an integer greater than 1, "∩" represents an intersection operation, and i and j represent preset state images.

[0070] In some embodiments, the target difference region between each pair of preset state images of the opening and closing device can represent the opening and closing direction of the opening and closing device. For example, the opening and closing device is a common door, and the target difference region is in the left or right region of the preset state image, i.e., the target difference region indicates that the opening and closing direction is on the left or right side of the door frame. For example, the opening and closing device is a roller shutter door, and the target difference region is in the lower region of the preset state image, i.e., the target difference region indicates that the opening and closing direction is below the door. For example, the opening and closing device is a double-leaf door, and the target difference region is in the middle region of the preset state image, i.e., the target difference region indicates that the opening and closing direction is in the middle position of the two leaves of the opening and closing device.

[0071] In the above manner, the target difference region between each pair of preset state images can be used to accurately determine the opening and closing direction of each type of opening and closing device, and the opening and closing region of each type of opening and closing device can be accurately obtained.

[0072] In some embodiments, before detecting the state of the opening and closing device, a to-be-detected image for detecting the state of the opening and closing device can be obtained.

[0073] Referring to Figure 4 , Figure 4 is a flowchart of a second embodiment of the state detection method of the opening and closing device. The method of this embodiment can include the following steps:

[0074] S21: Capturing a scene in which the opening and closing device is located to obtain a scene image.

[0075] In some embodiments, before the step S13, i.e., before determining the opening and closing state of the opening and closing device based on the comparison result of the detection image feature of the to-be-detected image and the state image feature of at least two state images, any step of this embodiment can be performed.

[0076] A scene in which the opening and closing device is located is captured by using a camera device to obtain at least one scene image. The angle at which the camera device captures the scene image is consistent with the angle at which the preset state image is captured.

[0077] In some embodiments, since the state change frequency of the opening and closing device is low in most cases, it is not necessary to perform continuous state detection on the frame of the scene image. The state detection can be performed on the scene image when it is determined that the detection condition is met.

[0078] S22: Determine whether there is a target entity in the scene image.

[0079] In some embodiments, the target detection can be performed on the current scene image, and the target entity for target detection can be a human target, an animal, a moving object, a marker, etc., which is not limited in the present application.

[0080] In some embodiments, the present embodiment takes the human target as an example for illustration, and the yolo v5 algorithm or the like can be used for target detection on the current scene image.

[0081] In some embodiments, if it is determined that there is no target entity in the frame of the scene image for target detection, step S23 is performed. The state of the opening and closing device is detected at a first preset time, for example, the opening and closing state of the opening and closing device is detected once every 1 minute, that is, the image part corresponding to the opening and closing device in the current scene image is taken as the image to be detected, and after one minute, the steps of the present embodiment are repeated to obtain the next frame of scene image for state detection of the opening and closing device.

[0082] In some embodiments, if it is determined that there is a target entity in the frame of the scene image for target detection, step S23 or step S24 is performed.

[0083] In the case of performing step S23, the state of the opening and closing device is detected at a second preset time, wherein the second preset time is less than the first preset time, for example, the detection time can be shortened to detect the opening and closing state of the opening and closing device once every 1 second.

[0084] In the case of performing step S24, the positional relationship between the target entity and the opening and closing device can be further determined.

[0085] S23: Take the image part corresponding to the opening and closing device in the scene image as the image to be detected.

[0086] Based on the recorded position of the opening and closing device, or the region detection of the opening and closing device on the scene image, the image part corresponding to each opening and closing device in the scene image is determined, so that the image part can be taken as the image to be detected.

[0087] S24: Determine whether the target entity and the opening and closing device have a positional overlap relationship.

[0088] In some embodiments, if it is determined that the target entity exists in the picture of the target detection scene image, it is further determined whether the target entity and the opening and closing device have a position overlapping relationship.

[0089] For example, taking the target entity as a human body target and the opening and closing device as a door as an example, the position relationship between the target frame of the human body and the position frame of the door is obtained, and it is determined whether there is a position overlapping relationship.

[0090] In some embodiments, if it is determined that there is no position overlapping relationship, step S23 is performed. Therefore, the image part corresponding to the opening and closing device in the scene image can be used as the detection image in response to the absence of the target entity in the scene image or the presence of the target entity in the scene image and the absence of the position overlapping relationship between the target entity and the opening and closing device.

[0091] In some embodiments, if it is determined that there is a position overlapping relationship, step S25 is performed.

[0092] S25: waiting for a preset interval time, and capturing the scene in which the opening and closing device is located to obtain a next frame of scene image.

[0093] In response to the presence of the target entity in the scene image and the presence of the position overlapping relationship between the target entity and the opening and closing device, in order to avoid false positives or false detections, the opening and closing state detection of the current scene image is not performed, and after waiting for a preset interval time, the step 21 of the above embodiment is performed again, that is, the scene in which the opening and closing device is located is captured to obtain a next frame of scene image.

[0094] In some embodiments, for each type or each opening and closing device, the above steps can be performed respectively or simultaneously to obtain the detection images of the opening and closing devices.

[0095] In the embodiment, by using the image part corresponding to the opening and closing device in the scene image as the detection image in response to the absence of the target entity in the scene image or the presence of the target entity in the scene image and the absence of the position overlapping relationship between the target entity and the opening and closing device, unnecessary state detection of the opening and closing device can be reduced, and false detection of the opening and closing state of the opening and closing device can be reduced.

[0096] In some embodiments, the step S13 of the above embodiment can be further expanded. That is, the comparison result of the detection image feature based on the detection image and at least two state image features is further expanded to determine the opening and closing state of the opening and closing device. For details, refer to the following embodiments.

[0097] In some embodiments, refer to Figure 5 , Figure 5 is a flowchart of a third embodiment of a state detection method of an opening and closing device. The method can include the following steps:

[0098] S31: based on the comparison result, a first state image feature that satisfies a similarity requirement with the first feature similarity between the first detection image feature is selected as a target state image feature.

[0099] The at least two state image features include at least one first state image feature, and each first state image feature is obtained by performing feature extraction on a target difference region of a preset state image.

[0100] Based on the position of the target difference region, a second difference region in the same region or position as the target difference region is determined from the to-be-detected image, and feature extraction is performed on the second difference region. The feature obtained by performing feature extraction can be used as the first detection image feature.

[0101] The comparison result of the first detection image feature of the to-be-detected image and the at least one first state image feature is obtained, and the comparison result can be represented by a first feature similarity.

[0102] In some embodiments, the first feature similarity of the first detection image feature of the to-be-detected image and the at least one first state image feature can be obtained.

[0103] In some embodiments, the similarity requirement can include at least one of the first feature similarity being greater than a first matching threshold, a maximum value of the first feature similarities, and the like.

[0104] For example, the first feature similarity of the first detection image feature and the first state image feature corresponding to a preset state image is greater than the first matching threshold, and the first state image feature corresponding to the preset state image can be used as the target state image feature.

[0105] For example, the first feature similarity of the first detection image feature and the first state image feature corresponding to a plurality of preset state images is greater than the first matching threshold, and the maximum value of the first feature similarities corresponds to a preset state image. The first state image feature corresponding to the preset state image corresponding to the maximum value can be used as the target state image feature.

[0106] For example, the first feature similarity of the first detection image feature and the first state image feature corresponding to a preset state image is less than or equal to the first matching threshold, and the maximum value of the first feature similarities corresponds to a preset state image. The first state image feature corresponding to the preset state image corresponding to the maximum value can be used as the target state image feature.

[0107] S32: based on the opening and closing degree of the preset state image corresponding to the target state image feature, an opening and closing state of the opening and closing device is obtained.

[0108] If the opening and closing state represented by the opening and closing degree of the preset state image corresponding to the target state image feature is the open state, it is determined that the opening and closing state of the opening and closing device is the open state. For example, if the opening and closing degree of the corresponding preset state image is any one of 15%-30%, 30%-70%, and greater than 70%, it can be determined that the opening and closing state of the opening and closing device is the open state.

[0109] In some embodiments, when it is determined that the opening and closing state of the opening and closing device is the open state, the opening and closing degree of the opening and closing device can also be determined based on the opening and closing degree of the preset state image corresponding to the target state image feature.

[0110] If the opening and closing state represented by the opening and closing degree of the preset state image corresponding to the target state image feature is the closed state, it is determined that the opening and closing state of the opening and closing device is the closed state. For example, if the opening and closing degree of the corresponding preset state image is 0, it is determined that the opening and closing state of the opening and closing device is the closed state.

[0111] In this embodiment, by using the similarity information of the target difference region, that is, by using the opening and closing direction of the opening and closing device, the opening and closing state of the opening and closing device can be determined, and the applicability of state detection of the opening and closing device can be further improved.

[0112] In some embodiments, please refer to Figure 6 , Figure 6 is a flowchart of the fourth embodiment of the state detection method of the opening and closing device. The method can include the following steps:

[0113] S41: Based on the comparison result, it is determined whether the second feature similarity between each second state image feature and the second detection image feature meets at least one state requirement.

[0114] Among them, the at least two state image features include the second state image features corresponding to each preset state image.

[0115] In some embodiments, the second detection image feature is obtained by feature extraction on the to-be-detected image, and the second feature similarity between the second detection image feature and each second state image feature is obtained. Each second feature similarity can be represented as: S i ={s1, s2, …, sk}, where Si represents the second feature similarity between the second detection image feature and the ith second state image feature, k is an integer, and represents the number of second state image features. k

[0116] Taking an example of the preset state image including four preset state images, each second feature similarity can be represented as: S i ={s1, s2, s3, s4}, i<=4.

[0117] ​In some embodiments, each second feature similarity is a comparison result of a second detection image feature of the to-be-detected image and each second state image feature. Based on the comparison result, it can be judged whether the second feature similarity meets at least one state requirement.

[0118] In some embodiments, each at least one state requirement corresponds to a different opening and closing state, and the at least one state requirement includes a state requirement corresponding to the open state and / or a state requirement corresponding to the closed state.

[0119] In some embodiments, the state requirement corresponding to the open state includes: the second feature similarity corresponding to a preset state image representing the closed state and being lower than a first similarity threshold, or the second feature similarity corresponding to at least one preset state image representing the open state and having an opening degree higher than a first degree threshold being higher than a second similarity threshold.

[0120] In some embodiments, the preset state image representing the closed state is a preset state image representing the closed state and having an opening degree less than a preset degree threshold. For example, the preset degree threshold can represent an opening degree of 30%.

[0121] For example, the first similarity threshold is a1, and the second similarity threshold is a2. Taking the preset state images including a preset state image 1 representing the closed state and having an opening degree of 0, a preset state image 2 representing the closed state and having an opening degree of 15%-30%, a preset state image 3 representing the closed state and having an opening degree of 30%-70%, and a preset state image 4 representing the closed state and having an opening degree greater than 70% as an example, the preset state image representing the closed state includes the preset state image 1, and the second feature similarity corresponding to the preset state image representing the closed state being lower than the first similarity threshold can be represented as the second feature similarity corresponding to the preset state image 1 being lower than the first similarity threshold. In addition, taking the first degree threshold being greater than or equal to 15% as an example, the preset state image representing the open state and having an opening degree higher than the first degree threshold includes the preset state image 2, the preset state image 3, and the preset state image 4, that is, the second feature similarity corresponding to the preset state image representing the open state and having an opening degree higher than the first degree threshold is all higher than the second similarity threshold, which is represented as at least one of the second feature similarities corresponding to the preset state image 2, the preset state image 3, and the preset state image 4 being higher than the second similarity threshold. Figure 2 Figure 2

[0122] The state requirement corresponding to the open state can be represented as: s1

[0123] ​​In some embodiments, if the second feature similarity does not satisfy the state requirement corresponding to the open state, the second feature similarity can be determined to satisfy the state requirement corresponding to the closed state. Alternatively, the state requirement corresponding to the closed state comprises: the second feature similarity corresponding to the preset state image representing the closed state is higher than a third similarity threshold, and the second feature similarity corresponding to the preset state image representing the open state and the opening degree being higher than the first degree threshold are all lower than a fourth similarity threshold, wherein the third similarity threshold is higher than or equal to the first similarity threshold, and the fourth similarity threshold is lower than or equal to the second similarity threshold.

[0124] In some embodiments, if it is determined that the second feature similarity satisfies the state requirement corresponding to the open state, step S42 is performed.

[0125] In some embodiments, if it is determined that the second feature similarity satisfies the state requirement corresponding to the closed state, step S43 is performed.

[0126] S42: In response to the second feature similarity satisfying the state requirement corresponding to the open state, it is determined that the open-closed state of the open-closed device is the open state.

[0127] If the second feature similarity satisfies the state requirement corresponding to the open state, it is determined that the open-closed state of the open-closed device is the open state.

[0128] S43: In response to the second feature similarity satisfying the state requirement corresponding to the closed state, it is determined that the open-closed state of the open-closed device is the closed state.

[0129] If the second feature similarity satisfies the state requirement corresponding to the closed state, it is determined that the open-closed state of the open-closed device is the closed state.

[0130] In the embodiment, by obtaining the second feature similarity between the second detection image feature of the to-be-detected image and each second state image feature, and determining whether the second feature similarity satisfies the state requirement corresponding to the open state or the state requirement corresponding to the closed state, the open-closed state of the open-closed device can be determined, which can improve the detection efficiency of detecting the open-closed state of the open-closed device.

[0131] In some embodiments, please refer to Figure 7 , Figure 7 is a flowchart of a fifth embodiment of a state detection method of an open-closed device. The method can include the following steps:

[0132] S51: Based on the comparison result, it is determined whether the second feature similarity between each second state image feature and the second detection image feature satisfies at least one state requirement.

[0133] Each of the at least one state requirement corresponds to a different open-closed state, and the at least one state requirement comprises a state requirement corresponding to the open state.

[0134] S52: In response to the second feature similarity satisfying the state requirement corresponding to the open state, determining that the opening and closing state of the opening and closing device is the open state.

[0135] In some embodiments, the specific implementation process of the above steps S51 to S52 of this embodiment can refer to the specific implementation process of the above steps S41 to S42, which will not be repeated here.

[0136] In some embodiments, when it is judged that the second feature similarity does not satisfy the state requirement corresponding to the open state, the opening and closing state of the opening and closing device can be determined as the closed state in response to the second feature similarity satisfying the state requirement corresponding to the closed state, or the following steps can be performed.

[0137] S53: In response to the second feature similarity not satisfying the state requirement corresponding to the open state, based on the comparison result, selecting the first state image feature with the first feature similarity between the first detection image feature satisfying the similarity requirement as the target state image feature.

[0138] In some embodiments, in the case that the second feature similarity does not satisfy the state requirement corresponding to the open state, that is, in the case that the features of the entire region of the to-be-detected image and each preset state image are compared and the state requirement corresponding to the open state is not satisfied, the features of the target difference region of the to-be-detected image and each preset state image can be further compared to further determine the opening and closing state of the opening and closing device.

[0139] Among the at least one first state image feature, each first state image feature is obtained by feature extraction on a target difference region of a preset state image.

[0140] In some embodiments, the at least one first state image feature corresponds to at least one of the preset state images including the preset state image representing the closed state and the preset state image representing the open state but the opening degree is less than the preset degree threshold.

[0141] In some embodiments, the at least one first state image feature includes the first state image feature corresponding to at least one preset state image representing the closed state. For example, the first state image feature includes the preset state image representing the closed state. Figure 1 The corresponding first state image feature fd1.

[0142] In some embodiments, the at least one first state image feature includes the first state image feature corresponding to at least one preset state image representing the closed state and the first state image feature corresponding to at least one preset state image representing the open state. For example, the first state image feature includes the preset state image representing the closed state and the preset state image representing the open state. Figure 1The first state image feature fd1 corresponds to the first state image feature fd2 of the preset state image 2.

[0143] In some embodiments, a corresponding similarity requirement can be set for each first state image feature.

[0144] The first feature similarity between the first detection image feature of the to-be-detected image and each first state image feature is obtained, and based on the first feature similarity, a first state image feature with a first feature similarity meeting a similarity requirement between the first detection image feature is selected as a target state image feature.

[0145] In some embodiments, the similarity requirement can include at least one of the first feature similarity being greater than a first matching threshold, a maximum value of each first feature similarity, and the like.

[0146] In some embodiments, the first detection image feature is compared with each first state image feature, and a first state image feature with a maximum first feature similarity is selected as a target state image feature.

[0147] S54: Based on the opening and closing degree of the preset state image corresponding to the target state image feature, the opening and closing state of the opening and closing device is obtained.

[0148] In some embodiments, if the opening and closing degree of the preset state image corresponding to the target state image feature represents the open state, the opening and closing state of the opening and closing device is determined as the closed state in response to the opening and closing degree of the preset state image corresponding to the target state image feature representing the closed state.

[0149] In some embodiments, if the opening and closing degree of the preset state image corresponding to the target state image feature represents the closed state, the opening and closing state of the opening and closing device is determined as the open state in response to the opening and closing degree of the preset state image corresponding to the target state image feature representing the open state.

[0150] Taking the first state image features fd1 and fd2 as examples, if the target state image feature is the first state image feature fd1, the opening and closing state of the opening and closing device is determined as the closed state, and if the target state image feature is the first state image feature fd2, the opening and closing state of the opening and closing device is determined as the open state.

[0151] In this embodiment, when the second feature similarity does not meet the state requirement corresponding to the open state, the first state image feature with the first feature similarity meeting the similarity requirement between the first detection image feature and the first state image feature is selected as the target state image feature based on the comparison result in response to the second feature similarity not meeting the state requirement corresponding to the open state, and the open and close state of the opening and closing device is obtained based on the opening and closing degree of the preset state image corresponding to the target state image feature. Even in the case where the opening degree of the opening and closing degree is small, the opening and closing state of the opening and closing device can be more accurately detected, and the accuracy of state detection of the opening and closing device is improved.

[0152] For the above-mentioned embodiments, the present application provides a state detection device of an opening and closing device. Please refer to Figure 8 , Figure 8 is a structural schematic diagram of an embodiment of the state detection device of the opening and closing device of the present application.

[0153] The state detection device 60 of the opening and closing device comprises an acquisition module 61, a feature module 62 and a detection module 63.

[0154] The acquisition module 61 is configured to acquire a plurality of preset state images of the opening and closing device, and the opening and closing device in each preset state image is in a different opening and closing degree.

[0155] The feature module 62 is configured to extract at least two state image features corresponding to at least two opening and closing degrees of the opening and closing device by using the plurality of preset state images.

[0156] The detection module 63 is configured to determine the opening and closing state of the opening and closing device based on the comparison result of the detection image feature of the to-be-detected image and the at least two state image features.

[0157] The specific implementation of this embodiment can refer to the implementation process of the above-mentioned embodiments, which will not be repeated here.

[0158] For the above-mentioned embodiments, the present application provides a computer device, please refer to Figure 9 , Figure 9 is a structural schematic diagram of an embodiment of the computer device of the present application. The computer device 70 comprises a memory 71 and a processor 72, wherein the memory 71 and the processor 72 are coupled to each other, the memory 71 stores program data, and the processor 72 is configured to execute the program data to realize any step of any embodiment of the above-mentioned state detection method of the opening and closing device.

[0159] In this embodiment, the processor 72 can also be referred to as a CPU (Central Processing Unit). The processor 72 can be an integrated circuit chip having a processing capability of signals. The processor 72 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor 72 can also be any conventional processor or the like.

[0160] The specific implementation of this embodiment can refer to the implementation process of the above embodiments, which will not be repeated here.

[0161] For the method of the above embodiments, it can be implemented in the form of a computer program, and thus the present application provides a storage device. Please refer to Figure 10 , Figure 10 is a structural schematic diagram of an embodiment of the storage device of the present application. The storage device 80 stores program data 81 capable of being executed by a processor. The program data 81 can be executed by the processor to implement any step of any embodiment of the state detection method of the opening and closing device.

[0162] The storage device 80 of this embodiment can be a U disk, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk or an optical disk, etc. which can store the program data 81, or can also be a server storing the program data 81. The server can send the stored program data 81 to other devices for execution, or can also execute the stored program data 81 by itself.

[0163] The specific implementation of this embodiment can refer to the implementation process of the above embodiments, which will not be repeated here.

[0164] In several embodiments provided in the present application, it should be understood that the disclosed method and device can be implemented in other ways. For example, the above-described device embodiment is only schematic, for example, the division of the module or unit is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual ones can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0165] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0166] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0167] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage device, which is a computer readable storage medium. Based on this understanding, the technical scheme of the present application or the essential part or all or part of the technical scheme that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for making an electronic device (which can be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the method of each embodiment of the present application.

[0168] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be realized by general computing devices, which can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be realized by program codes executable by computing devices, so that they can be stored in storage devices and executed by computing devices, or they can be made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to realize. Therefore, the present application is not limited to any specific hardware and software combination.

[0169] The above is only an embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method of detecting a state of an opening and closing device, characterized by, The method comprises: acquiring a plurality of preset state images of an opening and closing device, each of the plurality of preset state images being at a different opening and closing degree of the opening and closing device; extracting at least two state image features corresponding to at least two opening and closing degrees of the opening and closing device from the plurality of preset state images; determining an opening and closing state of the opening and closing device based on a comparison result of a detection image feature of a to-be-detected image and the at least two state image features. The at least two state image features include at least one first state image feature and a plurality of second state image features; the first state image feature is extracted based on a target difference region of at least one of the preset state images, and the target difference region represents an opening and closing direction of the opening and closing device; and the second state image feature is extracted from each of the preset state images.

2. The method of claim 1, wherein, The extraction of the at least two state image features corresponding to the at least two opening and closing degrees of the opening and closing device from the plurality of preset state images comprises: acquiring a target difference region between the plurality of preset state images, and extracting a feature of the target difference region of at least one of the preset state images to obtain at least one first state image feature; extracting a second state image feature corresponding to each of the preset state images.

3. The method of claim 2, wherein, The acquisition of the target difference region between the plurality of preset state images comprises: acquiring a first difference region between each two of the plurality of preset state images; obtaining the target difference region based on the first difference region between each two of the plurality of preset state images.

4. The method of claim 3, wherein, The acquisition of the first difference region between each two of the plurality of preset state images comprises: respectively taking each two of the plurality of preset state images as an image pair; dividing one of the preset state images in the image pair into a plurality of first regions, and dividing the other preset state image in the image pair into a plurality of second regions, the plurality of first regions and the plurality of second regions one-to-one corresponding, and each of the first regions and the corresponding second region forming a region pair; determining a first difference region between the image pair based on a region similarity of each of the region pairs in the image pair.

5. The method of claim 4, wherein, The determination of the first difference region between the image pair based on the region similarity of each of the region pairs in the image pair comprises: selecting at least one target region pair in the region pairs included in the image pair, the region similarity of the target region pair in the image pair satisfying a difference region requirement; and combining image positions corresponding to the at least one target region pair to obtain the first difference region between the image pair.

6. The method of claim 3, wherein, The determination of the first difference region between the image pair based on the region similarity of each of the region pairs in the image pair comprises: performing an intersection operation on the first difference regions between each two of the plurality of preset state images to obtain the target difference region.

7. The method of claim 1, wherein, The at least two state image features include at least one first state image feature, and each first state image feature is obtained by performing feature extraction on a target difference region of one of the preset state images. The comparison result of the detection image feature of the to-be-detected image and the at least two state image features is used to determine the opening and closing state of the opening and closing device, including: Based on the comparison result, the first state image feature that satisfies the similarity requirement with the first detection image feature in terms of first feature similarity is selected as a target state image feature, wherein the first detection image feature is obtained by performing feature extraction on a second difference region of the to-be-detected image. The opening and closing state of the opening and closing device is determined based on the opening and closing degree of the preset state image corresponding to the target state image feature.

8. The method of claim 1, wherein, The at least two state image features include a second state image feature corresponding to each of the preset state images. The comparison result of the detection image feature of the to-be-detected image and the at least two state image features is used to determine the opening and closing state of the opening and closing device, including: Based on the comparison result, it is determined whether the second feature similarity between each second state image feature and a second detection image feature satisfies at least one state requirement; wherein each of the at least one state requirement corresponds to a different opening and closing state, and the at least one state requirement includes a state requirement corresponding to an open state. In response to the second feature similarity satisfying the state requirement corresponding to the open state, it is determined that the opening and closing state of the opening and closing device is the open state.

9. The method of claim 8, wherein, The at least one state requirement also includes a state requirement corresponding to a closed state, and the comparison result of the detection image feature of the to-be-detected image and the at least two state image features is used to determine the opening and closing state of the opening and closing device, including: In response to the second feature similarity satisfying the state requirement corresponding to the closed state, it is determined that the opening and closing state of the opening and closing device is the closed state. Alternatively, the at least two state image features also include at least one first state image feature, and each first state image feature is obtained by performing feature extraction on a target difference region of one of the preset state images, and the comparison result of the detection image feature of the to-be-detected image and the at least two state image features is used to determine the opening and closing state of the opening and closing device, including: In response to the second feature similarity not satisfying the state requirement corresponding to the open state, the first state image feature that satisfies the similarity requirement with the first detection image feature in terms of first feature similarity is selected as a target state image feature based on the comparison result. The opening and closing state of the opening and closing device is determined based on the opening and closing degree of the preset state image corresponding to the target state image feature.

10. The method of claim 8, wherein, The state requirement corresponding to the open state includes: the second feature similarity of the preset state image whose opening and closing degree represents the closed state is lower than a first similarity threshold, or the second feature similarity of at least one of the preset state images whose opening and closing degree represents the open state and whose opening degree is higher than a first degree threshold is higher than a second similarity threshold.

11. The method according to claim 7 or 9, characterized in that, The at least one first state image feature corresponds to at least one preset state image, and the at least one preset state image includes a preset state image representing the closed state of the opening and closing degree and a preset state image representing the open state of the opening and closing degree less than a preset degree threshold; And / or, the opening and closing state of the opening and closing device is determined based on the opening and closing degree of the preset state image corresponding to the target state image feature, including: In response to the opening and closing degree of the preset state image corresponding to the target state image feature representing the closed state, the opening and closing state of the opening and closing device is determined as the closed state; In response to the opening and closing degree of the preset state image corresponding to the target state image feature representing the open state, the opening and closing state of the opening and closing device is determined as the open state.

12. The method of claim 1, wherein, Before determining the opening and closing state of the opening and closing device based on the comparison result of the detection image feature of the to-be-detected image and the at least two state image features, the method further includes: capturing a scene image of a scene in which the opening and closing device is located; In response to the target entity not existing in the scene image or the target entity existing in the scene image and the target entity not having a positional overlapping relationship with the opening and closing device, an image part corresponding to the opening and closing device in the scene image is taken as the to-be-detected image.

13. A computer device, comprising: The memory and the processor are coupled with each other, the memory stores program data, and the processor is configured to execute the program data to implement the steps of the method in any one of claims 1 to 12.

14. A memory device, comprising: The memory stores program data capable of being executed by the processor, and the program data is used to implement the steps of the method in any one of claims 1 to 12.

Images