A door installation control system and method for verifying a correct installation of a door

Abstract

The disclosure relates to a door installation control system (100) for verifying a correct installation of a door (1), comprising a processing circuitry (102a,102b) operatively connected to a camera sensor (5) and an output interface (20), and configured to obtain a real-time image data of an installed component (ic1,ic2,ic3) associated with installation of the door (1) and output visual data of the installed component (ic1,ic2,ic3) in real-time together with a visual augmented image of an augmented component (ac1,ac2,ac3) according to an expected placement to verify a correct installation of the installed component (ic1,ic2,ic3). The disclosure further relates to a method for verifying a correct installation of a door (1) and a computer program product (500).

Description

[0001] A door installation control system and method for verifying a correct installation of a door

[0002] Technical field

[0003] The present disclosure relates to a door installation control system for verifying a correct installation of a door, a method for verifying a correct installation of a door and a computer program product. More specifically, the disclosure relates to a door installation control system for verifying a correct installation of a door, a method for verifying a correct installation of a doorand a computer program product as defined in the introductory parts of the independent claims.

[0004] Background art

[0005] Installation of doors, and components associated with the installation of doors, requires manual work by a user, typically a door installer. One issue that the door installer can run into when installing a door is that the hardware part of the installation is incorrectly performed.

[0006] For example, the door installer may have used incorrect dimensions, e.g. the distance from a door operator to the wall may be incorrect, or that a component is installed with an incorrect angle in relation to another component, e.g. that a backplate is not horizontal, or that an arm system does not have a correct installation angle etc. Other problems are for example that a hardware component is incorrectly placed, or that a hardware component is missing after the installation etc.

[0007] An incorrect installation can affect the functionality of the door and its components, and also lead to e.g. unplanned required maintenance of the door or higher power consumption etc.Summa

[0008] It is an object of the present disclosure to mitigate, alleviate or eliminate one or more of the above-identified deficiencies and disadvantages in the prior art and solve at least the above mentioned problem.

[0009] According to a first aspect there is provided a door installation control system for verifying a correct installation of a door. The system comprises a camera sensor configured to obtain real-time image data of a component associated with installation of the door, an output interface configured to output visual data associated with installation of the door, a processing circuitry operatively connected to the camera sensor and the output interface configured to obtain a real-time image data of an installed component associated with installation of the door and output visual data of the installed component in real-time together with a visual augmented image of an augmented component according to an expected placement to verify a correct installation of the installed component.

[0010] An advantage with this first aspect is that it is very intuitive for the user, e.g. the door installer, to see via the output interface, visual data showing the installed component in realtime together with a visual augmented image of the augmented component, to understand if the installed component is correctly installed.

[0011] According to some embodiments, the door installation control system further comprises an input interface configured to receive predetermined reference installation data associated with the installation of the door and the processing circuitry is operatively connected to the camera sensor, the input interface and the output interface. The processing circuitry is further configured to receive predetermined reference installation data associated with installation of the door, identify the installed component by applying image recognition on the obtained real-time image data and utilize the predetermined reference installation data, and determine the placement of the identified installed component in relation to an expected placement according to the predetermined reference installation data.

[0012] An advantage with this embodiment is that the predetermined reference installation data supports the identification of the installed component and the determination if the placement of the identified installed component in relation to an expected placement is correct.

[0013] According to some embodiments, in a determination that the placement of the installed component in relation to the placement the augmented component deviates, thenoutput visual guidance indicative of instructions for correct placement of the installed component.

[0014] An advantage with this embodiment is that that the visual guidance, which can be seen by the user in real-time, is intuitive to the user, to e.g. adjust or re install a component so that the installation becomes correct.

[0015] According to some embodiments, the visual guidance is indicative of at least any of an angle, a distance, a measurement, a position, a direction, an adjustment, a replacement component, a reinstallation instruction, an alternative placement and an alternative component.

[0016] An advantage with this embodiment is that the user can get more precise information how to achieve a correct installation of the installed component, e.g. knowledge about a distance and a direction to obtain a correct installation.

[0017] According to some embodiments, the visual guidance is indicative of a level of deviation.

[0018] An advantage with this embodiment is that the level of deviation can intuitively indicate to the user how severe the incorrect installation is in relation to an expected placement to indicate e.g. how important the need for replacement or adjustment of the installed component is.

[0019] According to some embodiments the deviation is determined by comparing the placement of the installed component icl, ic2, ic3 in relation to the expected placement, as visualised with the augmented component acl, ac2, ac3.

[0020] According to some embodiments, the level of deviation is determined by comparing the deviation with at least a first predetermined threshold value of deviation.

[0021] An advantage with this embodiment is that the relationship between the different levels becomes more precise.

[0022] According to some embodiments, the predetermined reference installation data is received by obtaining user input via the input interface.

[0023] An advantage with this embodiment is that the user can select the reference installation data to be used to verify the correct installation of the installed component.According to some embodiments, the predetermined reference installation data is received upon obtaining door identity data via the input interface by at least any of receiving user input indicative of the door identity data via the input interface, scanning by the camera sensor a code indicative of the door identity data, obtaining image data of the door by the camera sensor and identifying the door using image recognition indicative of the door identity data, and scanning by the camera sensor a text indicative of the door identity data.

[0024] An advantage with this embodiment is that the predetermined reference installation data can be obtained intuitively by the user, e.g. by the door installer or by a door technician, when present in the visual vicinity at the location of the door, to verify the installation.

[0025] According to some embodiments, the predetermined reference installation data is generated by a learning model built on training data obtained over time by user verified door installations.

[0026] An advantage with this embodiment is that training data e.g. using images from custom door installations can be used to generate predetermined reference installation data.

[0027] According to some embodiments, the processing circuitry is further configured to output visual instruction data indicative of how to move the camera sensor in order to obtain the real-time image data from different angles and / or distances in relation to the door.

[0028] An advantage with this embodiment is that the user can get information how to obtain sufficient real-time image data, from e.g. different angles, to have enough and reliable realtime image data of the component associated with installation of the door.

[0029] According to a second aspect there is provided a method for verifying a correct installation of a door. The method comprising obtaining, by a camera sensor, a real-time image data of an installed component associated with installation of the door, and outputting, by an output interface, visual data of the installed component in real-time together with a visual augmented image of an augmented component according to an expected placement to verify a correct installation of the installed component.

[0030] An advantage with this second aspect is that it is very intuitive for the user, e.g. the door installer, to see via the output interface visual data showing the installed component in real-time together with a visual augmented image of the augmented component, to understand if the installed component is correctly installed.According to some embodiments, the method further comprises the step of receiving, by an input interface, predetermined reference installation data associated with installation of the door, the step of identifying the installed component by applying image recognition on the obtained real-time image data and utilizing the predetermined reference installation data, and the step of determining the placement of the identified installed component in relation to an expected placement according to the predetermined reference installation data.

[0031] An advantage with this embodiment is that the predetermined reference installation data supports the identification of the installed component and the determination if the placement of the identified installed component in relation to an expected placement is correct.

[0032] According to some embodiments, in a determining that the placement of the installed component in relation to the placement the augmented component deviates, the step of outputting, by the by output interface, visual guidance indicative of instructions for correct placement of the installed component.

[0033] An advantage with this embodiment is that that the visual guidance, which can be seen by the user in real-time, is intuitive to the user, to e.g. adjust or re install a component so that the installation becomes correct.

[0034] According to some embodiments, the method comprises the step of outputting, by the by output interface, visual instruction data indicative of how to move the camera sensor in order to obtain the real-time image data from different angles and / or distances in relation to the door.

[0035] An advantage with this embodiment is that the user can get information how to obtain sufficient real-time image data, from e.g. different angles, to have enough and reliable realtime image data of the component associated with installation of the door.

[0036] According to a third aspect there is provided a computer program product comprising a non-transitory computer readable medium, having thereon a computer program comprising program instructions, the computer program being loadable into a processing circuitry and configured to cause execution of the method according to any of the second aspectwhen the computer program is run by the processing circuitry.Effects and features of the second and third aspects are to a large extent analogous to those described above in connection with the first aspect. Embodiments mentioned in relation to the first aspect are largely compatible with the the second and third aspects.

[0037] The present disclosure will become apparent from the detailed description given below. The detailed description and specific examples disclose preferred embodiments of the disclosure by way of illustration only. Those skilled in the art understand from guidance in the detailed description that changes and modifications may be made within the scope of the disclosure.

[0038] Hence, it is to be understood that the herein disclosed disclosure is not limited to the particular component parts of the device described or steps of the methods described since such device and method may vary. It is also to be understood that the terminology used herein is for purpose of describing particular embodiments only, and is not intended to be limiting. It should be noted that, as used in the specification and the appended claim, the articles "a", "an", "the", and "said" are intended to mean that there are one or more of the elements unless the context explicitly dictates otherwise. Thus, for example, reference to "a unit" or "the unit" may include several devices, and the like. Furthermore, the words "comprising", "including", "containing" and similar wordings does not exclude other elements or steps.

[0039] Brief descriptions of the drawings

[0040] The above objects, as well as additional objects, features and advantages of the present disclosure, will be more fully appreciated by reference to the following illustrative and non-limiting detailed description of example embodiments of the present disclosure, when taken in conjunction with the accompanying drawings.

[0041] Figure 1 illustrates an example door installation control system for verifying a correct installation of a door according to an embodiment of the present disclosure, illustrated with a swing door as the door and wherein the camera sensor and the output interface are integrated in a smartphone.

[0042] Figure 2 illustrates example visual data as can be seen by the user via the output interface, with example installed components together with example visual augmented images of augmented components.Figure 3 illustrates example visual guidance as can be seen by the user via the output interface, indicative of instructions for correct placement of the installed component according to an embodiment of the present disclosure.

[0043] Figure 4 illustrates example visual guidance as can be seen by the user via the output interface, wherein the visual guidance is indicative of distances.

[0044] Figure 5 illustrates a flow chart of the method steps according to the second aspect of the disclosure.

[0045] Figure 6 illustrates a computer program product according to the third aspect of the disclosure.

[0046] Detailed description

[0047] The present disclosure will now be described with reference to the accompanying drawings, in which preferred example embodiments of the disclosure are shown. The disclosure may, however, be embodied in other forms and should not be construed as limited to the herein disclosed embodiments. The disclosed embodiments are provided to fully convey the scope of the disclosure to the skilled person.

[0048] The first aspect of this disclosure shows a door installation control system 100 for verifying a correct installation of a door 1.

[0049] According to some embodiments an installation of a door includes installation of a component associated with the door or installation of a component associated the surroundings of the door. For example, it can be a single component or multiple components, a subgroup of components, associated with installation of a door and / or a doorframe. The component can be a mechanical component, an electrical component, or any tangible component.

[0050] Figure 1 illustrates an example door installation control system 100 for verifying a correct installation of a door according to an embodiment of the present disclosure.

[0051] According to some embodiments, the door 1 is any of an overhead sectional door, a folding door, a vertical-lifting door, a high-speed door, a garage door, a mega door, a revolving door, a slider door, a hangar door, a sliding doorand a swing door. In further example embodiments the door 1 is any entrance shield. According to some embodiments, the door 1is any of a roller shutter, a shield and a roadblock. In the Figure 1 the door 1 is illustrated as a swing door.

[0052] The system 100 comprises a camera sensor 5 configured to obtain real-time image data of a component associated with installation of the door 1, and an output interface 20 configured to output visual data associated with installation of the door 1. In Figure 1 the camera sensor 5 and the output interface 20 are integrated in a portable electronic device 800, in the illustration a smartphone.

[0053] According to some embodiments the camera sensor 5 is an arbitrary stand-alone camera sensor. According to some embodiments the output interface 20 is an arbitrary standalone output interface. According to some embodiments camera sensor 5 and the output interface 20 are integrated in a portable electronic device configured to verify a correct installation of a door.

[0054] In an example camera sensor 5 is the camera of a smartphone and the output interface 20 is the display of a smartphone.

[0055] The system 100 comprises a processing circuitry 102a, 102b operatively connected to the camera sensor 5 and the output interface 20.

[0056] According to some embodiments the processing circuitry 102a is comprised in a portable electronic device 800. According to some embodiments, the processing circuitry 102b is comprised in a remote device 400 that is a stationary electronic device.

[0057] According to some embodiments the door installation control system 100 further comprises a memory 101a, 101b operatively connected to processing circuitry 102a, 102b.

[0058] According to some embodiments the memory 101a is comprised in in a portable electronic device 800. According to some embodiments, the memory 101b is comprised in a remote device 400 that is a stationary electronic device.

[0059] According to some embodiments the camera sensor 5, the output interface 20, and the processing circuitry 102a, 102b are operatively connected via a communication network 50 as illustrated in Figure 1. According to some embodiments the communication network 50 is a wireless communication network.

[0060] According to some embodiments the wireless communication network is a standardized wireless local area network such as a Wireless Local Area Network, WLAN,Bluetooth™, ZigBee, Ultra-Wideband, UWB, Radio Frequency Identification, RFID, or similar network.

[0061] According to some embodiments the wireless communication network is a standardized wireless wide area network such as a Global System for Mobile Communications, GSM, Extended GSM, General Packet Radio Service, GPRS, Enhanced Data Rates for GSM Evolution, EDGE, Wideband Code Division Multiple Access, WCDMA, Long Term Evolution, LTE, Narrowband-loT, 5G, Worldwide Interoperability for Microwave Access, WiMAX or Ultra Mobile Broadband, UMB or similar network.

[0062] According to some embodiments the wireless communication network can also be a combination of both a wireless local area network and a wireless wide area network.

[0063] According to some embodiments communication network 50 can be a combination of a wired communication network and a wireless communication network. According to some embodiments the communication network 50 is defined by common Internet Protocols.

[0064] The processing circuitry 102a, 102b is configured to obtain a real-time image data of an installed component icl,ic2,ic3 associated with installation of the door 1.

[0065] According to some embodiments the installed component icl, ic2,ic3 is at least any of a door control unit, a motor unit, a push arm, a pull arm, a stop arm, a cable, a transmission unit, a cover plate, a battery, a back plate, a joint part, a screw, a spacer, a nut, a sign, an handle, a lock, a hinge, or any component associated with installation of the door 1. In the example illustration in Figure 1, the installed component icl is a door control unit, the installed component ic2 is a push arm and the installed component ic3 is a cover plate.

[0066] The processing circuitry 102a, 102b is further configured to output visual data of the installed component icl,ic2,ic3 in real-time together with a visual augmented image of an augmented component acl,ac2,ac3 according to an expected placement to verify a correct installation of the installed component icl,ic2,ic3.

[0067] According to some embodiments to verify a correct installation of the installed component icl,ic2,ic3 also includes to verify if a component is missing, that has not been installed. According to some embodiments the image of the augmented component acl,ac2,ac3 reveals if a component is missing. According to some embodiments to verify a correct installation of the installed component icl,ic2,ic3 includes to verify if the installed component icl,ic2,ic3 is the correct component. According to some embodiments the imageof the augmented component 361,362,363 reveals if the installed component icl,ic2,ic3 is incorrect.

[0068] Figure 2 illustrates example visual data as can be seen by the user via the output interface 20, with example installed components icl,ic2,ic3 together with example visual augmented images of augmented components acl,ac2,ac3. It is understood that in the illustration in Figure 2, the visual data is exemplified in a two-dimensional drawing, but that the visual data can be seen in three dimensional when outputted via the output interface 20.

[0069] According to some embodiments the output visual data is provided by a video stream of obtained real-time images of the installed component icl,ic2,ic3 that are obtained by the camera sensor 5.

[0070] According to some embodiments the visual augmented image of the augmented component acl,ac2,ac3 is computer generated and provided in the output visual data.

[0071] According to some embodiments the augmented component acl, ac2, ac3 is visualised at an expected placement according to predetermined reference installation data. According to some embodiments the augmented component acl, ac2, ac3 is visualised at an expected placement in relation to the door 1. According to some embodiments the augmented component acl, ac2, ac3 is visualised at an expected placement in relation to the doorframe.

[0072] An advantage with this first aspect is that it is very intuitive for the user, e.g. the door installer, to see via the output interface 20, the visual data showing the installed component icl,ic2,ic3 in real-time together with the visual augmented image of the augmented component acl, ac2, ac3, to understand if the installed component is correctly installed.

[0073] According to some embodiments the door installation control system 100 further comprises an input interface 10 configured to receive predetermined reference installation data associated with the installation of the door 1.

[0074] According to some embodiments the predetermined reference installation data is retrieved from the memory 101a, 101b and received via the input interface 10. According to some embodiments the predetermined reference installation data is received via the input interface 10 from the remote device 400 via the communication network 50.

[0075] According to some embodiments the predetermined reference installation data comprising predefined installation data associated with a correct installation of a door 1.According to some embodiments the predetermined reference installation data is at least any of three-dimensional drawing data, computer aided design drawing data, artificial intelligence data, assembly data, component data, image data, meta data and training data.

[0076] According to some embodiments the predetermined reference installation data is virtual reality data.

[0077] According to some embodiments processing circuitry 102a, 102b is operatively connected to the camera sensor 5, the input interface 10 and the output interface 20, and the processing circuitry 102a, 102b is further configured to receive predetermined reference installation data associated with installation of the door 1, identify the installed component icl,ic2,ic3 by applying image recognition on the obtained real-time image data and utilize the predetermined reference installation data, and determine the placement of the identified installed component icl,ic2,ic3 in relation to an expected placement according to the predetermined reference installation data.

[0078] According to some embodiments the predetermined reference installation data comprises image data and by applying image recognition on the obtained real-time image data, the installed component icl,ic2,ic3 can be identified.

[0079] According to some embodiments placement of the identified installed component icl,ic2,ic3 can be determined in relation to an expected placement according to the predetermined reference installation data. According to some embodiments the predetermined reference installation data is indicative of an expected placement in relation to the door 1. According to some embodiments the predetermined reference installation data is indicative of an expected placement in relation to the door frame.

[0080] According to some embodiments the expected placement is determined by assistance of at least any of gyro data, accelerometer data, global positioning system data, and reference image data.

[0081] An advantage with this embodiment is that the predetermined reference installation data supports the identification of the installed component and the determination if the placement of the identified installed component in relation to an expected placement is correct.

[0082] According to some embodiments in a determination that the placement of the installed component icl,ic2,ic3 in relation to the placement of the augmented component361,362,363 deviates the processing circuitry 102s, 102b is further configured to output visual guidance vg i nd icative of instructions for correct placement of the installed component icl,ic2,ic3.

[0083] Figure 3 illustrates example visual guidance vg as can be seen by the user via the output interface, indicative of instructions for correct placement of the installed component icl,ic2,ic3. It is understood that in the illustration in Figure 3, the visual data is exemplified in a two-dimensional drawing, but that the visual data can be seen in three dimensional when outputted via the output interface 20.

[0084] In Figure 3 the visual guidance vg is illustrated in the form of an arrow indicating the direction the installed component icl,ic2,ic3 should be moved in order to be installed according to an expected placement.

[0085] According to some embodiments the visual guidance vg is indicative of at least any of an angle, a distance, a measurement, a position, a direction, an adjustment, a replacement component, a reinstallation instruction, an alternative placement and an alternative component.

[0086] In the example in Figure 3, the installed component icl is a door control unit incorrectly installed at the door frame, and the visual guidance vg is illustrated in the form of an arrow indicating in what direction the door control unit should be moved in order to be correctly installed.

[0087] In Figure 4, the installed component icl is a door control unit incorrectly installed at the door frame, and the visual guidance vg is illustrated in the form of a distance and indicating in what direction the door control unit should be moved in order to be correctly installed. It is understood that in the illustration in Figure 4, the visual data is exemplified in a two-dimensional drawing, but that the visual data can be seen in three dimensional when outputted via the output interface 20.

[0088] According to some embodiments the processing circuitry 102a, 102b is further configured to output at least a first visual guidance.

[0089] In an example, in a determination that the placement of plural installed components icl, ic2, ic3 deviates, plural visual guidance appears.In an example, in a determination that the placement of one single installed component deviates, plural visual guidance can appear so provide instructions for correct placement of that single installed component.

[0090] In Figure 4, the installed component icl is a door control unit incorrectly installed at the door frame, and the visual guidance vg are illustrated in the form of two distances with indications of directions to provide instructions for correct placement of the installed component.

[0091] An advantage with this embodiment is that the user can get more precise information how to achieve a correct installation of the installed component icl,ic2,ic3, e.g. knowledge about a distance and a direction to obtain a correct installation. Another advantage is that it may not be necessary to write explanatory texts that needs interpretation and translation but instead use intuitive graphical visual guidance vg that minimizes the need for texts but instead the user will intuitively see via the output interface what action that needs to be taken without explanatory texts.

[0092] According to some embodiments the visual guidance vg is changing in real-time so that a movement of the installed component icl, ic2, ic3 causes a change in the visual guidance vg in relation to the placement the augmented component acl, ac2, ac3 accordingly.

[0093] An advantage with this embodiment is that a correct installation of the installed component icl, ic2, ic3 can be verified in real-time. In an example a user, e.g. a door installer can, in real-time adjust an installed component icl, ic2, ic3, and get visual guidance vg in realtime to verify a correct installation. In an example the installed component is a push arm that is not precisely horizontally installed, but with some degrees deviation and the door installer can adjust the installation and verify in real-time when the push arm is horizontally installed.

[0094] According to some embodiments the visual guidance vg is indicative of a level of deviation. According to some embodiments the visual augmented image of the augmented component acl, ac2, ac3 is indicative of a level of deviation.

[0095] An advantage with these two embodiments is that the level of deviation can intuitively indicate to the user how severe the incorrect installation is in relation to an expected placement to indicate e.g. how important the need for replacement or adjustment of the installed component is.According to some embodiments the deviation is determined by comparing the placement of the installed component icl, ic2 ic3 in relation to the expected placement, as visualised with the augmented component acl, ac2, ac3.

[0096] According to some embodiments, the level of deviation is determined by comparing the deviation with at least a first predetermined threshold value of deviation.

[0097] According to some embodiments the level of deviation is represented by at least any of a colour, a scale, a percentage, and a graphic representation.

[0098] According to some embodiments in a determination that the deviation is below a first predetermined threshold value, the visual guidance vg and / or the visual augmented image of the augmented component acl, ac2, ac3 is represented by at least any of a first colour, a first scale, a first percentage, and a first graphic representation.

[0099] In an example, in a determination that the deviation is below a first predetermined threshold value for what is an acceptable placement of the installed component icl, ic2, ic3, the visual guidance vg and / or the visual augmented image of the augmented component acl, ac2, ac3 is represented by a green colour. In an example the installed component is a push arm that is not precisely horizontally installed, but with a five degrees deviation, which is acceptable.

[0100] According to some embodiments, in a determination that the deviation is above the first predetermined threshold value but below a second predetermined threshold value, the visual guidance vg and / or the visual augmented image of the augmented component acl, ac2, ac3 is represented by at least any of a second colour, a second scale, a second percentage, and a second graphic representation.

[0101] In an example, in a determination that the deviation is above the first predetermined threshold value but below a second predetermined threshold value for what is an acceptable placement of the installed component icl, ic2, ic3, the visual guidance vg and / or the visual augmented image of the augmented component acl, ac2, ac3 is represented by a yellow colour. In the example the yellow colour indicates that the installed component icl, ic2, ic3 will function but with the risk of shorter lifetime. In an example the installed component is a push arm that is not precisely horizontally installed, but with a 10 degrees deviation.

[0102] According to some embodiments, in a determination that the deviation is above the second predetermined threshold value, the visual guidance vg and / or the visual augmentedimage of the augmented component acl, ac2, ac3 is represented by at least any of a third colour, a third scale, a third percentage, and a third graphic representation.

[0103] In an example, in a determination that the deviation is above the second predetermined threshold value for what is a non-acceptable placement of the installed component icl, ic2, ic3, the visual guidance vg and / or the visual augmented image of the augmented component acl, ac2, ac3 is represented by a red colour. In an example the installed component is a push arm that is not precisely horizontally installed, but with a 15 degrees deviation.

[0104] An advantage with this embodiment is that the relationship between the different levels becomes more precise.

[0105] According to some embodiments the deviation is determined by comparing the placement of a first installed component icl in relation to the expected placement, as visualised with a first augmented component acl, and the level of deviation is determined by comparing the deviation with at least a first predetermined threshold value of deviation associated with the first installed component icl.

[0106] According to some embodiments the deviation is determined by comparing the placement of a second installed component ic2 in relation to the expected placement, as visualised with a second augmented component ac2, and the level of deviation is determined by comparing the deviation with at least a first predetermined threshold value of deviation associated with the second installed component ic2.

[0107] Hence, for the first installed component icl a first a predetermined threshold value of deviation associated with the first installed component icl is used for determining the deviation, while for the second installed component ic2 a first a predetermined threshold value of deviation associated with the second installed component ic2 is used for determining the deviation.

[0108] In an example, a first installed component icl may be installed allowing e.g. a certain degree of deviation, e.g. the first predetermined threshold value of deviation associated with the first installed component icl is five degrees, and hence "acceptable" if the deviation is four degrees, while a second installed component ic2 may be installed only allowing e.g. two degrees of deviation which is the first predetermined threshold value of deviation associated with the second installed component ic2, and hence "non-acceptable" if the deviation is four degrees.Different installed components may have different threshold values for determining the deviation for each component.

[0109] According to some embodiments the predetermined reference installation data is received by obtaining user input via the input interface 10. According to some embodiments the predetermined reference installation data is received after interaction of a user interface operatively connected to the input interface 10.

[0110] In an example, the user may manually select a door identity in a drop-down menu of the user interface operatively connected to the input interface 10. In an example, the user, e.g. a door installer, can select the door identity of the door to be installed via the drop-down menu, and the predetermined reference installation data is then retrieved from the memory 101a, 101b and received via the input interface 10.

[0111] In an example, the user may manually input custom made installation data such as distance data for installation of a certain component at a certain door, via the user interface operatively connected to the input interface 10.

[0112] An advantage with this embodiment is that the user can select the reference installation data to be used to verify the correct installation of the installed component.

[0113] According to some embodiments the predetermined reference installation data is received upon obtaining door identity data via the input interface 10 by receiving user input indicative of the door identity data via the input interface 10. In an example, a user can type in the door identity data via a user interface operatively connected to the input interface 10, and with knowledge of the door identity data, the predetermined reference installation data can then be retrieved from the memory 101a, 101b and received via the input interface 10.

[0114] According to some embodiments the predetermined reference installation data is received upon obtaining door identity data via the input interface 10 by scanning, by the camera sensor 5, a code indicative of the door identity data. In an example, a QR code may comprise the door identity data, and with knowledge of the door identity data the predetermined reference installation data can then be retrieved from the memory 101a, 101b and received via the input interface 10.

[0115] According to some embodiments the predetermined reference installation data is received upon obtaining door identity data via the input interface 10 by obtaining image data of the door 1, by the camera sensor 5, and identifying the door using image recognitionY1

[0116] indicative of the door identity data. In an example, the door is identified and associated with a certain door identity data, and with knowledge of the door identity data the predetermined reference installation data can then be retrieved from the memory 101a, 101b and received via the input interface 10.

[0117] According to some embodiments the predetermined reference installation data is received upon obtaining door identity data via the input interface 10 by scanning by the camera sensor 5 a text indicative of the door identity data. In an example, the door is identified by at least any of a sticker with door identity text, a sticker with an icon or a sticker with regulatory requirement text, and with knowledge of the door identity data the predetermined reference installation data can then be retrieved from the memory 101a, 101b and received via the input interface 10.

[0118] An advantage with this embodiment is that the predetermined reference installation data can be obtained intuitively by the user, e.g. by the door installer or by a door technician, when present in the visual vicinity at the location of the door, to verify the installation.

[0119] According to some embodiments the predetermined reference installation data is generated by a learning model built on training data obtained over time by user verified door installations.

[0120] In an example, a door installer that has verified a correct installation of a door, can be prompted to obtain image data of the door, or component or plural components associated with the installation of the door, to be stored in the memory 101a, 101b to be used in the future to verify a correct installation. In an example, image data of plural verified installations of a certain door can be used to build a database to be used in the future to verify a correct installation.

[0121] An advantage with this embodiment is that training data e.g. using images from custom door installations can be used to generate predetermined reference installation data.

[0122] According to some embodiments the processing circuitry 102a, 102b is further configured to output visual instruction data indicative of how to move the camera sensor 5 in order to obtain the real-time image data from different angles and / or distances in relation to the door 1.

[0123] In an example, in order to provide with correct and with sufficient image data of the component associated with installation of the door 1, the user can be prompted to move thecamera sensor 5 according to a certain movement in order to completely cover the component. In an example, the user gets intuitive feedback via the output interface 20, what part of the component has been covered and what part that has not been covered.

[0124] According to some embodiments the output visual instruction data comprising feedback data indicative of what part of the installed component icl, ic2, ic3 where real-time image data has been obtained and what part of the installed component icl, ic2, ic3 where real-time image data is desired. In an example, the feedback data can be colours, so that the covered part is green, and the uncovered part is red. In an example, the user is prompted to continue to move the camera sensor 5 until the whole component is covered, e.g. is green. In an example the user is prompted to move the camera sensor 5 closer to the component or to move the camera sensor 5 away from the component.

[0125] According to some embodiments, the level of detail of the real-time image data to be obtained can be set. In an example, the more precise a component has to be installed, e.g. very little deviation is accepted, the higher level of detail may be desired.

[0126] An advantage with this embodiment is that the user can get information how to obtain sufficient real-time image data, from e.g. different angles, to have enough and reliable realtime image data of the component associated with installation of the door.

[0127] The second aspect of this disclosure shows a method for verifying a correct installation of a door 1.

[0128] Figure 5 illustrates a flow chart of the method steps according to the second aspect of the disclosure.

[0129] The method comprising the step of S2 obtaining, by a camera sensor 5, a real-time image data of an installed component icl,ic2,ic3 associated with installation of the door 1, and the step of S5 outputting, by an output interface 20, visual data of the installed component icl,ic2,ic3 in real-time together with a visual augmented image of an augmented component acl,ac2,ac3 according to an expected placement to verify a correct installation of the installed component icl,ic2,ic3.

[0130] An advantage with this second aspect is that it is very intuitive for the user, e.g. the door installer, to see via the output interface 20 visual data showing the installed component icl,ic2,ic3 in real-time together with a visual augmented image of the augmented component acl,ac2,ac3, to understand if the installed component icl,ic2,ic3 is correctly installed.According to some embodiments the method further comprises the step of SI receiving, by an input interface 10, predetermined reference installation data associated with installation of the door 1, the step of S3 identifying the installed component icl,ic2,ic3 by applying image recognition on the obtained real-time image data and utilizing the predetermined reference installation data, and the step of S4 determining the placement of the identified installed component icl,ic2,ic3 in relation to an expected placement according to the predetermined reference installation data.

[0131] An advantage with this embodiment is that the predetermined reference installation data supports the identification of the installed component and the determination if the placement of the identified installed component in relation to an expected placement is correct.

[0132] According to some embodiments the method further comprises the step of S6, in that in determining that the placement of the installed component icl,ic2,ic3 in relation to the placement the augmented component acl,ac2,ac3 deviates, then outputting, by the by output interface 20, visual guidance vg indicative of instructions for correct placement of the installed component icl,ic2,ic3.

[0133] An advantage with this embodiment is that that the visual guidance vg, which can be seen by the user in real-time, is intuitive to the user, to e.g. adjust or re install a component so that the installation becomes correct.

[0134] According to some embodiments the method further comprises the step of outputting, by the by output interface 20, visual instruction data indicative of how to move the camera sensor 5 in order to obtain the real-time image data from different angles and / or distances in relation to the door 1.

[0135] An advantage with this embodiment is that the user can get information how to obtain sufficient real-time image data, from e.g. different angles, to have enough and reliable realtime image data of the component associated with installation of the door.

[0136] The third aspect of this disclosure shows a computer program product comprising a non-transitory computer readable medium, having thereon a computer program comprising program instructions, the computer program being loadable into a processing circuitry 102a, 102b and configured to cause execution of the method according to any of the second aspect when the computer program is run by the processing circuitry 102a, 102b.The person skilled in the art realizes that the present disclosure is not limited to the preferred embodiments described above. The person skilled in the art further realizes that modifications and variations are possible within the scope of the appended claims.

[0137] Additionally, variations to the disclosed embodiments can be understood and effected by the skilled person in practicing the claimed disclosure, from a study of the drawings, the disclosure, and the appended claims.

Claims

CLAIMS1. A door installation control system (100) for verifying a correct installation of a door (1), the system (100) comprises:• a camera sensor (5) configured to obtain real-time image data of a component associated with installation of the door (1);• an input interface (10) configured to receive predetermined reference installation data associated with the installation of the door (l);an output interface (20) configured to output visual data associated with installation of the door (l);a processing circuitry (102a, 102b) operatively connected to the camera sensor (5) , the input interface (10) and the output interface (20) configured to:receive predetermined reference installation data associated with installation of the door (1) wherein the predetermined reference installation data is generated by a learning model built on training data obtained over time by user verified door installations, wherein a door installer has verified a correct installation of a door;obtain a real-time image data of an installed component (icl, ic2, ic3) associated with installation of the door (1);identify the installed component (icl, ic2, ic3) by applying image recognition on the obtained real-time image data and utilize the predetermined reference installation data;determine the placement of the identified installed component (icl, ic2, ic3) in relation to an expected placement according to the predetermined reference installation data; andoutput visual data of the installed component (icl, ic2, ic3) in real-time together with a visual augmented image of an augmented component (acl, ac2, ac3) according to an expected placement to verify a correct installation of the installed component (icl, ic2, ic3).

2. The door installation control system (100) according to any of the preceding claims, wherein the processing circuitry (102a, 102b) is further configured to:in a determination that the placement of the installed component (icl, ic2, ic3) in relation to the placement the augmented component (acl, ac2, ac3) deviates:- output visual guidance (vg) indicative of instructions for correct placement of the installed component (icl, ic2, ic3).

3. The door installation control system (100) according to claim 2, wherein the visual guidance is indicative of at least any of an angle, a distance, a measurement, a position, a direction, an adjustment, a replacement component, a reinstallation instruction, an alternative placement and an alternative component.

4. The door installation control system (100) according to any of the claims 2-3, wherein the visual guidance is indicative of a level of deviation.

5. The door installation control system (100) according to claim 4, the deviation is determined by comparing the placement of the installed component (icl, ic2, ic3) in relation to the expected placement, as visualised with the augmented component (acl, ac2, ac3).

6. The door installation control system (100) according to any of the claims 2-5 wherein the predetermined reference installation data is received by obtaining user input via the input interface (10).

7. The door installation control system (100) according to any of the claims 2-6 wherein the predetermined reference installation data is received upon obtaining door identity data via the input interface (10) by at least any of:• receiving user input indicative of the door identity data via the input interface (10);• scanning, by the camera sensor (5), a code indicative of the door identity data;• obtaining image data of the door (1), by the camera sensor (5), and identifying the door using image recognition indicative of the door identity data; and• scanning by the camera sensor (5) a text indicative of the door identity data.

8. The door installation control system (100) according to any of the preceding claims, wherein the processing circuitry (102a, 102b) is further configured to:output visual instruction data indicative of how to move the camera sensor (5) in order to obtain the real-time image data from different angles and / or distances in relation to the door (1).

9. A method for verifying a correct installation of a door (1), the method comprising:(51) receiving, by an input interface (10), predetermined reference installation data associated with installation of the door (1) wherein the predetermined reference installation data is generated by a learning model built on training data obtained over time by user verified door installations, wherein a door installer has verified a correct installation of a door;(52) obtaining, by a camera sensor (5), a real-time image data of an installed component (icl, ic2, ic3) associated with installation of the door (1);(53) identifying the installed component (icl, ic2, ic3) by applying image recognition on the obtained real-time image data and utilizing the predetermined reference installation data;(54) determining the placement of the identified installed component (icl, ic2, ic3) in relation to an expected placement according to the predetermined reference installation data; and(55) outputting, by an output interface (20), visual data of the installed component (icl, ic2, ic3) in real-time together with a visual augmented image of an augmented component (acl, ac2, ac3) according to an expected placement to verify a correct installation of the installed component (icl, ic2, ic3).

10. The method according to claim9 further comprising:(S6) in a determining that the placement of the installed component (icl, ic2, ic3) in relation to the placement the augmented component (acl, ac2, ac3) deviates:- outputting, by the by output interface (20), visual guidance (vg) indicative of instructions for correct placement of the installed component (icl, ic2, ic3).

11. The method according to any of the claims 9-10 further comprising:outputting, by the by output interface (20), visual instruction data indicative of how to move the camera sensor (5) in order to obtain the real-time image data from different angles and / or distances in relation to the door (1).

12. A computer program product (500) comprising a non-transitory computer readable medium, having thereon a computer program comprising program instructions, the computer program being loadable into a processing circuitry (102a, 102b) and configured to cause execution of the method according to any of claims 9-11 when the computer program is run by the processing circuitry (102a, 102b).

Images