Method and system for providing information about a furniture part

Abstract

The invention relates to a method for providing information about a furniture part, comprising the following steps: - a user creating an image (5) of the furniture part by means of a camera; - transmitting the image (5) to an image recognition means which is self-learning and which is dedicatedly trained by means of images of furniture and furniture parts; - the image recognition means evaluating the image (5) and identifying the furniture part; - outputting a type designation of the furniture part. The invention also relates to a system composed of a mobile device (1) and a remote server, to which the mobile device (1) is connected for data communication, the mobile device comprising a camera and the server comprising an image recognition means. The system is characterized in that it is designed to carry out a method of this type.

Description

[0001] Method and arrangement for providing information about a furniture part

[0002] The invention relates to a method for providing information, particularly with regard to maintenance, assembly, and / or adjustments, for a furniture part. The invention further relates to an arrangement configured to carry out the method.

[0003] In addition to printed assembly or operating instructions included with a product, there are now a variety of ways to obtain information about a specific product via the Internet. These assembly or operating instructions can often also be accessed in digital form directly from the product manufacturer or from websites that provide a collection of such instructions. The manufacturers' own websites, online forums, or video collections often offer additional information on assembling or maintaining the product and how to make or change certain settings. Unlike in the past, the availability of information is less of a problem. Instead, it is becoming increasingly difficult to find the right information for a product and your current need from the wealth of information available.

[0004] This is especially true for furniture and information about furniture components, as for aesthetic reasons, there is a desire to arrange functional furniture components, such as furniture fittings, as discreetly or almost invisibly as possible in or on the furniture. One example is a drawer frame, which now often has adjustment options for the side and / or height position of a drawer front. Adjustment screws for adjusting the position of the front panel are usually hidden behind covers on the side frame or can be accessed from below with a tool, without any indication of this on the furniture itself.

[0005] Generally, a product is identified using a nameplate, which can also include machine-readable components such as barcodes or QR codes. Since these can negatively influence the design and appearance of the product, they are often not used on furniture parts or are only placed in positions that are visible before the furniture part is installed, but are as hidden as possible after the furniture part has been installed. By using image recognition software, it is now possible to identify products using product photos and an image comparison with images stored in a database. One such application is provided as a web application by Google LLC under the name "Google Lens". The results of a search using the "Google Lens" application are displayed in the form of a varying number of web links, via which more or less relevant further information can be retrieved.If specific information regarding furniture components, especially hidden furniture components and their maintenance, assembly and / or adjustments, is sought, a universally applicable tool such as “Google Lens” can only lead to the information sought randomly and not in a targeted manner.

[0006] It is therefore an object of the present invention to describe a method and a mobile device with which information regarding maintenance, assembly and / or possible adjustments and their implementation is provided for furniture parts, in particular fittings and also concealed fittings.

[0007] This problem is solved by a method and an arrangement having the respective features of the independent claim. Advantageous embodiments and further developments are the subject of the dependent claims.

[0008] A method according to the invention for providing information about a furniture part comprises the following steps: A user creates an image of the furniture part using a camera and transmits it to an image recognition device that is self-learning and specifically trained using images of furniture and furniture parts. The image is evaluated by the image recognition device, and the furniture part is identified. A type designation of the furniture part is then output.

[0009] The self-learning image recognition system, which has been specifically trained for furniture and furniture parts, enables reliable identification of furniture parts, even if they are built into furniture.

[0010] In an advantageous embodiment of the method, the

[0011] The image recognition system uses a neural network, particularly a convolutional neural network (CNN). It is also possible for the image recognition system to use a support vector machine, a random forest model, and / or a deep learning algorithm. These artificial intelligence (AI) methods are particularly well suited for the task of image recognition. These methods can be preceded by preparatory steps, e.g., for edge detection, brightness adjustment, and / or contrast adjustment.

[0012] In a further advantageous embodiment of the method, the following additional steps are carried out before the image is created: The user specifies the type of furniture in which the furniture part is already installed. The user is then provided with information as to which area of ​​the furniture should be represented in the image. In addition, a suitable recording perspective can also be suggested. This makes the image particularly well suited for analysis and shows features of the furniture part that are helpful for recognition, which increases the probability of a successful recognition. The information is preferably provided together with a generic image of the furniture, so that the information can be easily and intuitively understood and implemented.

[0013] In a further advantageous embodiment of the method, after the type designation has been output, further information about the furniture part is output, in particular regarding the technical data of the furniture part, maintenance, assembly and / or disassembly of the furniture part, and / or possible adjustments of the furniture part. This further information can be output in the form of text, images, and / or videos.

[0014] A mobile device with a camera can be used particularly advantageously to capture the image. The image is preferably transmitted via a network to a server on which the image recognition device is implemented. The mobile device can be, for example, a standard smartphone or a tablet computer. This allows the user to benefit simultaneously from the portability and availability of such a mobile device on the one hand and from the high computing power of the server on the other. The image recognition device provided on the server can be used by different users and accordingly receives a wide variety of images of furniture in which the requested furniture parts are installed. These images can then be advantageously used to further train the image recognition device.

[0015] An arrangement according to the invention is configured to carry out such a method and comprises a mobile device and a remote server to which the mobile device is connected via data communication. The mobile device has a camera, and the server has an image recognition device. This results in the advantages mentioned in connection with the method.

[0016] The invention is explained in more detail below using exemplary embodiments and figures. The figures show:

[0017] Fig. 1 is a flowchart of an embodiment of a method according to the invention;

[0018] Fig. 2 is a more detailed flow chart of a portion of the method of Figure 1;

[0019] Fig. 3 is a flowchart of a further embodiment of a method according to the invention;

[0020] Fig. 4-7 show various representations of a mobile device during the implementation of a first embodiment of a method according to the invention; and

[0021] Fig. 8-10 each show a representation of a mobile device during the implementation of the method according to the invention in an alternative embodiment.

[0022] Figures 1 and 2 show a first exemplary embodiment of a method according to the invention for providing information about a furniture part in the form of flow diagrams. Figure 3 shows a second exemplary embodiment of such a method according to the invention. Both exemplary embodiments are described below with reference to Figures 4 - 9, in which a mobile terminal, referred to below as mobile device 1, with its screen 2 is shown by way of example, while the methods according to the invention are carried out with this mobile device 1. In the exemplary embodiments shown, a smartphone is used as the mobile device 1. It should be noted that both the type of mobile device 1 and details of the display on its screen 2 are purely exemplary. A tablet computer, for example, can also be used as the mobile device 1.A suitable mobile device only requires access to a screen and a camera, although integrating the screen and camera into the mobile device is preferred for ease of use. VR (virtual reality) or AR (augmented reality) headsets could also be used as a mobile device, but the use of such headsets is not required, as a common and widely used smartphone or tablet computer already meets all the necessary requirements.

[0023] The method according to Figure 1 begins in a first step S1 with the starting of an application on a mobile terminal, for example the mobile device 1 according to Figures 4 - 9. In a next step S2, safety instructions are displayed on the mobile device 1, which particularly alert a user to possible dangers when working on the furniture parts.

[0024] The method according to the invention serves to provide information about a furniture part, in particular one that is already installed in a piece of furniture. In a next step S3, the user points the mobile device 1 with its camera at this furniture part or at an area of ​​the furniture in which the furniture part is installed, or at an area of ​​the furniture whose design is connected to the furniture part and therefore allows inferences to be made about the furniture part. Alternatively, it is also possible to point the camera exclusively at the furniture part if it is separate from the furniture.

[0025] In the following step S4, an image 5 of the furniture or furniture part is created using a camera of the mobile device 1. The image 5 is also referred to below as camera image 5.

[0026] To assist the user in capturing a meaningful camera image 5, one embodiment of the method according to the invention can provide the user with information on suitable recording positions and perspectives. To this end, the user is first asked to specify the type of furniture in which the furniture part is installed or is to be installed. This can be done, for example, using a predefined selection list or by entering free text.

[0027] Figure 4 shows the mobile device 1 with its screen 2 after a cabinet with drawers, for example used as a base cabinet in a fitted kitchen, has been selected as the furniture type.

[0028] The application started in step S1 then represents a generic image 3 of the furniture, here the drawer cabinet with a body 30 and a plurality of drawers 31, one of which, here the upper one as an example, is pulled out, so that a drawer side 32 and a drawer corner 33 (here the transition area between a drawer side 32 and a rear wall of the drawer 31) of the drawer 31 can be seen.

[0029] In this generic image 3, instructions 4 are displayed in the form of arrows, which indicate suitable, possible recording areas or recording directions for the camera recording 5 to be taken.

[0030] Figures 5a and 5b each show an example of a camera image 5 of the furniture taken in accordance with instructions 4. In Figure 5a, the camera was aimed at the drawer side 32, and in Figure 5b, at the drawer corner 33.

[0031] The information 4 ensures that the camera recording 5 has the highest possible information content with regard to the image section and the elements of the furniture shown as well as with regard to the perspective in order to be able to identify the relevant piece of furniture as accurately as possible.

[0032] In a next step S5, the camera recording 5 is transferred to a

[0033] The data is forwarded to an image recognition system, which identifies the furniture piece based on an image comparison. If the optional steps for providing the user with hints regarding a suitable shooting position and perspective have been used, the type of furniture specified by the user is known. In this case, this information can be used, simplifying recognition and increasing accuracy.

[0034] Preferably, the image recognition device is not implemented on the mobile device 1 itself, but is executed as a program on a server accessible from the mobile device 1 via a network. For this purpose, the mobile device 1 sends a corresponding request to the server, with which the camera image 5 or, if appropriate, several camera images 5 are transmitted to the image recognition device. After recognition has been completed, information with the precise designation of the furniture part is transmitted back to the mobile device 1 and can be displayed, as shown for the drawer in Figure 6 as an example.

[0035] The image recognition system operates in a self-learning manner, for example, using a neural network or other algorithms used in the field of AI (artificial intelligence), such as deep learning. The image recognition system is initially trained using a large number of images of the furniture parts to be recognized, installed in as many different pieces of furniture as possible, under different lighting conditions. The self-learning process can be continued even when the method is used by a user. For example, a situation may arise in which the result cannot be clearly recognized with sufficient probability. In this case, two or more possible results can be communicated to the user, who may be easily able to distinguish which result is the correct one based on additional information.The user then selects the correct result on mobile device 1, which forwards this information to the server and thus to the image recognition system. Using this information and the transmitted camera image 5, the image recognition system can then be further trained.

[0036] In addition to providing the recognition result in step S5 (see Figure 6), the user is given the opportunity to retrieve further information about this furniture part, for example, the recognized drawer type, as the process continues. This additional information is preferably provided in groups so that the user can find the information relevant to them as quickly as possible.

[0037] The various groups are presented to the user in step S6. The user can then select the group they are interested in. In the flowchart in Figure 1, various selection options, referred to as "functions" in the flowchart, are shown as examples in steps S7 to S11. In the example, the user can choose between the groups or functions "Assembly / Installation", "Adjustment Options", "Troubleshooting", "Disassembly / Replacement" and / or General ("FAQs"). Each of the selection options in steps S7 to S11 can contain further sub-selections, which will be explained in more detail below in connection with Figure 2.

[0038] Typically, after selection or subselection, the mobile device 1 issues instructions that are as intuitive and interactive as possible to perform the selected function, for example, assembly or adjustment. This can include both textual information displayed on the screen 2 of the mobile device 1, as well as graphically animated image sequences and / or videos.

[0039] A subsequent step S13 allows you to branch back to step S3 to retrieve information about another piece of furniture or to query other information about the same piece of furniture. If this is not necessary, the application on mobile device 1 can be terminated in a final step S14.

[0040] Figure 2 also shows in a flow chart the possibilities resulting from step S8 of Figure 1 in more detail.

[0041] After selecting the "Adjustment Options" function group, various options are displayed as action indicators 7 on the screen 2 of the mobile device 1, as shown by way of example in Figure 7. These relate to a height adjustment shown on the left, a side adjustment shown in the middle, and an inclination adjustment of the drawer shown on the right.

[0042] The user can select one of these action displays 7, whereupon the method continues with a step S20, S23, or S26. The additional information relevant to the respective action is displayed in steps S21 or S22 or S24 or S25 or S27 or S28, respectively, either as a video or as an image sequence, depending on the user's preference.

[0043] After displaying or playing back the corresponding video or image sequence, a subsequent step S29 allows you to select whether the retrieved video or image sequence should be repeated or whether you want to retrieve additional information from the group of setting options. If this is not the case, the program branches back to step S13 according to Figure 1.

[0044] The exemplary embodiments of the method in Figures 1 and 2 show the use of the method according to the invention directly by a user using a mobile device 1. In a further embodiment, it is conceivable to also use the method according to the invention within the service department of a furniture parts manufacturer or a furniture supplier. A possible application scenario is illustrated by way of example in the flowchart in Figure 3.

[0045] In a first step S30, a user, also called customer in the diagram, sends a photo or, if necessary, a video of a piece of furniture to a service department.

[0046] In a step S31, this starts a program, preferably on a desktop PC or laptop, which includes an image recognition device or has a connection to an image recognition device implemented on a server.

[0047] In a subsequent step S32, the photo or video sent in step S30 and received by the service department is loaded using the program started in step S31 and fed to the image recognition device.

[0048] In a step S33, this identifies the furniture component in a manner comparable to that described in connection with step S5 in the embodiment of Figure 1. This embodiment is explicitly referenced here. If the image recognition device is implemented on a server, it is also conceivable to query the same image recognition device directly from a user, as explained in Figures 1 and 2, and also to use it for a query from a service department.

[0049] The information output by the image recognition device is used in a step S34 to compile the information relevant to the customer about the furniture part and to forward it to him in a step S35.

[0050] If necessary, the process can be repeated with additional photos or videos by branching back to step S32 from the next step S36. If not necessary, the process can be terminated in step S37.

[0051] The flow diagrams in Figures 1 to 3 were previously explained using Figures 4 to 7 as examples, which relate to drawers as furniture parts.

[0052] Figures 8 to 10 show, analogously to Figure 4, the screen 2 of the mobile device 1 for other types of furniture parts, specifically for a hinge 35 in Figure 8, a sliding door fitting 37 in Figure 9 and a translation-rotation fitting 44 in Figure 10.

[0053] The hinge 35 shown in Figure 8 serves to pivotally mount a door 34 on a body 30. The sliding door fitting 37 shown in Figure 9 serves to guide a sliding door 36 on a body 30.

[0054] Here, too, instructions 4 are provided regarding a suitable area and perspective when creating the photographic image 5 of the furniture part or piece of furniture. Particularly in the case of large fittings or fittings arranged and interacting in a distributed manner on the furniture, enlarged sections 8 can be used to simplify localization of the corresponding fitting part while still being able to depict necessary details. Figure 9 uses such enlarged sections 8 to depict the sliding door fitting 37 on the furniture in more detail, along with a separate door-side section 38 of the sliding door fitting 37 with the sliding door 36 removed.

[0055] In the example of Figure 10, a cabinet is shown as a piece of furniture with a body 30, which has a fixed cabinet part 39 and a rotatable cabinet part 40. The rotatable cabinet part 40 is cuboid-shaped with two closed sides 41 and two open sides that allow access to an interior space 42. For example, shelves 43 are arranged in the interior space 42 of the furniture shown in order to structure the interior space 42.

[0056] The rotatable cabinet part 40 is mounted on the furniture body 30 by means of the previously mentioned translational rotation fitting 44. The translational rotation fitting 44 enables a combined pivoting and sliding movement of the rotatable cabinet part 40, by means of which the rotatable cabinet part 40 can be moved between two rotational end positions, which differ by a rotation angle of 180° around a vertical axis. In both rotational end positions, the rotatable cabinet part 40 fits essentially seamlessly into the outline of the furniture body 30. In order to be able to move the rotatable cabinet part 40 between the two rotational end positions, it is extended out of the furniture body 30 by the translational rotation fitting 44 until the vertical edges of the rotatable furniture part 40 do not impede the rotational movement.

[0057] In the right-hand part of the screen 2, the furniture is shown in an intermediate position during the transition between the rotational end positions. Here, too, indications 4 are provided indicating a suitable area and a suitable viewing direction for creating a photographic image 5 of the furniture or furniture part, in this case to indicate the area of ​​the furniture in which the translational-rotational fitting 44 can be easily identified using the method according to the invention.

[0058] Reference symbol

[0059] 1 mobile device

[0060] 2 screens

[0061] 3 generic illustration of a piece of furniture

[0062] 4 Note (arrow)

[0063] 5 photographic images of the furniture

[0064] 6 Result display

[0065] 7 Action display

[0066] 8 Detail enlargement

[0067] 30 Corpus

[0068] 31 drawers

[0069] 32 Drawer side

[0070] 33 Drawer corner

[0071] 34 Door

[0072] 35 Hinge

[0073] 36 sliding door

[0074] 37 Sliding fitting

[0075] 38 door-side part of the sliding fitting

[0076] 39 fixed cabinet part

[0077] 40 rotating cabinet part

[0078] 41 pages

[0079] 42 Interior

[0080] 43 shelf

[0081] 44 translational rotation fitting

[0082] S1 - S14 Process step

[0083] S20 - S29 Process step

[0084] S30 - S37 Process step

Claims

Claims 1 . A method for providing information about a furniture part, comprising the following steps: - Creating an image (5) of the furniture part using a camera by a user; - transmitting the image (5) to an image recognition device which is self-learning and is specifically trained using images of furniture and furniture parts; - evaluating the image (5) and identifying the furniture part by the image recognition device; - Output of a type designation of the furniture part.

2. The method according to claim 1, wherein the image recognition device comprises a neural network, in particular a convolved neural network.

3. The method according to claim 1, wherein the image recognition device uses a support vector machine, a random forest model and / or a deep learning algorithm.

4. Method according to one of claims 1 to 3, wherein the furniture part is incorporated into a piece of furniture.

5. The method according to claim 4, wherein the following steps are performed before creating the image: - Specifying a furniture type of the furniture by a user; and - Providing instructions (4) to the user as to which area of the furniture should be represented by the illustration (5).

6. Method according to claim 5, wherein the instructions (4) are output together with a generic image (3) of the furniture.

7. Method according to one of claims 1 to 6, wherein after the type designation has been output, further information on the furniture part is output.

8. Method according to claim 7, wherein the further information - technical data of the furniture part, - maintenance of the furniture part, - assembly and / or disassembly of the furniture part and / or - relate to settings of the furniture part.

9. Method according to claim 8, wherein the further information in Output in the form of text, images and / or videos.

10. Method according to one of claims 1 to 9, wherein the user uses a mobile device (1) with a camera for recording the image (5) and wherein the image (5) is transmitted via a network to a server on which the image recognition device is implemented. 11 . Arrangement comprising a mobile device (1) and a remote server to which the mobile device (1) is connected for data purposes, wherein the mobile device has a camera and the server has a Image recognition device, characterized in that the arrangement is designed to carry out a method according to claim 10.

Images