Method, apparatus, and device for generating three-dimensional house types

Abstract

An embodiment of the present application provides a method, device, and equipment for generating a three-dimensional room type. The method includes: obtaining at least two images of different perspectives of a room; generating three-dimensional layout diagrams respectively corresponding to the at least two images; determining splicing positions corresponding to the three-dimensional layout diagrams based on the at least two images; and performing splicing processing on the three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional room type corresponding to the room. The technical solution provided by the present application effectively solves the problems of high cost, inconvenient operation, and high shooting skill requirements when using a panoramic camera to obtain a panoramic view by obtaining ordinary images and outputting a three-dimensional room type of the photographed room. Furthermore, the applicable application scope of the method is expanded, greatly facilitating operations such as decoration design for a room, and is conducive to promoting the rapid development of related applications.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to a method, apparatus, and device for generating a three-dimensional house type. Background Art

[0002] In the field of intelligent home decoration technologies, in order to implement room rendering operations, generally the following steps are included: The user inputs a house type drawing to be rendered, and then selects different decoration styles, so that room rendering drawings with different decoration styles can be generated. In the prior art, in order to ensure the effect and accuracy of the room rendering drawing, the house type Figure 1 to be rendered input by the user is generally a panoramic image obtained by indoor collection using an expensive and non-portable panoramic camera.

[0003] However, for users, obtaining a panoramic image by using a specific panoramic camera for image collection is not only expensive, inconvenient to operate, but also requires relatively high shooting skills, thus limiting the application scope of the above technical solution and being unfavorable for the rapid development of the number of users. Summary of the Invention

[0004] Embodiments of this application provide a method, apparatus, and device for generating a three-dimensional house type to solve the problems that when using a panoramic camera for image collection to obtain a panoramic image, it is not only expensive, inconvenient to operate, but also requires relatively high shooting skills.

[0005] In a first aspect, embodiments of this application provide a method for generating a three-dimensional house type, including:

[0006] Obtain at least two images of different perspectives of a room;

[0007] Generate three-dimensional layout diagrams respectively corresponding to the at least two images;

[0008] Based on the at least two images, determine splicing positions corresponding to the three-dimensional layout diagrams;

[0009] Perform splicing processing on the three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional house type corresponding to the room.

[0010] In a second aspect, embodiments of this application provide a device for generating a three-dimensional house type, including:

[0011] A first acquisition module, configured to obtain at least two images of different perspectives of a room;

[0012] A first generation module, configured to generate three-dimensional layout diagrams respectively corresponding to the at least two images;

[0013] A first determination module, configured to determine splicing positions corresponding to the three-dimensional layout diagrams based on the at least two images;

[0014] A first processing module, configured to perform splicing processing on the three-dimensional layout diagram according to the splicing position to generate a three-dimensional house type corresponding to the room.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor; wherein, the memory is used to store one or more computer instructions, and when the one or more computer instructions are executed by the processor, the method for generating a three-dimensional house type shown in the first aspect above is implemented.

[0016] In a fourth aspect, an embodiment of the present invention provides a computer storage medium for storing a computer program, and when the computer program is executed by a computer, the method for generating a three-dimensional house type shown in the first aspect above is implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a method for generating a three-dimensional house type, including:

[0018] Obtaining at least two images of different perspectives of a room;

[0019] Generating two-dimensional layout diagrams respectively corresponding to the at least two images;

[0020] Determining camera parameters corresponding to the at least two images;

[0021] Generating a three-dimensional house type corresponding to the at least two images based on the at least two images, the two-dimensional layout diagrams, and the camera parameters.

[0022] In a sixth aspect, an embodiment of the present application provides a device for generating a three-dimensional house type, including:

[0023] A second obtaining module, configured to obtain at least two images of different perspectives of a room;

[0024] A second generating module, configured to generate two-dimensional layout diagrams respectively corresponding to the at least two images;

[0025] A second determining module, configured to determine camera parameters corresponding to the at least two images;

[0026] A second processing module, configured to generate a three-dimensional house type corresponding to the at least two images based on the at least two images, the two-dimensional layout diagrams, and the camera parameters.

[0027] In a seventh aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor; wherein, the memory is used to store one or more computer instructions, and when the one or more computer instructions are executed by the processor, the method for generating a three-dimensional house type shown in the fifth aspect above is implemented.

[0028] In an eighth aspect, an embodiment of the present invention provides a computer storage medium for storing a computer program, which, when executed by a computer, implements the method for generating a three-dimensional house type shown in the fifth aspect above.

[0029] In a ninth aspect, an embodiment of the present invention provides a method for generating a three-dimensional house type, including:

[0030] Obtaining at least two three-dimensional layout diagrams of different perspectives of a room;

[0031] Determining splicing positions corresponding to the at least two three-dimensional layout diagrams;

[0032] Performing splicing processing on the at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional house type corresponding to the room.

[0033] In a tenth aspect, an embodiment of the present invention provides a method for generating a three-dimensional house type, including:

[0034] A third acquisition module for obtaining at least two three-dimensional layout diagrams of different perspectives of a room;

[0035] A third determination module for determining splicing positions corresponding to the at least two three-dimensional layout diagrams;

[0036] A third processing module for performing splicing processing on the at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional house type corresponding to the room.

[0037] In an eleventh aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor; wherein, the memory is used for storing one or more computer instructions, and when the one or more computer instructions are executed by the processor, the method for generating a three-dimensional house type shown in the ninth aspect above is implemented.

[0038] In a twelfth aspect, an embodiment of the present invention provides a computer storage medium for storing a computer program, which, when executed by a computer, implements the method for generating a three-dimensional house type shown in the ninth aspect above.

[0039] The method, apparatus, and device for generating a three-dimensional room type provided by the embodiments of the present application obtain at least two images of different perspectives of a room, generate three-dimensional layout diagrams respectively corresponding to the at least two images, determine the splicing positions corresponding to the three-dimensional layout diagrams based on the at least two images, and then perform splicing processing on the three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional room type corresponding to the room. It effectively realizes that an ordinary image acquisition device can be used to perform image acquisition operations on the room, and the obtained images can be used to output the three-dimensional room type of the photographed room. The three-dimensional room type can include information such as the positions of doors and windows, thus effectively solving the problems existing in obtaining a panoramic image by using a panoramic camera, such as high price, inconvenient operation, and high shooting skill requirements. Moreover, it effectively expands the applicable application range of this method, greatly facilitating operations such as decoration design for a room and contributing to the rapid development of related applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 Schematic diagram of a method for generating a three-dimensional room type provided by an embodiment of the present application;

[0042] Figure 2 Flow chart of a method for generating a three-dimensional room type provided by an embodiment of the present application;

[0043] Figure 3 Schematic diagram of the shooting angles of at least two images provided by an embodiment of the present application;

[0044] Figure 4 Flow chart of generating at least two three-dimensional layout diagrams corresponding to the at least two images provided by an embodiment of the present application;

[0045] Figure 5 Flow chart of determining the splicing positions corresponding to the at least two three-dimensional layout diagrams based on the at least two images provided by an embodiment of the present application;

[0046] Figure 6 Flow chart of determining the splicing positions corresponding to the at least two three-dimensional layout diagrams based on the two-dimensional wall images corresponding to all wall information and the at least two images provided by an embodiment of the present application;

[0047] Figure 7A schematic flowchart for determining the splicing position corresponding to two adjacent three-dimensional layout diagrams based on the image similarity provided by the embodiments of the present application;

[0048] Figure 8 A schematic flowchart for another method of generating a three-dimensional house type provided by the embodiments of the present application;

[0049] Figure 9 A schematic flowchart for splicing the at least two three-dimensional layout diagrams according to the splicing position to generate a three-dimensional house type corresponding to a room provided by the embodiments of the present application;

[0050] Figure 10 A schematic flowchart for optimizing the spliced house type data to generate the three-dimensional house type provided by the embodiments of the present application;

[0051] Figure 11 A schematic flowchart for yet another method of generating a three-dimensional house type provided by the embodiments of the present application;

[0052] Figure 12 A schematic flowchart for a method of generating a three-dimensional house type provided by the application embodiments of the present application;

[0053] Figure 13 A schematic flowchart for obtaining camera parameters provided by the application embodiments of the present application;

[0054] Figure 14 A schematic flowchart for determining the splicing position provided by the application embodiments of the present application;

[0055] Figure 15 A schematic diagram after the splicing operation provided by the application embodiments of the present application;

[0056] Figure 16 A schematic flowchart for a method of generating a three-dimensional house type provided by the embodiments of the present application;

[0057] Figure 17 A schematic flowchart for another method of generating a three-dimensional house type provided by the embodiments of the present application;

[0058] Figure 18 A schematic structural diagram of a device for generating a three-dimensional house type provided by the embodiments of the present application;

[0059] Figure 19 For Figure 18 A schematic structural diagram of the electronic device corresponding to the device for generating the three-dimensional house type shown;

[0060] Figure 20 A schematic structural diagram of a device for generating a three-dimensional house type provided by the embodiments of the present application;

[0061] Figure 21 The Figure 20 structural schematic diagram of an electronic device corresponding to the three-dimensional house type generation device shown in

[0062] Figure 22 structural schematic diagram of a three-dimensional house type generation device provided by an embodiment of the present application;

[0063] Figure 23 The Figure 22 structural schematic diagram of an electronic device corresponding to the three-dimensional house type generation device shown in Detailed implementation manners

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0065] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "the", and "said" used in the embodiments of the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. "Plural" generally includes at least two, but does not exclude the case of including at least one.

[0066] It should be understood that the term " / and / " used herein is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after.

[0067] Depending on the context, the words "if" and "when" as used herein can be interpreted as "when...", "when...", "in response to determining", or "in response to detecting". Similarly, depending on the context, the phrases "if determined" or "if detected (stated condition or event)" can be interpreted as "when determined", "in response to determining", "when detected (stated condition or event)", or "in response to detecting (stated condition or event)".

[0068] It should also be noted that the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a commodity or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or elements inherent to such commodity or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the commodity or system including said element.

[0069] In addition, the step timings in the following method embodiments are only examples and not strictly limited.

[0070] To facilitate the understanding of the technical solutions provided in the embodiments of the present application by those skilled in the art, the related technologies are described below:

[0071] In the field of intelligent home decoration technology, in order to be able to perform room rendering operations, generally the following steps are included: the user inputs the floor plan to be rendered, and then selects different decoration styles, so that room rendering diagrams with different decoration styles can be generated. The currently more common way to obtain the floor plan is to adopt the indoor panoramic acquisition scheme of a panoramic camera. Specifically, the implementation steps for obtaining the indoor layout diagram (layout) based on the panoramic camera include:

[0072] (1) Use a panoramic camera to collect images of the room to obtain a 360-degree panoramic image of the room.

[0073] (2) Use a convolutional neural network (Convolutional Neural Networks, abbreviated as CNN) to identify the boundaries of the floor-wall, ceiling-wall, and wall-wall in the panoramic image.

[0074] (3) Based on the boundary line information, camera internal parameters, and the height of the camera from the ground during shooting, determine the floor plan data of the indoor floor plan.

[0075] However, the above technical solutions have the following defects: panoramic images are not easy to obtain, and a professional panoramic camera is required for shooting, which is not only expensive, inconvenient to operate, but also requires high shooting skills, and ordinary users cannot operate it, thus limiting the application scope of the above technical solutions and being unfavorable for the rapid development of the number of users.

[0076] To solve the above technical problems, this embodiment provides a method, device, and equipment for generating a three-dimensional floor plan. The execution subject of this method can be a device for generating a three-dimensional floor plan, and the generating device can be communicatively connected to an image acquisition device.

[0077] Among them, the image acquisition device can be any computing device with certain image acquisition functions and computing capabilities. Specifically, in implementation, the image acquisition device can be a camera, a video camera, an intelligent terminal with a shooting function (such as a mobile phone, a tablet computer), and so on. In addition, the basic structure of the image acquisition device may include: at least one processor. The number of processors depends on the configuration and type of the image acquisition device. The image acquisition device may also include a memory, which can be volatile, such as RAM, or non-volatile, such as read-only memory (ROM), flash memory, etc., or may also include both types at the same time. The memory usually stores an operating system (OS), one or more application programs, and may also store program data, etc. In addition to the processing unit and the memory, the image acquisition device also includes some basic configurations, such as a network card chip, an IO bus, a display component, and some peripheral devices. Optionally, some peripheral devices may include, for example, a keyboard, a mouse, a stylus, a printer, etc. Other peripheral devices are well known in the art and will not be elaborated here. Optionally, the image acquisition device can be a PC (personal computer) terminal, a handheld terminal (such as a smart phone, a tablet computer), etc.

[0078] The generation device refers to a device that can provide computing and processing services in a network virtual environment, usually referring to a device that uses the network for information planning and data processing. In physical implementation, the generation device can be any device that can provide computing services, respond to service requests, and perform processing, such as: it can be a cluster server, a conventional server, a cloud server, a cloud host, a virtual center, etc. The composition of the generation device mainly includes a processor, a hard disk, a memory, a system bus, etc., which is similar to a general computer architecture.

[0079] In the above embodiment, the image acquisition device can be network-connected to the generation device, and this network connection can be a wireless or wired network connection. If the image acquisition device and the generation device are communicatively connected, the network mode of the mobile network can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), WiMax, etc.

[0080] In the embodiment of the present application, an image acquisition device is used to perform image acquisition operations on a room, so as to obtain multiple images of different perspectives of the room. The multiple images refer to images with a quantity greater than or equal to two, and the shooting perspective of each image in the multiple images can be less than or equal to a set value, that is, the images obtained by the image acquisition device are not panoramic images. After obtaining the multiple images, the images can be uploaded to a three-dimensional house type generation device, so that the generation device can analyze and process the uploaded images.

[0081] The three-dimensional house type generation device is used to receive multiple images uploaded by a client, that is, to enable the generation device to obtain multiple non-panoramic format images. Then, as Figure 1 shown, the generation device can analyze and process the multiple images to generate a two-dimensional layout diagram corresponding to each image. The generated two-dimensional layout diagram may include wall information in the room; then, in combination with each image and the camera parameters corresponding to the image (the camera parameters may include at least one of the following: camera internal parameters, camera external parameters), analyze and process the two-dimensional layout diagram to generate a three-dimensional layout diagram corresponding to each image; then, based on at least two images, analyze and process the generated multiple three-dimensional layout diagrams to determine the splicing positions corresponding to the multiple three-dimensional layout diagrams. It can be understood that multiple images can generate multiple three-dimensional layout diagrams, and a splicing position can be determined between any two three-dimensional layout diagrams. That is, when the number of three-dimensional layout diagrams is N, the determined splicing positions are also N. After obtaining the splicing positions, all the obtained three-dimensional layout diagrams can be spliced according to the splicing positions, so as to generate a three-dimensional house type corresponding to the room.

[0082] The technical solution provided in this embodiment effectively realizes that an ordinary image acquisition device can be used to perform image acquisition operations on a room, and the three-dimensional house type of the photographed room can be output through the obtained images. The three-dimensional house type may include information such as the positions of doors and windows. Thus, it effectively solves the problems existing when using a panoramic camera for image acquisition to obtain a panoramic image, including not only high price, inconvenient operation, but also high shooting skill requirements. Moreover, it effectively expands the applicable application range of this method, greatly facilitating operations such as decoration design for a room, and is conducive to promoting the rapid development of related applications.

[0083] The following specifically describes the three-dimensional house type generation method, device, and equipment provided in each embodiment of the present application through an exemplary application scenario.

[0084] Figure 2 Schematic flowchart of a method for generating a three-dimensional house type provided by an embodiment of the present application; refer to the appendix Figure 2 As shown, an embodiment of the present application provides a method for generating a three-dimensional house type. The execution subject of this method can be a three-dimensional house type generation device. It can be understood that this three-dimensional house type generation device can be implemented as software, or a combination of software and hardware. Specifically, the method for generating a three-dimensional house type may include:

[0085] Step S201: Obtain at least two images of different perspectives of the room.

[0086] Step S202: Generate three-dimensional layout diagrams respectively corresponding to the at least two images.

[0087] Step S203: Based on the at least two images, determine the splicing positions corresponding to the three-dimensional layout diagrams.

[0088] Step S204: Perform splicing processing on the three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional house type corresponding to the room.

[0089] The following is a detailed description of each of the above steps:

[0090] Step S201: Obtain at least two images of different perspectives of the room.

[0091] Among them, when there is a need to generate a three-dimensional house type for a room, an image acquisition device can be used to perform an image shooting operation on the room, so as to obtain at least two images of different perspectives of the room. After the image acquisition device obtains at least two images, the at least two images can be transmitted to the three-dimensional house type generation device, so that the generation device can stably obtain at least two images. Of course, those skilled in the art can also use other methods to obtain at least two images of different perspectives of the room. For example, at least two images of a room can be stored in a preset area, and the at least two images of different perspectives of the room can be obtained by accessing the preset area.

[0092] In addition, the shooting angle of any one of the at least two obtained images is less than or equal to a set value, where the set value is a shooting angle limit value pre-configured to define that the image is not a panoramic image. In different application scenarios, the size or value range of the set value may be different. For example, in some application scenarios, the set value may be 200° and so on; or, in some other application scenarios, the set value may be 220° and so on. When the shooting angle of the image is less than or equal to the set value, it can be determined that the image is not a panoramic image. When the shooting angle of the image is greater than the set value, it can be determined that the image is a panoramic image. Specifically, in this embodiment, the at least two images obtained by the image acquisition device (instead of the panoramic camera for generating panoramic images) are not panoramic images, thus effectively solving the problem of relatively high image acquisition cost in the prior art when a panoramic camera is required to obtain panoramic images.

[0093] In some instances, different shooting angles of the at least two images indicate that there is an overlapping area between the shooting areas corresponding to the shooting angles of the at least two images. It should be noted that when performing an image acquisition operation for a room, the size of the shooting angle of the acquired images is related to the number of images. When the shooting angle of the image is larger, the number of obtained images can be smaller; when the shooting angle of the image is smaller, the number of obtained images can be larger.

[0094] Specifically, in this embodiment, the number of the at least two images with different perspectives of the obtained room is not limited. Those skilled in the art can make different configurations according to different room types. For example, when shooting images of a room with a symmetric rectangular house type, when the shooting perspectives corresponding to two images can cover the entire room, only two images can be obtained. At this time, the shooting perspectives corresponding to the two images are different, and the sum of the shooting perspective ranges corresponding to the two images is greater than 360° to generate a complete three-dimensional house type corresponding to the room; when the shooting perspectives corresponding to the two images cannot cover the entire room, three images or four images can be obtained. The shooting perspectives corresponding to the three images or four images are different, and the sum of the shooting perspective ranges corresponding to the three images or four images is greater than 360° to generate a complete three-dimensional house type corresponding to the room.

[0095] For a square room, when three images can cover the entire room, three images can be obtained. The shooting perspectives corresponding to the three images are different, and the sum of the shooting perspective ranges corresponding to the three images is greater than 360°; when four images can cover the entire room, four images can be obtained. The shooting perspectives corresponding to the four images are different, and the sum of the shooting perspective ranges corresponding to the four images is greater than 360°. In some instances, such as Figure 3As shown, when the image acquisition device is a mobile phone, considering that the field of view angle of the main camera of most users' mobile phones is about 60 degrees, therefore, when obtaining four images for a room, the shooting angles of the four images can be from the four corners of the wall and shoot diagonally across, that is, the four shooting perspectives can correspond to the four room apexes. At this time, the four images can correspond to different shooting perspectives.

[0096] Step S202: Generate three-dimensional layout diagrams respectively corresponding to at least two images.

[0097] After obtaining at least two images, the at least two images can be analyzed and processed, so as to generate three-dimensional layout diagrams respectively corresponding to the at least two images. Specifically, the implementation manner of generating the three-dimensional layout diagram in this embodiment is not limited, and those skilled in the art can set it according to specific application scenarios and application requirements. For example: a machine learning model for analyzing and processing at least two images is pre-trained, and the machine learning model is used to analyze and process the at least two images, so as to generate three-dimensional layout diagrams respectively corresponding to the at least two images. The three-dimensional layout diagrams can include: texture information, three-dimensional wall information, the spatial position relationship between each wall, and so on.

[0098] Of course, those skilled in the art can also adopt other methods to generate three-dimensional layout diagrams respectively corresponding to at least two images, as long as the accuracy and reliability of obtaining the three-dimensional layout diagrams can be ensured.

[0099] Step S203: Based on at least two images, determine the splicing positions corresponding to the three-dimensional layout diagrams.

[0100] After obtaining the three-dimensional layout diagrams respectively corresponding to at least two images, since the three-dimensional layout diagrams are respectively corresponding to at least two images, and the shooting perspectives corresponding to the at least two images are different, therefore, the number of three-dimensional layout diagrams is at least two, and the at least two three-dimensional layout diagrams correspond to different regions of the room. In order to generate a complete three-dimensional room type corresponding to the room, after obtaining at least two images, the three-dimensional layout diagrams can be analyzed and processed based on the at least two images to determine the splicing positions corresponding to the three-dimensional layout diagrams.

[0101] In addition, the specific implementation method for determining the splicing position corresponding to the three-dimensional layout diagram in this embodiment is not limited, and those skilled in the art can set it according to specific application requirements. For example, the adjacent relationship between any two images is determined based on at least two images. When the at least two images are four images, namely image a, image b, image c, and image d, there is an adjacent relationship between image a and image b, an adjacent relationship between image a and image d, and no adjacent relationship between image a and image c. After obtaining the adjacent relationship between any two images, the splicing position corresponding to the three-dimensional layout diagram can be determined based on the adjacent relationship between any two images described above.

[0102] Step S204: Perform splicing processing on the three-dimensional layout diagram according to the splicing position to generate a three-dimensional house type corresponding to the room.

[0103] After obtaining the splicing position, splicing processing can be performed on the three-dimensional layout diagrams corresponding to at least two images according to the splicing position, so that a three-dimensional house type corresponding to the room can be generated. This three-dimensional house type is the three-dimensional space layout diagram corresponding to a room. The three-dimensional space layout diagram can include information such as the wall surface information, door and window positions, position information, and size information of the room. Through the three-dimensional space layout diagram, the layout trend of the room can be clearly and intuitively seen.

[0104] After generating the three-dimensional house type corresponding to the room, decoration processing, rendering processing, or design processing can be performed on the room based on the three-dimensional house type. Specifically, the decoration parameters can be adjusted according to different application requirements and design requirements. When the decoration parameters are different, room decoration models corresponding to different decoration parameters can be generated.

[0105] The method for generating a three-dimensional house type provided in this embodiment obtains at least two images from different perspectives of a room, generates three-dimensional layout diagrams corresponding to the at least two images respectively, determines the splicing position corresponding to the three-dimensional layout diagram based on the at least two images, and then performs splicing processing on the three-dimensional layout diagram according to the splicing position to generate a three-dimensional house type corresponding to the room. It effectively realizes that image acquisition operations can be performed on a room through an ordinary image acquisition device, and the three-dimensional house type of the photographed room can be output through the obtained images. The three-dimensional house type can include information such as door and window positions, thus effectively solving the problems that exist when using a panoramic camera for image acquisition to obtain a panoramic view, including high cost, inconvenient operation, and high shooting skill requirements. Moreover, it effectively expands the applicable application range of this method, greatly facilitating operations such as decoration design for a room, and is conducive to promoting the rapid development of related applications.

[0106] Figure 4Schematic flowchart of generating at least two 3D layout diagrams corresponding to at least two images provided by an embodiment of the present application; on the basis of the above embodiment, refer to the attached Figure 4 As shown in the figure, this embodiment provides an implementation manner for generating 3D layout diagrams respectively corresponding to at least two images. Specifically, generating 3D layout diagrams respectively corresponding to at least two images in this embodiment may include:

[0107] Step S401: Generate 2D layout diagrams respectively corresponding to at least two images.

[0108] Among them, the 2D layout diagram refers to a graph used to divide the walls in each room in the image, which can reflect the positional relationship between each wall in the image. After obtaining at least two images, at least two images can be analyzed and processed to generate 2D layout diagrams respectively corresponding to at least two images. The generated 2D layout diagrams may include wall information in the room. When generating 2D layout diagrams respectively corresponding to at least two images, the following steps may be included: Analyze and process at least two images through a deep learning network, so as to obtain 2D layout diagrams respectively corresponding to at least two images.

[0109] Specifically, when analyzing and processing at least two images through a deep learning network, 2D layout diagrams respectively corresponding to at least two images can be generated by detecting the edges of the corners; or, 2D layout diagrams respectively corresponding to at least two images can also be generated by detecting the key points of the corners; or, the wall area can also be directly segmented, so as to generate 2D layout diagrams respectively corresponding to at least two images.

[0110] Of course, those skilled in the art can also adopt other methods to generate 2D layout diagrams respectively corresponding to at least two images, as long as the quality and efficiency of generating the 2D layout diagrams can be ensured, which will not be elaborated here.

[0111] Step S402: Determine camera parameters corresponding to at least two images.

[0112] After obtaining at least two images, at least two images can be analyzed and processed to determine camera parameters corresponding to at least two images. Specifically, the camera parameters may include at least one of the following: camera internal parameters, camera external parameters; at this time, determining camera parameters corresponding to at least two images may include: calculating vanishing point information corresponding to at least two images; according to the vanishing point information, calculating camera internal parameters corresponding to at least two images; according to the vanishing point information and camera internal parameters, determining camera external parameters corresponding to at least two images.

[0113] Among them, the vanishing point, also known as the disappearing point, is used to describe infinity in the physical world. Two lines that are parallel to each other in the physical world can converge and intersect at a point during the two-dimensional projection of the camera, and this point is the vanishing point or disappearing point. Specifically, calculating the vanishing point information corresponding to at least two images may include: obtaining the line segment information included in at least two images, and then calculating the vanishing point corresponding to at least two images based on the line segment information. After obtaining the vanishing point information, the camera internal parameters corresponding to at least two images can be calculated based on the vanishing point characteristics corresponding to the vanishing point information.

[0114] After obtaining the vanishing point information and the camera internal parameters, the vanishing point information and the camera internal parameters can be analyzed and processed to determine the camera external parameters corresponding to at least two images. The camera external parameters may include: the rotation matrix and / or the translation matrix between the world coordinate system and the camera coordinate system.

[0115] Step S403: Generate at least two three-dimensional layout diagrams based on at least two images, the two-dimensional layout diagram, and the camera parameters.

[0116] Among them, after obtaining at least two images, the two-dimensional layout diagram, and the camera parameters, at least two images, the two-dimensional layout diagram, and the camera parameters can be analyzed and processed, so that three-dimensional layout diagrams corresponding to at least two images can be generated respectively. In some examples, generating three-dimensional layout diagrams corresponding to at least two images based on at least two images, the two-dimensional layout diagram, and the camera parameters may include: obtaining (setting) the height information between the image shooting position corresponding to at least two images and the ground; determining the spatial constraint relationship corresponding to the pixel points in at least two images based on the height information, at least two images, and the camera parameters; performing a three-dimensional reconstruction operation on the pixel points in at least two two-dimensional layout diagrams based on the spatial constraint relationship to generate a three-dimensional layout diagram.

[0117] Specifically, when using the image acquisition device to perform image acquisition operations, there will be a corresponding height information between the image shooting position corresponding to the image acquisition device and the ground. It can be understood that the above height information can be preset or input by the user; for different application scenarios or application requirements, different height information or the same height information can be set or input. For example, the height information can be 1.5m, 1.4m, etc.

[0118] After setting the height information existing between the image shooting position and the ground, the height information existing between the image shooting position and the ground can be obtained, and then the height information, at least two images, and the camera parameters can be analyzed and processed to determine the spatial constraint relationship of the pixel points in at least two images.

[0119] In some examples, the spatial constraint relationships of pixel points in at least two images may include at least one of the following: for each point on the intersection line of the wall surface and the ground and the floor, the coordinate Z in the world coordinate system (i.e., the height information H between the image capture position and the ground) is known, then the mapping from the point in the pixel coordinate system to the world coordinate system can be realized. For the points on the wall surface, the depth information from the camera is the same as that of the points on the intersection line of the wall surface and the ground, so that each point on the wall surface can be three-dimensionally reconstructed. For the points on the roof, through the three-dimensional coordinates of the points on the intersection line of each wall surface and the roof, the height of the roof from the ground can be obtained, and then each point on the roof can also be three-dimensionally reconstructed.

[0120] After obtaining the spatial constraint relationships, the three-dimensional reconstruction operation can be performed on the pixel points in at least two two-dimensional layout diagrams based on the spatial constraint relationships, so that a three-dimensional layout diagram can be generated, effectively ensuring the accuracy and reliability of obtaining the three-dimensional layout diagram.

[0121] Figure 5 Schematic flow diagram for determining the splicing position corresponding to the three-dimensional layout diagram based on at least two images provided by the embodiments of the present application; on the basis of the above embodiments, continue to refer to the attached Figure 5 As shown, this embodiment provides an implementation manner for determining the splicing position corresponding to the three-dimensional layout diagram. Specifically, determining the splicing position corresponding to the three-dimensional layout diagram based on at least two images in this embodiment includes:

[0122] Step S501: Extract the wall surface information included in each three-dimensional layout diagram.

[0123] Step S502: Generate a two-dimensional wall surface image corresponding to the wall surface information.

[0124] Step S503: Determine the splicing position corresponding to the three-dimensional layout diagram based on the two-dimensional wall surface images corresponding to all the wall surface information and at least two images.

[0125] Among them, after obtaining the three-dimensional layout diagrams respectively corresponding to at least two images, since at least one wall surface may be included in the three-dimensional layout diagram, in order to accurately determine the splicing position corresponding to at least two three-dimensional layout diagrams, each three-dimensional layout diagram can be analyzed and processed to extract the wall surface information included in each three-dimensional layout diagram. It can be understood that the number of the extracted wall surface information can be one or more. After obtaining the wall surface information, a two-dimensional wall surface image corresponding to the wall surface information can be generated based on the three-dimensional layout diagram. Specifically, a two-dimensional wall surface image can be generated for each wall surface information. Therefore, one three-dimensional layout diagram can correspond to one or more two-dimensional wall surface images.

[0126] In some examples, generating a two-dimensional wall image corresponding to wall information may include: obtaining a constraint relationship for generating the two-dimensional wall image; and generating a two-dimensional wall image corresponding to the wall information based on the constraint relationship and a three-dimensional layout diagram. Among them, the constraint relationship may include: the area of the region corresponding to the wall information is positively correlated with the image resolution of the two-dimensional wall image.

[0127] Specifically, during the image acquisition operation, the images corresponding to different field of view angles or different shooting angles may include different wall information. For example, the number of wall information included in some pictures is one or more, and when the number of wall information is multiple, the wall areas corresponding to the multiple wall information may be different. Based on the above statements, the three-dimensional layout diagram corresponding to the above images may include one or more wall information, and when the number of wall information is multiple, the wall areas of the multiple wall information may be different. Therefore, the two-dimensional wall images corresponding to a three-dimensional layout diagram may be different. In order to accurately generate a two-dimensional wall image corresponding to the wall information, a scaling relationship between the wall information in the three-dimensional layout diagram and the two-dimensional wall image is pre-configured. Specifically, when the area of the region corresponding to the wall information in the three-dimensional layout diagram is large, a two-dimensional wall image with a higher image resolution can be generated. When the area of the region corresponding to the wall information in the three-dimensional layout diagram is small, a two-dimensional wall image with a smaller image resolution can be generated.

[0128] After configuring the scaling relationship between the wall information in the three-dimensional layout diagram and the two-dimensional wall image, the constraint relationship for generating the two-dimensional wall image can be obtained, and then the constraint relationship and the three-dimensional layout diagram can be analyzed and processed to generate a two-dimensional wall image corresponding to the wall information.

[0129] After obtaining the two-dimensional wall images corresponding to all the wall information in the three-dimensional layout diagram, all the two-dimensional wall images and at least two images can be analyzed and processed to determine the splicing positions corresponding to at least two three-dimensional layout diagrams. It can be understood that the number of splicing positions is related to the number of three-dimensional layout diagrams. When the number of three-dimensional layout diagrams is two, the number of splicing positions is two. When the number of three-dimensional layout diagrams is three, the number of splicing positions can be three.

[0130] In this embodiment, by extracting the wall information included in each three-dimensional layout diagram, generating a two-dimensional wall image corresponding to the wall information, and then determining the splicing positions corresponding to at least two three-dimensional layout diagrams based on the two-dimensional wall images corresponding to all the wall information and at least two images, the accuracy and reliability of determining the splicing positions are effectively guaranteed, and it is further convenient to splice at least two three-dimensional layout diagrams based on the splicing positions.

[0131] Figure 6 A flowchart for determining the splicing position corresponding to a three-dimensional layout diagram based on two-dimensional wall images corresponding to all wall information and at least two images provided by an embodiment of the present application; on the basis of the above embodiment, refer to the appendix Figure 6 As shown, this embodiment provides another implementation manner for determining the splicing position corresponding to a three-dimensional layout diagram. Specifically, determining the splicing position corresponding to a three-dimensional layout diagram based on two-dimensional wall images corresponding to all wall information and at least two images in this embodiment may include:

[0132] Step S601: Obtain the image similarity corresponding to any two two-dimensional wall images.

[0133] Step S602: Based on at least two images, determine the image adjacent relationship of at least two three-dimensional layout diagrams.

[0134] Step S603: Based on the image similarity, determine the splicing position corresponding to two adjacent three-dimensional layout diagrams.

[0135] Among them, after obtaining all two-dimensional wall images corresponding to all three-dimensional layout diagrams, an image similarity calculation operation can be performed on any two two-dimensional wall images, so that the image similarity between any two two-dimensional wall images can be obtained. After obtaining at least two images, at least two images can be analyzed and processed to determine the image adjacent relationship of at least two three-dimensional layout diagrams. In some examples, determining the image adjacent relationship of at least two three-dimensional layout diagrams based on at least two images may include: determining a first adjacent relationship corresponding to at least two images; based on the first adjacent relationship, determining the image adjacent relationship of at least two three-dimensional layout diagrams.

[0136] Specifically, after obtaining at least two images, an image similarity calculation operation can be performed on at least two images, so that a first adjacent relationship corresponding to at least two images can be determined. It can be understood that the first adjacent relationship may include an adjacent relationship and a non-adjacent relationship. For example, when at least two images include image a, image b, image c, and image d, there is an adjacent relationship between image a and image b, an adjacent relationship between image a and image c, and no adjacent relationship between image a and image d, that is, there is a non-adjacent relationship between image a and image d.

[0137] Since the three-dimensional layout diagram is determined based on the images, therefore, after determining the first adjacent relationship corresponding to at least two images, the image adjacent relationship of at least two three-dimensional layout diagrams can be determined based on the first adjacent relationship. Specifically, the image adjacent relationship of at least two three-dimensional layout diagrams corresponds to the first adjacent relationship.

[0138] After obtaining the image similarity and the image adjacency relationship of at least two 3D layout diagrams, the at least two 3D layout diagrams can be analyzed and processed based on the image similarity, so that the splicing positions corresponding to two adjacent 3D layout diagrams can be obtained.

[0139] It should be noted that the execution order between the above steps S601 - step S602 in this embodiment is not limited to the execution order described in the embodiment. Those skilled in the art can adjust the execution order between steps S601 - step S602 according to specific application requirements and design requirements. For example: step S602 can be executed before step S601, or step S602 can be executed simultaneously with step S602, which will not be elaborated here.

[0140] In some examples, after determining the first adjacency relationship corresponding to at least two images, the method in this embodiment may further include: performing a scaling process on at least two images so that the wall heights included in all images are the same.

[0141] Specifically, in order to further improve the quality and efficiency of analyzing and recognizing images, after analyzing and processing at least two images to determine the first adjacency relationship corresponding to at least two images, a scaling process can be performed on at least two images so that the wall heights included in all images are the same. Specifically, the wall height included in any one of the at least two images can be used as a reference to adjust the wall heights included in other images; or, a preset wall height can be obtained, and based on the set wall height, the wall heights included in all images can be adjusted. Of course, those skilled in the art can also use other methods to perform a scaling process on at least two images, as long as it can ensure that the wall heights included in all images are the same; in this way, when performing an image matching operation on at least two images, the accuracy of image matching can be effectively improved.

[0142] In this embodiment, by obtaining the image similarity corresponding to any two 2D wall images, then determining the image adjacency relationship of at least two 3D layout diagrams based on at least two images, and determining the splicing positions corresponding to two adjacent 3D layout diagrams based on the image similarity, the accuracy and reliability of determining the splicing positions are effectively ensured.

[0143] Figure 7 It is a flow chart for determining the splicing positions corresponding to two adjacent 3D layout diagrams based on image similarity provided by an embodiment of the present application; on the basis of the above embodiment, continue to refer to the appendix Figure 7As shown, this embodiment provides an implementation method for determining the splicing position corresponding to two adjacent three-dimensional layout diagrams based on image similarity. Specifically, the method for determining the splicing position corresponding to two adjacent three-dimensional layout diagrams based on image similarity in this embodiment may include:

[0144] Step S701: Obtain the first splicing wall surface corresponding to the highest image similarity;

[0145] Step S702: Determine the splicing position corresponding to two adjacent three-dimensional layout diagrams based on the first splicing wall surface.

[0146] For example, two adjacent three-dimensional layout diagrams are respectively Image A and Image B. Image A may correspond to two-dimensional wall surface images a1, a2, and a3, and Image B may correspond to two-dimensional wall surface images b1 and b2. Then, the image similarity corresponding to any two two-dimensional wall surface images can be obtained. For example: the first similarity between two-dimensional wall surface image a1 and two-dimensional wall surface image b1, the second similarity between two-dimensional wall surface image a2 and two-dimensional wall surface image b1, the third similarity between two-dimensional wall surface image a3 and two-dimensional wall surface image b1, the fourth similarity between two-dimensional wall surface image a1 and two-dimensional wall surface image b2, the fifth similarity between two-dimensional wall surface image a2 and two-dimensional wall surface image b2, and the sixth similarity between two-dimensional wall surface image a3 and two-dimensional wall surface image b2.

[0147] After obtaining the above-mentioned image similarities, the highest image similarity can be obtained. Assuming that the highest image similarity is the fourth similarity, the first splicing wall surface corresponding to the highest image similarity can be obtained as two-dimensional wall surface image a1 and two-dimensional wall surface image b2. After obtaining the first splicing wall surface, the first splicing wall surface can be analyzed and processed to determine the splicing position corresponding to two adjacent three-dimensional layout diagrams.

[0148] In some examples, determining the splicing position corresponding to two adjacent three-dimensional layout diagrams based on the first splicing wall surface may include: performing a feature extraction operation on the first splicing wall surface to obtain a first wall surface feature and a second wall surface feature; determining at least one splicing position corresponding to two adjacent three-dimensional layout diagrams based on the first wall surface feature and the second wall surface feature.

[0149] Specifically, after obtaining the first spliced wall surface pair, feature extraction operations can be performed on the first spliced wall surface pair, so that the first wall surface feature and the second wall surface feature can be obtained. After obtaining the first wall surface feature and the second wall surface feature, the first wall surface feature and the second wall surface feature can be analyzed and matched, so that the position with the highest matching degree between the two walls can be obtained. Then, the position with the highest matching degree can be determined as at least one splicing position corresponding to two adjacent three-dimensional layout diagrams.

[0150] It should be noted that after obtaining a splicing position corresponding to the first spliced wall surface, other splicing positions can be determined based on the adjacent relationship between the determined splicing position and at least two three-dimensional layout diagrams, so as to obtain all splicing positions corresponding to all three-dimensional layout diagrams.

[0151] In this embodiment, by obtaining the first spliced wall surface pair corresponding to the highest image similarity, and then determining the splicing position corresponding to two adjacent three-dimensional layout diagrams based on the first spliced wall surface pair, the accuracy and reliability of determining the splicing position are effectively guaranteed, and the quality and efficiency of splicing at least two three-dimensional layout diagrams based on the splicing position are further improved.

[0152] Figure 8 It is a schematic flowchart of another method for generating a three-dimensional house type provided by an embodiment of the present application; on the basis of the above embodiment, continue to refer to the attached Figure 8 As shown, after determining the splicing position corresponding to two adjacent three-dimensional layout diagrams, the method in this embodiment may further include:

[0153] Step S801: Detect whether the splicing position is reasonable.

[0154] After obtaining the splicing position, the three-dimensional layout diagram can be spliced based on the splicing position, and the splicing effect after splicing may meet the set requirements, or the splicing effect may not meet the set requirements. Therefore, in order to ensure the quality and effect of splicing the three-dimensional layout diagram based on the splicing position, after obtaining the splicing position, it is possible to detect whether the splicing position is reasonable. Specifically, detecting whether the splicing position is reasonable may include: pre-splicing the three-dimensional layout diagram based on the splicing position to obtain a pre-spliced house type; identifying the wall surface features corresponding to each wall surface in the pre-spliced house type; and detecting whether the splicing position is reasonable based on the wall surface features corresponding to each wall surface.

[0155] Specifically, after obtaining the splicing position, the three-dimensional layout diagram can be processed for splicing based on the splicing position, so that a pre-spliced house type can be obtained. After obtaining the pre-spliced house type, a wall feature extraction operation can be performed on the pre-spliced house type, so that the wall features corresponding to each wall in the pre-spliced house type can be obtained. After obtaining the wall features corresponding to each wall, the wall features corresponding to each wall can be analyzed to detect whether the splicing position is reasonable. It should be noted that for different wall features, different methods can be used to detect whether the splicing position is reasonable.

[0156] In some examples, when the wall feature includes the size feature of the wall, the detection of whether the splicing position is reasonable based on the wall features corresponding to each wall in this embodiment may include: determining the size deviation information between any two walls in the pre-spliced house type based on the size features corresponding to each wall; when the size deviation information is less than the set value, it is determined that the splicing position is reasonable; or, when the size deviation information is greater than or equal to the set value, it is determined that the splicing position is unreasonable.

[0157] Specifically, when the wall feature includes the size feature of the wall, for example, when the wall feature includes the height information and width information of the wall, the size deviation information between any two walls in the pre-spliced house type can be determined, for example, height deviation information, width deviation information, etc. After obtaining the size deviation information, the size deviation information can be analyzed and compared with the set value. When the size deviation information is less than the set value, it indicates that the image splicing effect in the pre-spliced house type meets the set requirements, and thus it can be determined that the splicing position is relatively reasonable. When the size deviation position is greater than or equal to the set value, it indicates that the image splicing effect in the pre-spliced house type does not meet the set requirements, and thus it can be determined that the splicing position is unreasonable.

[0158] In other examples, when the wall feature includes the relative position feature between walls, the detection of whether the splicing position is reasonable based on the wall features corresponding to each wall in this embodiment may include: detecting whether there is wall intersection or wall occlusion between any two walls in the pre-spliced house type based on the relative position features between each wall; when there is wall intersection or wall occlusion, it is determined that the splicing position is unreasonable; or, when there is no wall intersection and no wall occlusion, it is determined that the splicing position is reasonable.

[0159] Of course, those skilled in the art can also use other methods to detect whether the splicing position is reasonable, as long as the accuracy and reliability of detecting whether the splicing position is reasonable can be ensured, which will not be elaborated here.

[0160] Step S802: When the splicing position is reasonable, the three-dimensional layout diagram is spliced based on the splicing position. Or,

[0161] Step S803: When the splicing position is unreasonable, obtain the second splicing wall surface corresponding to the second highest image similarity, and determine the splicing position corresponding to the second splicing wall surface.

[0162] When the result of detecting the splicing position is that the splicing position is reasonable, it indicates that the splicing effect of the pre-spliced house type meets the preset requirements at this time. Then, the three-dimensional layout diagram can be directly spliced based on the splicing position. When the result of detecting the splicing position is that the splicing position is unreasonable, it indicates that the splicing effect of the pre-spliced house type does not meet the preset requirements at this time. At this time, in order to improve the quality and effect of the pre-spliced house type, the second splicing wall surface corresponding to the second highest image similarity can be obtained, and then the splicing position corresponding to the second splicing wall surface can be determined. Specifically, the specific implementation method of determining the splicing position corresponding to the second splicing wall surface is similar to the implementation method of determining the splicing position corresponding to the first splicing wall surface, and will not be elaborated here.

[0163] In this embodiment, by detecting whether the splicing position is reasonable, when the splicing position is reasonable, the three-dimensional layout diagram is spliced based on the splicing position. When the splicing position is unreasonable, the second splicing wall surface corresponding to the second highest image similarity is obtained, and the splicing position corresponding to the second splicing wall surface is determined, thereby effectively realizing that different data processing operations can be performed based on different results of detecting the splicing position, and further ensuring the quality and effect of splicing the three-dimensional layout diagram.

[0164] Figure 9 This is a schematic flowchart of splicing a three-dimensional layout diagram according to the splicing position provided by the embodiment of the present application to generate a three-dimensional house type corresponding to a room; on the basis of the above embodiment, refer to the attached Figure 9 As shown, this embodiment provides an implementation method for generating a three-dimensional house type corresponding to a room. Specifically, the splicing process of the three-dimensional layout diagram according to the splicing position in this embodiment to generate a three-dimensional house type corresponding to a room may include:

[0165] Step S901: Splice all the three-dimensional layout diagrams based on the splicing position to obtain spliced house type data corresponding to the room;

[0166] Step S902: Detect whether there are gaps at all splicing positions in the spliced house type data;

[0167] Step S903: When there are no gaps at all splicing positions in the spliced house type data, determine the spliced house type data as the three-dimensional house type; or,

[0168] Step S904: When there is a gap at a splicing position in the spliced room type data, optimize the spliced room type data to generate a three-dimensional room type.

[0169] Among them, after obtaining the splicing position, all three-dimensional layout diagrams can be spliced based on the splicing position, so as to obtain spliced room type data corresponding to a room (corresponding to the entire room). After obtaining the spliced room type data, the spliced room type data can be analyzed to detect whether there are gaps at all splicing positions in the spliced room type data. When there are no gaps at all splicing positions in the spliced room type data, it means that the spliced room type data obtained based on the splicing position meets the set requirements, so the spliced room type data can be determined as the three-dimensional room type. When there is a gap at a splicing position in the spliced room type data, it means that the spliced room type data obtained based on the splicing position does not meet the set requirements. Regarding the quality and effect of splicing the room type data, the spliced room type data can be optimized to generate a three-dimensional room type.

[0170] For example, the splicing positions include: position a, position b, and position c, and the three-dimensional room types include: data A, data B, and data C. Based on the splicing position a, data A and data B are spliced, and then based on the splicing position b, data B and data C are spliced, and based on the splicing position c, data A and data C are spliced, so as to obtain spliced room type data corresponding to a room. Then, it is detected whether there are gaps corresponding to the above positions a, b, and c in the spliced room type data. Assuming that there are no gaps at positions a, b, and c, the spliced room type data can be determined as the three-dimensional room type; when there is a gap at position b, that is, after splicing data A and data B based on the splicing position a to generate data AB and splicing data A and data C based on the splicing position c to generate CA, when splicing data AB and CA based on the splicing position b, there is an error between data AB and data CA. At this time, in order to ensure the quality and effect of splicing the room type data, the spliced room type data can be optimized to generate a three-dimensional room type, and the generated three-dimensional room type meets the set requirements.

[0171] In this embodiment, it is effectively realized that when the generated spliced room type data meets the set requirements, the spliced room type data can be directly determined as the final three-dimensional room type; when the generated spliced room type data does not meet the set requirements, the spliced room type data can be optimized, so that a three-dimensional room type that meets the set requirements can be generated, further improving the accuracy and reliability of the method.

[0172] Figure 10It is a schematic flowchart of optimizing the spliced room type data provided by the embodiment of the present application to generate a three-dimensional room type; on the basis of the above embodiment, refer to the attached Figure 10 As shown, this embodiment provides an implementation manner for generating a three-dimensional room type. Specifically, the optimization processing of the spliced room type data in this embodiment to generate a three-dimensional room type may include:

[0173] Step S1001: Optimize all splicing positions based on the existence of a gap at a splicing position in the spliced room type data to obtain optimized splicing positions.

[0174] Among them, when there is a gap at a splicing position in the spliced room type data, it means that the obtained spliced room type data does not meet the set requirements. At this time, in order to make the spliced room type data meet the set requirements, all splicing positions can be optimized based on the gap existing at a splicing position in the spliced room type data to obtain optimized splicing positions. In some examples, optimizing all splicing positions based on the existence of gaps at at least one splicing position in the spliced room type data to obtain optimized splicing positions may include: obtaining the splicing matching degree corresponding to each splicing position for the three-dimensional layout diagram; determining the two three-dimensional layout diagrams corresponding to the highest splicing matching degree as the reference image pair; adjusting the splicing positions corresponding to the three-dimensional layout diagram based on the reference image pair and the gap to obtain optimized splicing positions.

[0175] Specifically, after performing the splicing operation on all three-dimensional layout diagrams based on the splicing position, the splicing matching degree corresponding to each splicing position for the three-dimensional layout diagram can be obtained. After obtaining the splicing matching degree corresponding to each splicing position, the highest splicing matching degree can be obtained, and then the two three-dimensional layout diagrams corresponding to the highest splicing matching degree are determined as the reference image pair. It can be understood that the number of reference image pairs can be one or more. After obtaining the reference image pair, the splicing positions corresponding to the three-dimensional layout diagram can be adjusted based on the reference image pair and the gap, so as to obtain optimized splicing positions.

[0176] In some examples, the reference image pair may include: a first image pair in the first direction and a second image pair in the second direction; at this time, the adjustment of the splicing positions corresponding to the three-dimensional layout diagram based on the reference diagram and the gap in this embodiment to obtain optimized splicing positions may include: determining the first adjustment distance of the splicing position in the first direction based on the first image pair and the gap; determining the second adjustment distance of the splicing position in the second direction based on the second image pair and the gap; respectively adjusting the splicing position in the first direction and the second direction based on the first adjustment distance and the second adjustment distance to obtain optimized splicing positions.

[0177] The above-mentioned first direction may be the length direction of a room, the second direction may be the width direction of the room, and the gap existing at the splicing position may include at least one of the following: a length gap in the length direction and a width gap in the width direction. Therefore, in order to ensure the quality and effect of optimizing the splicing position, two optimization operations can be performed on the splicing position, that is, optimization operations are performed in the length direction and the width direction respectively.

[0178] Specifically, after obtaining the first image pair and the gap, the first adjustment distance of the splicing position in the length direction can be determined based on the first image pair and the gap. After obtaining the second image pair and the gap, the second adjustment distance of the splicing position in the length direction can be determined based on the second image pair and the gap. Then, the splicing position can be adjusted in the first direction and the second direction respectively based on the first adjustment distance and the second adjustment distance to obtain the optimized splicing position. It can be understood that the obtained optimized splicing position is used to ensure that there is no gap in the spliced room type data.

[0179] Step S1002: Perform splicing processing on all three-dimensional layout diagrams based on the optimized splicing position to generate a three-dimensional room type corresponding to the room.

[0180] After obtaining the optimized splicing position, splicing processing can be performed on all three-dimensional layout diagrams based on the optimized splicing position, so that a three-dimensional room type corresponding to the room can be generated, and the generated three-dimensional room type meets the set requirements.

[0181] In this embodiment, all splicing positions are optimized based on the existence of a gap at a splicing position in the spliced room type data to obtain the optimized splicing position. Then, splicing processing is performed on all three-dimensional layout diagrams based on the optimized splicing position, so that a three-dimensional room type that meets the set requirements and corresponds to the room can be obtained, further ensuring the quality and effect of the generation of the three-dimensional room type and improving the stability and reliability of the method.

[0182] Figure 11 It is a schematic flowchart of another method for generating a three-dimensional room type provided by an embodiment of the present application; on the basis of any one of the above embodiments, continue to refer to the attached Figure 11 As shown, the method in this embodiment may further include:

[0183] Step S1101: Identify the entities and entity feature information included in the room.

[0184] Step S1102: Perform fusion processing on the entities, entity feature information, and three-dimensional room type to generate a target three-dimensional room type.

[0185] After obtaining at least two images of different perspectives of the room, the at least two images can be analyzed to identify the entities included in the room and the entity feature information. The entities included in the above room can include: doors and windows, furniture, household appliances, etc. The entity feature information can include: entity contours and entity geometric dimensions. The entity feature data includes at least one of the following: the starting point of the entity, the ending point of the entity, the recognized geometric dimension of the entity, the type information of the entity, and the spatial position relationship between the entity and other entities.

[0186] After obtaining the entity and the entity feature information, the entity, the entity feature information, and the three-dimensional house type can be fused to generate a target three-dimensional house type incorporating the entity. This enables the user to more intuitively obtain the entities included in the room through the target three-dimensional house type, which is conducive to ensuring or improving the quality and effect of rendering or decorating the target three-dimensional house type, and further enhancing the practicality of this method.

[0187] Based on any of the above embodiments, to improve the practicality of this method, after generating the three-dimensional house type corresponding to the room, the method of this embodiment may further include:

[0188] Step S1201: Generate a construction house type corresponding to the three-dimensional house type.

[0189] Step S1202: Obtain construction verification information based on the three-dimensional house type and the construction house type.

[0190] Among them, after obtaining the three-dimensional house type, construction operations can be performed based on the three-dimensional house type, thereby generating a construction house type corresponding to the three-dimensional house type. It can be understood that the construction house type can refer to the actual house type constructed based on the three-party house type. After obtaining the construction house type, the three-dimensional house type and the construction house type can be compared to obtain construction verification information, which is used to identify the matching information between the construction house type and the three-dimensional house type. For example: the matching information between the door in the three-dimensional house type and the door in the construction house type, the matching information between the wall in the three-dimensional house type and the wall in the construction house type, etc.

[0191] In some instances, after obtaining the construction verification information, the construction verification information can be displayed so that the user can quickly and directly obtain the quality and effect of the construction operations based on the three-dimensional house type through the construction verification information, further enhancing the practicality of this method.

[0192] In some instances, after obtaining the construction verification information, it is possible to extract the room type objects in the construction room type that do not meet the set requirements, such as doors, windows, walls, etc. After obtaining the above room type objects, it is possible to perform a correction operation on the construction room type based on the data corresponding to the room type objects in the three-dimensional room type, so as to obtain a corrected construction room type that meets the set requirements. This not only ensures the quality and effect of the construction operation based on the three-dimensional room type, but also ensures that the construction room type can meet the user's set requirements.

[0193] Based on any one of the above embodiments, after generating the three-dimensional room type corresponding to the room, the method in this embodiment may further include:

[0194] Step S1301: Obtain a display request for the three-dimensional room type.

[0195] Step S1302: Based on the display request, use a set device to display the three-dimensional room type.

[0196] Among them, after obtaining the three-dimensional room type corresponding to the room, when the user has a display requirement for the three-dimensional room type, the user can input an execution operation to the generation device of the three-dimensional room type. After obtaining the execution operation, a display request for the three-dimensional room type can be generated; then, based on the display request, the three-dimensional room type can be displayed using a set device.

[0197] In some instances, the display request may include at least one of the following: augmented reality display request, virtual reality display request, mixed reality display request, image reality display request; correspondingly, the set device may include at least one of the following: augmented reality device, virtual reality device, mixed reality device, image reality device.

[0198] Specifically, when obtaining an augmented reality (AR) display request, the corresponding set device may be an AR device. It can be understood that the above AR device may be a head-mounted display device, such as an AR glasses. Then, based on the AR display request, the obtained three-dimensional room type can be displayed using the AR device.

[0199] When obtaining a virtual reality (VR) display request, the corresponding set device may be a VR device. It can be understood that the above VR device may be a head-mounted display device, such as a VR glasses. Then, based on the VR display request, the obtained three-dimensional room type can be displayed using the VR device.

[0200] When a Mixed Reality (MR; or, Hybrid Reality, HR for short) display request is obtained, the corresponding setting device can be an augmented reality MR device. It can be understood that the above MR device can be a head-mounted display device, such as: MR glasses. Then, based on the MR display request, the obtained three-dimensional house type can be displayed using the MR device.

[0201] When a Cinematic Reality (CR) display request is obtained, the corresponding setting device can be an augmented reality CR device. It can be understood that the above CR device can be a head-mounted display device, such as: CR glasses. Then, based on the CR display request, the obtained three-dimensional house type can be displayed using the CR device.

[0202] In this embodiment, by obtaining the display request for the three-dimensional house type, and then based on the display request and using the setting device to display the three-dimensional house type, it effectively realizes that when there is a display requirement for the three-dimensional house type, the three-dimensional house type can be displayed based on the display request and the setting device, so that the user can directly know the house layout and room effect of the three-dimensional house type, further improving the practicability of this method.

[0203] Based on any of the above embodiments, after generating the three-dimensional house type corresponding to the room, the method in this embodiment may further include:

[0204] Step S1401: Obtain the generation quality of the three-dimensional house type.

[0205] Step S1402: When the generation quality does not meet the set conditions, generate an image reshooting request to re-obtain at least two images of different perspectives of the room based on the image reshooting request.

[0206] Among them, after obtaining the three-dimensional house type, the three-dimensional house type can be analyzed and processed to obtain the generation quality of the three-dimensional house type. Specifically, the specific acquisition method of the generation quality of the three-dimensional house type in this embodiment is not limited, and those skilled in the art can set it according to specific application scenarios or application requirements. For example: there is a pre-set evaluation rule for analyzing and processing the three-dimensional house type, and the evaluation rule is used to analyze and process the three-dimensional house type, so that the generation quality of the three-dimensional house type can be obtained; or, there is a pre-configured machine learning model for determining the generation quality of the three-dimensional house type. After obtaining the three-dimensional house type, the three-dimensional house type can be input into the machine learning model, so that the generation quality of the three-dimensional house type can be obtained.

[0207] It can be understood that the generation quality can be represented in the form of a score or a grade. For example, the generation quality can be 80 points, 90 points, 95 points, etc.; or, the generation quality can be the first grade used to identify higher quality, or the generation quality can be the second grade used to identify general quality, or the generation quality can be the third grade used to identify lower quality, etc.

[0208] After obtaining the generation quality, it can be detected whether the generation quality meets the set conditions. Among them, set conditions for analyzing and processing the three-dimensional house type are pre-configured. It can be understood that the set conditions corresponding to different application scenarios can be different. After obtaining the generation quality and the set conditions, it can be detected whether the generation quality meets the set conditions. When the generation quality does not meet the set conditions, it means that the generated three-dimensional house type cannot meet the user's design requirements. At this time, an image reshooting request can be generated to re-obtain at least two images from different perspectives of the room based on the image reshooting request, and then a three-dimensional house type can be regenerated based on the at least two re-obtained images, so that the regenerated three-dimensional house type can meet the user's design requirements. When the generation quality meets the set conditions, it means that the generated three-dimensional house type can meet the user's design requirements, thus ensuring the generation quality and effect of the three-dimensional house type.

[0209] For example, set conditions for analyzing and processing the three-dimensional house type are pre-configured. The set conditions include the minimum quality limit for meeting the user's design requirements. For example, the minimum quality limit can be 90 points. After obtaining the three-dimensional house type, the generation quality of the three-dimensional house type can be obtained. When the generation quality is 93 points or 95 points, it proves that the generation quality of the above three-dimensional house type meets the set conditions, and then the generated three-dimensional house type can be output. When the generation quality is 85 points or 88 points, it proves that the generation quality of the above three-dimensional house type does not meet the set conditions. At this time, an image reshooting request can be generated to re-obtain at least two images from different perspectives of the room based on the image reshooting request, and then a three-dimensional house type can be re-established based on the at least two re-obtained images, and the generation quality of the three-dimensional house type can be obtained. When the generation quality meets the set conditions, the re-established three-dimensional house type can be output.

[0210] In this embodiment, by obtaining the generation quality of the three-dimensional house type, when the generation quality does not meet the set conditions, an image reshooting request is generated to re-obtain at least two images from different perspectives of the room based on the image reshooting request, effectively realizing that a three-dimensional house type can be regenerated based on the at least two re-obtained images, so that the regenerated three-dimensional house type can meet the user's design requirements, and further improving the quality and effect of the three-dimensional house type generation.

[0211] In specific applications, refer to the appendixFigure 12 As shown, taking the example of obtaining four images of a room through a mobile phone, the present application embodiment provides a method for generating a three-dimensional house type. This method can use several hand-taken pictures taken from different angles, and through methods such as house type reconstruction, house type splicing, and global optimization of the hand-taken pictures, finally obtain a whole-house floor plan. This can greatly release the potential of users, thereby promoting the development of the intelligent home improvement industry. In addition, the execution subject of this method can be a three-dimensional house type generation device, and this generation device can include: an input module, a 2d layout detection module communicatively connected to the input module, an internal and external parameter calibration module, a single Figure 3 d layout (layout diagram) reconstruction module, a layout splicing module, a global optimization module, an entity recognition module, and an output module. Specifically, this method can include the following steps:

[0212] Step 1: Obtain four hand-taken pictures through the input module.

[0213] Given that the field of view angle of the main camera of most users' mobile phones is about 60 degrees, therefore, the shooting angles of the 4 pictures can be taken obliquely across from the four corners, as Figure 3 shown.

[0214] It should be noted that for the number of input hand-taken pictures, different numbers of hand-taken pictures can be input for different scenarios. For example, in some application scenarios, 2, 3, or more hand-taken pictures can also be input for subsequent whole-house layout reconstruction operations. In addition, the shooting positions, shooting angles, etc. of the hand-taken pictures can also be adjusted arbitrarily, as long as there is an overlapping area between the images obtained by shooting.

[0215] Step 2: Use the 2d layout detection module to obtain four 2d layout diagrams corresponding to the four hand-taken pictures.

[0216] Step 3: Use the internal and external parameter calibration module to obtain the camera parameters corresponding to the four hand-taken pictures.

[0217] Specifically, as shown in the appendix Figure 13 shown, when determining the camera parameters corresponding to the four images, a method for calibrating internal and external parameters based on vanishing points can be used to obtain them. When the image acquisition device is the main camera of a mobile phone, since the distortion of the main camera of a mobile phone is relatively small, therefore, the calibration of the camera distortion coefficient can be not considered. When the camera parameters include camera internal parameters and camera external parameters, the following will describe in detail each step included in determining the camera parameters corresponding to the four images:

[0218] Step 3.1: Line segment extraction.

[0219] Four hand - taken pictures are obtained through a mobile phone. The line segment extraction operation is performed on the four hand - taken pictures through the Line Segment Detector (LSD) or a deep - learning network to obtain the line segments included in each hand - taken picture.

[0220] Step 3.2: Vanishing point calculation.

[0221] Based on the line segments included in each image, using the constraint condition that the directions of the three vanishing points are perpendicular to each other, all the vanishing points corresponding to all the line segments are counted, the number of line segments covered by a group of three mutually perpendicular vanishing points is counted, and the group of vanishing points with the largest number of line segments is obtained. The above - mentioned vanishing points are used as the target group of vanishing points (vpx, vpy, vpz), where vpx, vpy, and vpz are used to represent the vanishing points in the x, y, and z directions respectively.

[0222] Step 3.3: Camera internal parameter calculation.

[0223] Based on the three vanishing points (vpx, vpy, vpz) obtained previously, the calculation of the camera internal parameters is carried out.

[0224] Let K represent the camera internal parameter matrix, and the representation of K is as follows:

[0225]

[0226] Among them, fx is the focal length information in the X direction, fy is the focal length information in the Y direction, cx is the coordinate information in the X direction, and cy is the coordinate information in the Y direction. For K, it is usually considered that fx = fy. Therefore, there are only three unknowns: the principal point coordinates f, cx, and cy.

[0227] In addition, w represents the intermediate result corresponding to the operation of the camera internal parameter matrix. K is calculated using w, and the relationship between w and K is as follows: w=(KK T ) -1 , for each group of vanishing points vpi, vpj, since three vanishing points can be obtained from one picture and any two vanishing points are perpendicular to each other; a linear equation about the elements of w can be generated: The constraints of three pairs of vanishing points are combined to obtain the equation Aw = 0, where A is a 3×4 matrix (a system of equations can be obtained from three vanishing points), w is obtained from the null vector of A, that is, the value of w is obtained through A, and then the value of the internal parameter matrix K can be obtained by using the Cholesky decomposition of w, thus completing the calibration of the camera internal parameters.

[0228] Step 3.4: Camera external parameter calculation.

[0229] For each hand - photographed image, assuming that the world coordinate system coincides exactly with the camera coordinate system, there is only a rotation transformation and no translation for the extrinsic camera parameters between the world coordinate system and the camera coordinate system. At this time, only the rotation matrix between the world coordinate system and the camera coordinate system needs to be calculated to complete the calibration operation of the extrinsic camera parameters.

[0230] The relationship between the vanishing point and the camera parameters is as follows:

[0231]

[0232]

[0233] Where α is a set coefficient, Vp Z and Vp x are the vanishing point in the Z - direction and the vanishing point in the X - direction respectively, K is the camera intrinsic matrix, r x 、r y and r z are the rotation amounts corresponding to the X - direction, Y - direction, and Z - direction between the world coordinate system and the camera coordinate system respectively.

[0234] Based on the above formula, the expression between the rotation vector and the vanishing point can be obtained as follows:

[0235]

[0236] Here, Vpx and Vpz represent the vanishing points in the x - direction and z - direction respectively. The direction vectors of the two directions are directly calculated through the vanishing points. After obtaining the vectors of any two directions, the vector of the other direction is obtained by cross - multiplying the first two vectors, that is, r z =r y ×r x , thus obtaining the rotation matrix R and completing the calibration of the extrinsic camera parameters.

[0237] Step 4: Use the single Figure 3 d layout reconstruction module to obtain the 3d layout maps corresponding to the four 2d layout maps.

[0238] Specifically, use the 3d layout reconstruction module to perform 3d reconstruction on each pixel point in the 2d layout map based on the hand - photographed image, 2d layout map, and camera parameters, thereby obtaining the single Figure 3 d layout map.

[0239] Set the height of the camera from the ground as h (for example: 1.5m or 1.6m, etc.) when taking images. The mapping relationship between the world coordinate system and the pixel coordinate system is as follows:

[0240]

[0241] Among them, α is a preset coefficient, (u, v) is a point in the pixel coordinate system, K is the camera internal parameter, R is the rotation matrix (corresponding to the camera external parameter), (X, Y, Z) is a point in the world coordinate system, X is the value of the point in the world coordinate system in the horizontal direction, Y is the value of the point in the world coordinate system in the vertical direction, Z is the value of the point in the world coordinate system in the height direction, and among them, Z is h.

[0242] Specifically, there is the following constraint relationship between the pixel points in the 2d layout diagram and the pixel points in the 3d layout diagram: for each point on the wall surface and the intersection line of the ground and the floor, the coordinate Z in the world coordinate system is known, and then the mapping from the point in the pixel coordinate system to the world coordinate system can be realized. For the points on the wall surface, the depth information from the camera is the same as that of the points on the intersection line of the wall surface and the ground, so that each point on the wall surface can be 3d reconstructed. For the points on the roof, through the 3d coordinates of the points on the intersection line of each wall surface and the roof, the height of the roof from the ground can be obtained, and then each point on the roof can also be 3d reconstructed. Thus, through the 3d reconstruction operation on the 2d pixel points, the 3d layout corresponding to the 2d layout diagram is obtained.

[0243] Step 5: Use the layout splicing module to determine the splicing position of the 3d layout diagram and perform the splicing operation based on the splicing position.

[0244] In order to obtain the overall three-dimensional room type of a room, it is necessary to splice the 3d layouts corresponding to the 4 views. The layout splicing module can find the corresponding splicing position between two adjacent 3d layout diagrams, and then can perform the splicing operation based on the splicing position. Specifically, refer to the appendix Figure 14 As shown, the splicing operation based on the splicing position includes the following steps:

[0245] Step 5.1: Preprocessing.

[0246] The preprocessing process can achieve the following two functions:

[0247] (1) Sort the four hand-taken pictures to obtain the adjacent relationship between any two hand-taken pictures. Specifically, it can be judged which two views are adjacent to each other.

[0248] Among them, the adjacent relationship between any two hand-taken pictures is judged through the feature point matching operation. Specifically, the number of matching feature points between any two hand-taken pictures can be obtained. When the number of matching feature points is large, the probability that the two hand-taken pictures are determined to be adjacent images is large; when the number of matching feature points is small, the probability that the two hand-taken pictures are determined to be adjacent images is small.

[0249] (2) Scale the dimensions of each view to ensure that the wall heights before splicing all views are equal.

[0250] Step 5.2: Generation of 2D wall images.

[0251] Based on the reconstructed 3D layout diagram, generate a corresponding 2D wall image for each wall included in the 3D layout diagram. Among them, there is a corresponding scaling relationship between the 2D wall image and the wall in the 3D layout diagram. Based on the above scaling relationship, the 2D wall images of each wall included in the 3D layout diagram can be obtained.

[0252] Step 5.3: Registration of 2D wall images.

[0253] Based on the results of the above Step 5.1 and Step 5.2, perform image registration operations on the 2D wall images corresponding to different 3D layout diagrams to obtain the splicing positions corresponding to two adjacent 3D layout diagrams. Specifically, the matching algorithm can be to extract and match feature points based on traditional feature descriptors such as SIFT, or directly judge the similarity between two images based on deep learning algorithms, and then determine the splicing positions corresponding to the 3D layout diagrams.

[0254] After obtaining the spliced wall corresponding to the splicing position, it is also possible to judge whether the determined two spliced walls are reasonable. For example, if the widths of the two walls are too different, or there are situations such as walls blocking each other or walls crossing after splicing, it is determined that the splicing position or the spliced wall is not suitable for splicing. Therefore, it is necessary to find another pair of spliced walls with a slightly lower matching feature intensity until a suitable pair of registered wall surfaces is obtained.

[0255] Step 6: Perform optimization operations using the global optimization module.

[0256] After obtaining the splicing position, the four views can be spliced based on the splicing position, that is, the views Figure 1-2 2-3, 3-4, 4-1 all clearly define the matching surfaces and splicing positions. However, after performing the image splicing operation using the splicing position, there may be a situation where the head and tail cannot be joined after splicing, such as Figure 15 shown.

[0257] Therefore, after obtaining the splicing position, it is also possible to judge whether there will be a situation where the spliced image data cannot be joined after performing the splicing operation at the splicing position. Judge whether the splicing positions of all views match the obtained splicing positions. If they do not match, optimization is required; if not, it means that the splicing position is very accurate and no fine-tuning is required at this time.

[0258] For example, as shown in the reference appendix Figure 15 , when splicing Figure 1 and Figure 2 to obtain Figure 12 , and then splicing Figure 3 and Figure 4 to obtain View 34, when splicing Figure 12 with View 34, a situation where they cannot be correctly joined occurs, that is, a gap of length a is generated. Simply put, Figure 1 - Figure 4 are supposed to be joined together. There was originally a splicing position between Figure 2 and Figure 3 , but they cannot be joined, and it is found that there is a gap of a.

[0259] To solve the above problem, the global optimization module will simultaneously consider the gap corresponding to the splicing position between 12 and 34, and then use a pair with strong registration features to guide the splicing distance of the other pair, so as to ensure that the spliced room type data takes both the head and the tail into account.

[0260] For example: when the number of feature point matches between Figure 1 and Figure 2 is less than the number of matching feature points between Figure 3 and Figure 4 , then Figure 3 and Figure 4 can be used as a benchmark to adjust the positions of Figure 1 and Figure 2 so that the gap a disappears. In addition, after optimizing the splicing of Figure 12 and View 34, it is also possible to optimize the splicing distance for the two pairs of Figure 1 and Figure 4 , Figure 2 and Figure 3 once, so as to complete the global optimization. During the above optimization process, there is also a mutual guiding relationship between Figure 1 and Figure 4 , Figure 2 and Figure 3 . The image pair with more matching feature points is determined as the benchmark to adjust the other image pair, so that separate optimization operations can be carried out in the length direction and the width direction, effectively ensuring the quality and effect of the optimization.

[0261] Step 7: Hard decoration recognition.

[0262] After obtaining the complete 2D and 3D layout images of the room, the hard decoration recognition module is used to identify the position and size information of the doors and windows on each wall. Specifically, a CNN network can be used to detect and identify the positions of the doors and windows, and then the identified entities are fused with the 3D house type, so as to generate a 3D house type integrated with entities and entity features. After the above series of operations, the 2D house type layout diagram and 3D house type layout diagram of the whole house can be obtained from the 4 2D hand-taken pictures taken at different angles, and entity information such as the positions of the doors and windows is included, which is beneficial to improving the quality and effect of the user's viewing of the 3D house type.

[0263] The generation method provided by this application embodiment takes several hand-taken pictures at several different angles of the house through the user's mobile phone, and then analyzes and processes the hand-taken pictures, and reconstructs the house type diagram of the room through the layout splicing algorithm to obtain the 2D and 3D layouts of the whole room and house type information such as doors and windows. This effectively solves the problem of inconvenient operation in image acquisition using panoramic devices, greatly reduces the usage threshold of the 3D house type generation method, expands the applicable scope of this method, has great value for the growth of the user base in the field of intelligent home decoration, and further improves the practicality of this method.

[0264] Figure 16 It is a schematic flowchart of a method for generating a 3D house type provided by an embodiment of this application; refer to the appendix Figure 16 As shown, this embodiment provides a method for generating a 3D house type. The execution subject of this method can be a 3D house type generation device. It can be understood that this 3D house type generation device can be implemented as software, or a combination of software and hardware. Specifically, the method for generating a 3D house type can include:

[0265] Step S1601: Obtain at least two images of the room from different perspectives.

[0266] Among them, the implementation manner and implementation effect of the above steps in this embodiment are similar to those of step S201 in the above embodiment. For specific reference, please refer to the above description and will not be elaborated here.

[0267] Step S1602: Generate 2D layout diagrams corresponding to at least two images respectively.

[0268] Step S1603: Determine the camera parameters corresponding to at least two images.

[0269] Step S1604: Generate a 3D house type corresponding to at least two images based on at least two images, 2D layout diagrams, and camera parameters.

[0270] Among them, the implementation manner and implementation effect of the above steps in this embodiment are similar to those of steps S401 - S403 in the above embodiment. For specific details, please refer to the above description and will not be elaborated here.

[0271] The technical solution in this embodiment provides at least two images from different perspectives of a room. Through methods such as 2D layout map recognition operations and camera parameter recognition operations, a three - dimensional room type corresponding to a single image can be obtained, thereby effectively improving the practicality of this method.

[0272] In some instances, the shooting perspectives of the images are different, and there is an overlapping area between the shooting areas corresponding to the shooting perspectives of at least two images.

[0273] In some instances, the 2D layout map includes wall information in the room.

[0274] In some instances, determining the camera parameters corresponding to at least two images may include: calculating the vanishing point information corresponding to at least two images; calculating the camera internal parameters corresponding to at least two images according to the vanishing point information; and determining the camera external parameters corresponding to at least two images according to the vanishing point information and the camera internal parameters.

[0275] In some instances, generating a three - dimensional room type corresponding to at least two images based on at least two images, a 2D layout map, and camera parameters may include: obtaining the height information between the image shooting positions corresponding to at least two images and the ground; determining the spatial constraint relationship corresponding to the pixel points in at least two images based on the height information, at least two images, and camera parameters; and performing a 3D reconstruction operation on the pixel points in the 2D layout map based on the spatial constraint relationship to generate three - dimensional room types corresponding to at least two images respectively.

[0276] The specific implementation manner, implementation effect, and implementation principle of the above method in this embodiment are similar to those of the method in the corresponding embodiment above Figure 2 - Figure 15 For specific details, please refer to the above description and will not be elaborated here.

[0277] Figure 17 It is a flowchart of another method for generating a three - dimensional room type provided by an embodiment of the present application; referring to the appendix Figure 17 As shown, this embodiment provides another method for generating a three - dimensional room type. The execution subject of this method can be a three - dimensional room type generation device. It can be understood that this three - dimensional room type generation device can be implemented as software, or a combination of software and hardware. Specifically, this method for generating a three - dimensional room type may include:

[0278] Step S1701: Obtain at least two three - dimensional layout maps from different perspectives of the room.

[0279] Step S1702: Determine the splicing positions corresponding to at least two three-dimensional layout diagrams.

[0280] Step S1703: Perform splicing processing on at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional house type corresponding to the room.

[0281] The following is a detailed description of each of the above steps:

[0282] Step S1701: Obtain at least two three-dimensional layout diagrams of different perspectives of the room.

[0283] Among them, when there is a need to generate a three-dimensional house type for a room, at least two three-dimensional layout diagrams of different perspectives of the room can be obtained. Specifically, in this embodiment, the specific implementation method for obtaining at least two three-dimensional layout diagrams of different perspectives of the room is not limited, and those skilled in the art can set it according to specific application requirements and design requirements. For example: at least two hand-taken images of a room can be obtained, and at least two three-dimensional layout diagrams can be obtained by analyzing and processing the hand-taken images; or, at least two three-dimensional layout diagrams corresponding to a room are pre-stored, and at least two three-dimensional layout diagrams can be obtained by accessing a preset area.

[0284] It should be noted that the perspectives of at least two three-dimensional layout diagrams are different, that is, the perspectives of any two of the at least two three-dimensional layout diagrams are different, so that the reconstruction operation of the area within a larger perspective range in the room can be realized. In some other examples, in order to accurately obtain the three-dimensional house type corresponding to the whole room, there is an overlapping area between any two of the at least two three-dimensional layout diagrams, so that the reconstruction operation of the area within all perspective ranges in the room can be realized.

[0285] Step S1702: Determine the splicing positions corresponding to at least two three-dimensional layout diagrams.

[0286] Step S1703: Perform splicing processing on at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional house type corresponding to the room.

[0287] Among them, in this embodiment, the implementation methods and implementation effects of the above steps are similar to those of steps S203 - S204 in the above embodiment. For specific references, please refer to the above description, and details will not be repeated here.

[0288] In the technical solution of this embodiment, by obtaining at least two three-dimensional layout diagrams of different perspectives of a room, determining the splicing positions corresponding to the at least two three-dimensional layout diagrams, and then performing splicing processing on the at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional room type corresponding to the room, a method for splicing three-dimensional layout diagrams corresponding to different views is effectively realized, which can effectively solve the problem of large parallax splicing in the spliced view when performing view splicing operations for a room, and further improves the practicability of this method.

[0289] In some examples, performing splicing processing on at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional room type corresponding to the room may include: performing splicing processing on all the three-dimensional layout diagrams based on the splicing positions to obtain spliced room type data corresponding to the room; detecting whether there are gaps at all the splicing positions in the spliced room type data; when there are no gaps at all the splicing positions in the spliced room type data, determining the spliced room type data as the three-dimensional room type; or, when there is a gap at a splicing position in the spliced room type data, performing optimization processing on the spliced room type data to generate the three-dimensional room type.

[0290] In some examples, performing optimization processing on the spliced room type data to generate the three-dimensional room type includes: optimizing all the splicing positions based on the existence of a gap at a splicing position in the spliced room type data to obtain optimized splicing positions; performing splicing processing on all the three-dimensional layout diagrams based on the optimized splicing positions to generate a three-dimensional room type corresponding to the room.

[0291] In some examples, optimizing all the splicing positions based on the existence of a gap at a splicing position in the spliced room type data to obtain optimized splicing positions may include: obtaining the splicing matching degree corresponding to any one splicing position for at least two three-dimensional layout diagrams; when the splicing matching degree is greater than or equal to a preset threshold, determining the two three-dimensional layout diagrams corresponding to the splicing matching degree as a reference image pair; adjusting the splicing positions corresponding to the at least two three-dimensional layout diagrams based on the reference image pair and the gap to obtain optimized splicing positions.

[0292] In some examples, the reference image pair includes: a first image pair in the first direction and a second image pair in the second direction; adjusting the splicing positions corresponding to the at least two three-dimensional layout diagrams based on the reference diagram and the gap to obtain optimized splicing positions may include: determining a first adjustment distance of the splicing position in the first direction based on the first image pair and the gap; determining a second adjustment distance of the splicing position in the second direction based on the second image pair and the gap; respectively adjusting the splicing position in the first direction and the second direction based on the first adjustment distance and the second adjustment distance to obtain optimized splicing positions.

[0293] In this embodiment, the specific implementation manner, implementation effect, and implementation principle of the above method are similar to those of the method in the corresponding embodiment above Figure 2 - Figure 15 and can be specifically referred to the above description, which will not be elaborated here.

[0294] Figure 18 FIG. is a schematic structural diagram of a three-dimensional house type generation device provided by an embodiment of the present application; refer to the attached Figure 18 As shown, this embodiment provides a three-dimensional house type generation device, which can execute the above Figure 2 shown three-dimensional house type generation method. Specifically, the generation device may include:

[0295] A first acquisition module 11, configured to acquire at least two images of different perspectives of the room;

[0296] A first generation module 12, configured to generate three-dimensional layout diagrams respectively corresponding to the at least two images;

[0297] A first determination module 13, configured to determine a splicing position corresponding to the three-dimensional layout diagram based on the at least two images;

[0298] A first processing module 14, configured to perform splicing processing on the three-dimensional layout diagram according to the splicing position to generate a three-dimensional house type corresponding to the room.

[0299] In some instances, the shooting perspectives of the images are different, and there is an overlapping area between the shooting areas corresponding to the shooting perspectives of the at least two images.

[0300] In some instances, when the first generation module 12 generates three-dimensional layout diagrams respectively corresponding to the at least two images, the first generation module 12 is configured to execute: generating two-dimensional layout diagrams respectively corresponding to the at least two images; determining camera parameters corresponding to the at least two images; generating three-dimensional layout diagrams respectively corresponding to the at least two images based on the at least two images, the two-dimensional layout diagrams, and the camera parameters.

[0301] In some instances, the two-dimensional layout diagram includes wall surface information in the room.

[0302] In some instances, the camera parameters include at least one of the following: camera internal parameters, camera external parameters; when the first generation module 12 determines the camera parameters corresponding to the at least two images, the first generation module 12 is configured to execute: calculating vanishing point information corresponding to the at least two images; calculating camera internal parameters corresponding to the at least two images according to the vanishing point information; determining camera external parameters corresponding to the at least two images according to the vanishing point information and the camera internal parameters.

[0303] In some examples, when the first generation module 12 generates three-dimensional layout diagrams respectively corresponding to at least two images based on the at least two images, the two-dimensional layout diagram, and the camera parameters, the first generation module 12 is configured to perform: obtaining height information between the image capture positions corresponding to the at least two images and the ground; determining spatial constraint relationships corresponding to the pixel points in the at least two images based on the height information, the at least two images, and the camera parameters; and performing three-dimensional reconstruction operations on the pixel points in the two-dimensional layout diagram based on the spatial constraint relationships to generate three-dimensional layout diagrams.

[0304] In some examples, when the first determination module 13 determines the splicing positions corresponding to the three-dimensional layout diagrams based on the at least two images, the first determination module 13 is configured to perform: extracting wall surface information included in each three-dimensional layout diagram; generating two-dimensional wall surface images corresponding to the wall surface information; and determining the splicing positions corresponding to the three-dimensional layout diagrams based on the two-dimensional wall surface images corresponding to all the wall surface information and the at least two images.

[0305] In some examples, when the first determination module 13 generates two-dimensional wall surface images corresponding to the wall surface information, the first determination module 13 is configured to perform: obtaining constraint relationships for generating the two-dimensional wall surface images; and generating two-dimensional wall surface images corresponding to the wall surface information based on the constraint relationships and the three-dimensional layout diagrams.

[0306] In some examples, the constraint relationships include: the area of the region corresponding to the wall surface information is positively correlated with the image resolution of the two-dimensional wall surface image.

[0307] In some examples, when the first determination module 13 determines the splicing positions corresponding to the three-dimensional layout diagrams based on the two-dimensional wall surface images corresponding to all the wall surface information and the at least two images, the first determination module 13 is configured to perform: obtaining the image similarity corresponding to any two of the two-dimensional wall surface images; determining the image adjacency relationships of at least two three-dimensional layout diagrams based on the at least two images; and determining the splicing positions corresponding to two adjacent three-dimensional layout diagrams based on the image similarity.

[0308] In some examples, when the first determination module 13 determines the splicing positions corresponding to two adjacent three-dimensional layout diagrams based on the image similarity, the first determination module 13 is configured to perform: obtaining a first splicing wall surface pair corresponding to the highest image similarity; and determining the splicing positions corresponding to the two adjacent three-dimensional layout diagrams based on the first splicing wall surface pair.

[0309] In some examples, when the first determination module 13 determines the splicing positions corresponding to two adjacent three-dimensional layout diagrams based on the first splicing wall surface, the first determination module 13 is configured to perform: performing a feature extraction operation on the first splicing wall surface to obtain a first wall surface feature and a second wall surface feature; determining at least one splicing position corresponding to two adjacent three-dimensional layout diagrams based on the first wall surface feature and the second wall surface feature.

[0310] In some examples, when the first determination module 13 determines the image adjacency relationship of at least two three-dimensional layout diagrams based on at least two images, the first determination module 13 is configured to perform: determining a first adjacency relationship corresponding to the at least two images; determining the image adjacency relationship of the at least two three-dimensional layout diagrams based on the first adjacency relationship.

[0311] In some examples, after determining the first adjacency relationship corresponding to the at least two images, the first processing module 14 in this embodiment is configured to perform: performing a scaling process on the at least two images so that the wall surface heights included in all the images are the same.

[0312] In some examples, after determining the splicing positions corresponding to two adjacent three-dimensional layout diagrams, the first processing module 14 in this embodiment is configured to perform: detecting whether the splicing positions are reasonable; when the splicing positions are reasonable, performing a splicing process on the three-dimensional layout diagrams based on the splicing positions; or, when the splicing positions are unreasonable, obtaining a second splicing wall surface corresponding to the second highest image similarity and determining the splicing positions corresponding to the second splicing wall surface.

[0313] In some examples, when the first processing module 14 detects whether the splicing positions are reasonable, the first processing module 14 is configured to perform the following steps: performing a pre-splicing on the three-dimensional layout diagrams based on the splicing positions to obtain a pre-spliced house type; identifying the wall surface features corresponding to each wall surface in the pre-spliced house type; detecting whether the splicing positions are reasonable based on the wall surface features corresponding to each wall surface.

[0314] In some examples, the wall surface features include: the size features of the wall surface; when the first processing module 14 detects whether the splicing positions are reasonable based on the wall surface features corresponding to each wall surface, the first processing module 14 is configured to perform the following steps: determining the size deviation information between any two wall surfaces in the pre-spliced house type based on the size features corresponding to each wall surface; when the size deviation information is less than a set value, determining that the splicing positions are reasonable; or, when the size deviation information is greater than or equal to the set value, determining that the splicing positions are unreasonable.

[0315] In some instances, the wall features include: the relative position features between the walls; when the first processing module 14 detects whether the splicing position is reasonable based on the wall features corresponding to each wall, the first processing module 14 is configured to perform the following steps: detect whether there is wall intersection or wall occlusion between any two walls in the pre-spliced room type based on the relative position features between the walls; when there is wall intersection or wall occlusion, determine that the splicing position is unreasonable; or, when there is no wall intersection and no wall occlusion, determine that the splicing position is reasonable.

[0316] In some instances, when the first processing module 14 performs splicing processing on the three-dimensional layout diagrams according to the splicing position to generate a three-dimensional room type corresponding to the room, the first processing module 14 is configured to perform the following steps: perform splicing processing on all the three-dimensional layout diagrams based on the splicing position to obtain the spliced room type data corresponding to the room; detect whether there are gaps at all the splicing positions in the spliced room type data; when there are no gaps at all the splicing positions in the spliced room type data, determine the spliced room type data as the three-dimensional room type; or, when there is a gap at one splicing position in the spliced room type data, perform optimization processing on the spliced room type data to generate the three-dimensional room type.

[0317] In some instances, when the first processing module 14 performs optimization processing on the spliced room type data to generate the three-dimensional room type, the first processing module 14 is configured to perform the following steps: optimize all the splicing positions based on the existence of a gap at one splicing position in the spliced room type data to obtain the optimized splicing positions; perform splicing processing on all the three-dimensional layout diagrams based on the optimized splicing positions to generate the three-dimensional room type corresponding to the room.

[0318] In some instances, when the first processing module 14 optimizes all the splicing positions based on the existence of a gap at one splicing position in the spliced room type data to obtain the optimized splicing positions, the first processing module 14 is configured to perform the following steps: obtain the splicing matching degree corresponding to each splicing position for the three-dimensional layout diagram; determine the two three-dimensional layout diagrams corresponding to the highest splicing matching degree as the reference image pair; adjust the splicing positions corresponding to the three-dimensional layout diagrams based on the reference image pair and the gap to obtain the optimized splicing positions.

[0319] In some examples, the reference image pair includes: a first image pair in a first direction and a second image pair in a second direction; when the first processing module 14 adjusts the splicing position corresponding to the three-dimensional layout diagram based on the reference diagram and the gap to obtain the optimized splicing position, the first processing module 14 is configured to perform the following steps: determining a first adjustment distance of the splicing position in the first direction based on the first image pair and the gap; determining a second adjustment distance of the splicing position in the second direction based on the second image pair and the gap; respectively adjusting the splicing position in the first direction and the second direction based on the first adjustment distance and the second adjustment distance to obtain the optimized splicing position.

[0320] In some examples, the first processing module 14 in this embodiment is configured to perform: identifying the entities included in the room and the entity feature information; performing a fusion process on the entities, the entity feature information, and the three-dimensional room type to generate a target three-dimensional room type.

[0321] In some examples, after generating the three-dimensional room type corresponding to the room, the first processing module 14 in this embodiment is further configured to: generate a construction room type corresponding to the three-dimensional room type; obtaining construction verification information according to the three-dimensional room type and the construction room type.

[0322] In some examples, after generating the three-dimensional room type corresponding to the room, the first processing module 14 in this embodiment is further configured to: obtain a display request for the three-dimensional room type; based on the display request, and using a set device to display the three-dimensional room type.

[0323] In some examples, the display request includes at least one of the following: an augmented reality display request, a virtual reality display request, a mixed reality display request, an image reality display request; correspondingly, the set device includes at least one of the following: an augmented reality device, a virtual reality device, a mixed reality device, an image reality device.

[0324] In some examples, after generating the three-dimensional room type corresponding to the room, the first processing module 14 in this embodiment is further configured to: obtain the generation quality of the three-dimensional room type; when the generation quality does not meet the set conditions, generate an image reshooting request to re-obtain at least two images of different perspectives of the room based on the image reshooting request.

[0325] Figure 18 The device shown can execute Figure 1 - Figure 15 the method of the embodiment shown. For parts not described in detail in this embodiment, reference may be made to the relevant description of Figure 1 - Figure 15 the embodiment shown. The execution process and technical effects of this technical solution are referred to Figure 1 - Figure 15 the description in the embodiment shown, and will not be elaborated here.

[0326] In a possible design, Figure 18 The structure of the generating device for the three-dimensional house type shown can be implemented as an electronic device, and this electronic device can be various devices such as a mobile phone, a tablet computer, a server, etc. As Figure 19 shown, this electronic device may include: a first processor 21 and a first memory 22. Among them, the first memory 22 is used to store a program for the corresponding electronic device to execute the method for generating the three-dimensional house type provided in the above Figure 1 - Figure 15 shown embodiment, and the first processor 21 is configured to execute the program stored in the first memory 22.

[0327] The program includes one or more computer instructions. Among them, when the one or more computer instructions are executed by the first processor 21, the following steps can be implemented:

[0328] Obtain at least two images of different perspectives of the room;

[0329] Generate three-dimensional layout diagrams respectively corresponding to the at least two images;

[0330] Based on the at least two images, determine the splicing positions corresponding to the three-dimensional layout diagrams;

[0331] Perform splicing processing on the at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional house type corresponding to the room.

[0332] Furthermore, the first processor 21 is also used to execute all or part of the steps in the foregoing Figure 1 - Figure 15 shown embodiment.

[0333] Among them, the structure of the electronic device may further include a first communication interface 23 for the electronic device to communicate with other devices or communication networks.

[0334] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by an electronic device, which includes a program involved in executing the method for generating a three-dimensional house type in the above Figure 1 - Figure 15 shown method embodiment.

[0335] Figure 20 FIG. is a schematic structural diagram of a generating device for a three-dimensional house type provided in an embodiment of the present application; referring to the attached Figure 20 shown, this embodiment provides a generating device for a three-dimensional house type, and this generating device for a three-dimensional house type is used to execute the method for generating a three-dimensional house type shown in the above Figure 16 shown, specifically, this generating device may include:

[0336] A second obtaining module 31, configured to obtain at least two images of different perspectives of the room.

[0337] The second generation module 32 is used to generate two-dimensional layout diagrams respectively corresponding to at least two images.

[0338] The second determination module 33 is used to determine camera parameters corresponding to at least two images.

[0339] The second processing module 34 is used to generate a three-dimensional house type corresponding to at least two images based on at least two images, the two-dimensional layout diagram, and the camera parameters.

[0340] In some instances, the shooting perspectives of the images are different, and there is an overlapping area between the shooting areas corresponding to the shooting perspectives of at least two images.

[0341] In some instances, the two-dimensional layout diagram includes wall information in the room.

[0342] In some instances, when the second determination module 33 determines the camera parameters corresponding to at least two images, the second determination module 33 is used to perform: calculating vanishing point information corresponding to at least two images; calculating the camera internal parameters corresponding to at least two images according to the vanishing point information; and determining the camera external parameters corresponding to at least two images according to the vanishing point information and the camera internal parameters.

[0343] In some instances, when the second processing module 34 generates a three-dimensional house type corresponding to at least two images based on at least two images, the two-dimensional layout diagram, and the camera parameters, the second processing module 34 is used to perform: obtaining height information between the image shooting positions corresponding to at least two images and the ground; determining spatial constraint relationships corresponding to the pixel points in at least two images based on the height information, at least two images, and the camera parameters; and performing a three-dimensional reconstruction operation on the pixel points in the two-dimensional layout diagram based on the spatial constraint relationships to generate a three-dimensional house type corresponding to at least two images.

[0344] Figure 20 The device shown can execute Figure 1 、 Figure 12 - Figure 14 、 Figure 16 the methods of the embodiments shown. For parts not described in detail in this embodiment, reference can be made to the relevant descriptions of Figure 1 、 Figure 12 - Figure 14 、 Figure 16 the embodiments shown. For the execution process and technical effects of this technical solution, refer to the descriptions in Figure 1 、 Figure 12 - Figure 14 、 Figure 16 the embodiments shown, which will not be elaborated here.

[0345] In a possible implementation, Figure 20 the structure of the three-dimensional house type generation device shown can be implemented as an electronic device, and this electronic device can be various devices such as a mobile phone, a tablet computer, a server, etc. As Figure 21As shown, the electronic device may include: a second processor 41 and a second memory 42. Among them, the second memory 43 is used to store the program for the corresponding electronic device to execute the three-dimensional house type generation method provided in the above Figure 1 , Figure 12 - Figure 14 , Figure 16 illustrated embodiments. The second processor 41 is configured to execute the program stored in the second memory 42.

[0346] The program includes one or more computer instructions. Among them, when the one or more computer instructions are executed by the second processor 41, the following steps can be implemented:

[0347] Obtain at least two images of different perspectives of the room;

[0348] Generate two-dimensional layout diagrams respectively corresponding to the at least two images;

[0349] Determine the camera parameters corresponding to the at least two images;

[0350] Based on the at least two images, the two-dimensional layout diagrams and the camera parameters, generate a three-dimensional house type corresponding to the at least two images.

[0351] Optionally, the second processor 41 is further configured to execute all or part of the steps in the foregoing Figure 1 , Figure 12 - Figure 14 , Figure 16 illustrated embodiments.

[0352] Among them, the structure of the electronic device may further include a second communication interface 43 for the terminal to communicate with other devices or communication networks.

[0353] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the electronic device, which includes a program related to the three-dimensional house type generation method in the method embodiments described above Figure 1 , Figure 12 - Figure 14 , Figure 16 illustrated.

[0354] Figure 22 is a schematic structural diagram of a three-dimensional house type generation device provided in an embodiment of the present application; referring to the attached Figure 22 shown, this embodiment provides a three-dimensional house type generation device, which can execute the three-dimensional house type generation method corresponding to the above Figure 17 . Specifically, the generation device may include:

[0355] A third acquisition module 51 for acquiring at least two three-dimensional layout diagrams of different perspectives of the room;

[0356] A third determination module 52 for determining the splicing positions corresponding to the at least two three-dimensional layout diagrams;

[0357] The third processing module 53 is configured to splice at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional house type corresponding to the room.

[0358] In some examples, when the third processing module 53 splices at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional house type corresponding to the room, the third processing module 53 is configured to perform: splicing all the three-dimensional layout diagrams based on the splicing positions to obtain spliced house type data corresponding to the room; detecting whether there are gaps at all the splicing positions in the spliced house type data; when there are no gaps at all the splicing positions in the spliced house type data, determining the spliced house type data as the three-dimensional house type; or, when there is a gap at a splicing position in the spliced house type data, performing optimization processing on the spliced house type data to generate the three-dimensional house type.

[0359] In some examples, when the third processing module 53 performs optimization processing on the spliced house type data to generate the three-dimensional house type, the third processing module 53 is configured to perform: optimizing all the splicing positions based on the existence of a gap at a splicing position in the spliced house type data to obtain optimized splicing positions; splicing all the three-dimensional layout diagrams based on the optimized splicing positions to generate a three-dimensional house type corresponding to the room.

[0360] In some examples, when the third processing module 53 optimizes the splicing positions based on the gaps existing at the splicing positions to obtain optimized splicing positions, the third processing module 53 is configured to perform: obtaining the splicing matching degrees corresponding to each splicing position for at least two three-dimensional layout diagrams; determining the two three-dimensional layout diagrams corresponding to the highest splicing matching degree as a reference image pair; adjusting the splicing positions corresponding to at least two three-dimensional layout diagrams based on the reference image pair and the gaps to obtain optimized splicing positions.

[0361] In some examples, the reference image pair includes: a first image pair in a first direction and a second image pair in a second direction; when the third processing module 53 adjusts the splicing positions corresponding to at least two three-dimensional layout diagrams based on the reference diagram and the gaps to obtain optimized splicing positions, the third processing module 53 is configured to perform: determining a first adjustment distance of the splicing position in the first direction based on the first image pair and the gaps; determining a second adjustment distance of the splicing position in the second direction based on the second image pair and the gaps; respectively adjusting the splicing position in the first direction and the second direction based on the first adjustment distance and the second adjustment distance to obtain optimized splicing positions.

[0362] Figure 22 The shown device can execute Figure 1 、 Figure 12 - Figure 14 、 Figure 17For the method of the illustrated embodiment, for parts not described in detail in this embodiment, reference may be made to the relevant descriptions of the embodiments shown in Figure 1 , Figure 12 - Figure 14 , Figure 17 . For the execution process and technical effects of this technical solution, refer to the descriptions in the embodiments shown in Figure 1 , Figure 12 - Figure 14 , Figure 17 . Details are not repeated herein.

[0363] In a possible implementation, Figure 22 the structure of the generating device for the three-dimensional house type shown can be implemented as an electronic device, which can be various devices such as a mobile phone, a tablet computer, a server, etc. As shown in Figure 23 , the electronic device may include: a third processor 61 and a third memory 62. Among them, the third memory 63 is used to store a program for the corresponding electronic device to execute the method for generating the three-dimensional house type provided in the embodiment shown in Figure 17 , and the third processor 61 is configured to execute the program stored in the third memory 62.

[0364] The program includes one or more computer instructions. When one or more computer instructions are executed by the third processor 61, the following steps can be implemented:

[0365] Obtain at least two three-dimensional layout diagrams of different perspectives of the room;

[0366] Determine the splicing positions corresponding to the at least two three-dimensional layout diagrams;

[0367] Perform splicing processing on the at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional house type corresponding to the room.

[0368] Optionally, the third processor 61 is further configured to execute all or part of the steps in the foregoing embodiment shown in Figure 17 .

[0369] Among them, the structure of the electronic device may further include a third communication interface 63, which is used for the terminal to communicate with other devices or communication networks.

[0370] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the electronic device, which includes a program involved in the method for generating the three-dimensional house type in the foregoing method embodiment shown in Figure 17 .

[0371] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0372] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of adding a necessary general hardware platform, and of course, it can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a computer product. This application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0373] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable devices generate a device for realizing the functions specified in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0374] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device realizes the functions specified in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0375] These computer program instructions can also be loaded onto a computer or other programmable device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for realizing the functions specified in one Figure 1 one process or multiple processes and / or blocks Figure 1Steps of the functions specified in one or more boxes.

[0376] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0377] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

[0378] Computer-readable media includes both permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0379] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a three-dimensional house type, characterized in that, Including: Obtain at least two images of different perspectives of a room, wherein the sum of the shooting perspective ranges corresponding to the at least two images is greater than 360°; Generate three-dimensional layout diagrams respectively corresponding to the at least two images; Based on the at least two images, determine the splicing positions corresponding to the three-dimensional layout diagrams; Perform splicing processing on the three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional room type corresponding to the room; Wherein, generating three-dimensional layout diagrams respectively corresponding to the at least two images includes: Generate two-dimensional layout diagrams respectively corresponding to the at least two images; Determine camera parameters corresponding to the at least two images, where the camera parameters include at least one of the following: a point where the two-dimensional projections of the camera converge and intersect, a rotation matrix and / or a translation matrix between the world coordinate system and the camera coordinate system; Based on the at least two images, the two-dimensional layout diagrams, and the camera parameters, generate three-dimensional layout diagrams respectively corresponding to the at least two images.

2. The method according to claim 1, wherein Based on the at least two images, the two-dimensional layout diagrams, and the camera parameters, generating three-dimensional layout diagrams respectively corresponding to the at least two images includes: Obtain the height information between the image shooting positions corresponding to the at least two images and the ground; Based on the height information, the at least two images, and the camera parameters, determine the spatial constraint relationships corresponding to the pixel points in the at least two images; Perform three-dimensional reconstruction operations on the pixel points in the two-dimensional layout diagrams based on the spatial constraint relationships to generate the three-dimensional layout diagrams.

3. The method according to claim 1, wherein Based on the at least two images, determining the splicing positions corresponding to the three-dimensional layout diagrams includes: Extract the wall surface information included in each three-dimensional layout diagram; Generate two-dimensional wall surface images corresponding to the wall surface information; Based on the two-dimensional wall surface images corresponding to all the wall surface information and the at least two images, determine the splicing positions corresponding to the three-dimensional layout diagrams.

4. The method according to claim 3, wherein Generating two-dimensional wall surface images corresponding to the wall surface information includes: Obtain the constraint relationships for generating the two-dimensional wall surface images; Based on the constraint relationships and the three-dimensional layout diagrams, generate two-dimensional wall surface images corresponding to the wall surface information.

5. The method according to claim 3, wherein Based on the two-dimensional wall surface images corresponding to all the wall surface information and the at least two images, determining the splicing positions corresponding to the three-dimensional layout diagrams includes: Obtain the image similarity corresponding to any two two-dimensional wall surface images; Based on the at least two images, determine the image adjacent relationships of the at least two three-dimensional layout diagrams; Based on the image similarity, determine the splicing positions corresponding to two adjacent three-dimensional layout diagrams.

6. The method according to claim 5, wherein Based on the image similarity, determining the splicing positions corresponding to two adjacent three-dimensional layout diagrams includes: Obtain the first splicing wall surface pair corresponding to the highest image similarity; Based on the first splicing wall surface pair, determine the splicing positions corresponding to two adjacent three-dimensional layout diagrams.

7. The method according to claim 6, characterized in that, Based on the first splicing wall surface pair, determining the splicing positions corresponding to two adjacent three-dimensional layout diagrams includes: Perform feature extraction operations on the first splicing wall surface pair to obtain a first wall surface feature and a second wall surface feature; Based on the first wall feature and the second wall feature, determine at least one splicing position corresponding to two adjacent three-dimensional layout diagrams.

8. The method according to claim 6, characterized in that, Based on the at least two images, determine the image adjacency relationship of the at least two three-dimensional layout diagrams, including: Determine a first adjacency relationship corresponding to the at least two images; Based on the first adjacency relationship, determine the image adjacency relationship of the at least two three-dimensional layout diagrams.

9. The method according to claim 5, characterized in that After determining the splicing position corresponding to two adjacent three-dimensional layout diagrams, the method further includes: Detect whether the splicing position is reasonable; When the splicing position is reasonable, perform splicing processing on the three-dimensional layout diagram based on the splicing position; or, When the splicing position is unreasonable, obtain a second splicing wall surface corresponding to the second highest image similarity, and determine a splicing position corresponding to the second splicing wall surface.

10. The method according to claim 9, characterized in that, Detecting whether the splicing position is reasonable includes: Perform pre-splicing on the three-dimensional layout diagram based on the splicing position to obtain a pre-spliced room type; Identify the wall features corresponding to each wall in the pre-spliced room type; Based on the wall features corresponding to each wall, detect whether the splicing position is reasonable.

11. The method according to claim 10, wherein The wall features include: the size features of the wall; based on the wall features corresponding to each wall, detecting whether the splicing position is reasonable includes: Based on the size features corresponding to each wall, determine the size deviation information between any two walls in the pre-spliced room type; When the size deviation information is less than a set value, determine that the splicing position is reasonable; or, When the size deviation information is greater than or equal to the set value, determine that the splicing position is unreasonable.

12. The method according to claim 10, wherein The wall features include: the relative position features between walls; based on the wall features corresponding to each wall, detecting whether the splicing position is reasonable includes: Based on the relative position features between each wall, detect whether there is wall intersection or wall occlusion between any two walls in the pre-spliced room type; When there is wall intersection or wall occlusion, determine that the splicing position is unreasonable; or, When there is no wall intersection and no wall occlusion, determine that the splicing position is reasonable.

13. The method according to claim 1, characterized in that, Perform splicing processing on the three-dimensional layout diagram according to the splicing position to generate a three-dimensional room type corresponding to the room, including: Perform splicing processing on all the three-dimensional layout diagrams based on the splicing position to obtain spliced room type data corresponding to the room; Detect whether there are gaps at all the splicing positions in the spliced room type data; When there are no gaps at all the splicing positions in the spliced room type data, determine the spliced room type data as the three-dimensional room type; or, When there is a gap at a splicing position in the spliced room type data, perform optimization processing on the spliced room type data to generate the three-dimensional room type.

14. The method according to claim 13, wherein Performing optimization processing on the spliced room type data to generate the three-dimensional room type includes: Based on the existence of a gap at a splicing position in the spliced room type data, optimize all the splicing positions to obtain optimized splicing positions; Perform splicing processing on all three-dimensional layout diagrams based on the optimized splicing positions to generate the three-dimensional house type corresponding to the room.

15. The method according to claim 14, wherein Optimize all splicing positions based on the existence of a gap at a splicing position in the spliced house type data to obtain optimized splicing positions, including: Obtain the splicing matching degree corresponding to each splicing position of the three-dimensional layout diagram; Determine the two three-dimensional layout diagrams corresponding to the highest splicing matching degree as the reference image pair; Adjust the splicing positions corresponding to the three-dimensional layout diagrams based on the reference image pair and the gap to obtain the optimized splicing positions.

16. The method according to claim 15, wherein The reference image pair includes: a first image pair in the first direction and a second image pair in the second direction; adjusting the splicing positions corresponding to the three-dimensional layout diagrams based on the reference diagram and the gap to obtain the optimized splicing positions includes: Determine the first adjustment distance of the splicing position in the first direction based on the first image pair and the gap; Determine the second adjustment distance of the splicing position in the second direction based on the second image pair and the gap; Adjust the splicing position in the first direction and the second direction respectively based on the first adjustment distance and the second adjustment distance to obtain the optimized splicing position.

17. The method according to any one of claims 1 to 16, characterized in that, After generating the three-dimensional house type corresponding to the room, the method further includes: Generate a construction house type corresponding to the three-dimensional house type; Obtain construction verification information according to the three-dimensional house type and the construction house type.

18. The method according to any one of claims 1 to 16, characterized in that, After generating the three-dimensional house type corresponding to the room, the method further includes: Obtain a display request for the three-dimensional house type; Based on the display request, display the three-dimensional house type using a set device.

19. The method according to claim 18, characterized in that, The display request includes at least one of the following: an augmented reality display request, a virtual reality display request, a mixed reality display request, an image reality display request; Correspondingly, the set device includes at least one of the following: an augmented reality device, a virtual reality device, a mixed reality device, an image reality device.

20. The method according to any one of claims 1-16, characterized in that, After generating the three-dimensional house type corresponding to the room, the method further includes: Obtain the generation quality of the three-dimensional house type; When the generation quality does not meet the set conditions, generate an image reshooting request to re-obtain at least two images of different perspectives of the room based on the image reshooting request.

21. A method for generating a three-dimensional house type, characterized in that, Include: Obtain at least two images of different perspectives of the room, where the total shooting perspective range corresponding to the at least two images is greater than 360°; Generate two-dimensional layout diagrams corresponding to the at least two images respectively; Determine the camera parameters corresponding to the at least two images, where the camera parameters include at least one of the following: a point where the two-dimensional projection of the camera converges and intersects, a rotation matrix and / or a translation matrix between the world coordinate system and the camera coordinate system; Generate a three-dimensional house type corresponding to the at least two images based on the at least two images, the two-dimensional layout diagrams, and the camera parameters.

22. A method for generating a three-dimensional house type, characterized in that, Include: Obtain at least two three-dimensional layout diagrams of different perspectives of the room, where the total shooting perspective range corresponding to the at least two three-dimensional layout diagrams is greater than 360°; Determine the splicing positions corresponding to the at least two three-dimensional layout diagrams; Perform splicing processing on the at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional room type corresponding to the room; Among them, obtaining at least two three-dimensional layout diagrams of different perspectives of the room includes: Generate two-dimensional layout diagrams respectively corresponding to different perspectives of the room; Determine the camera parameters corresponding to different perspectives of the room, where the camera parameters include at least one of the following: a point where the two-dimensional projections of the camera converge and intersect during the projection process, a rotation matrix and / or a translation matrix between the world coordinate system and the camera coordinate system; Based on different perspectives of the room, the two-dimensional layout diagrams, and the camera parameters, generate three-dimensional layout diagrams respectively corresponding to different perspectives of the room.

23. A generating device for a three-dimensional house type, characterized in that, Include: A first acquisition module, configured to acquire at least two images of different perspectives of the room, where the total shooting perspective range corresponding to the at least two images is greater than 360°; A first generation module, configured to generate three-dimensional layout diagrams respectively corresponding to the at least two images; A first determination module, configured to determine the splicing positions corresponding to the three-dimensional layout diagrams based on the at least two images; A first processing module, configured to perform splicing processing on the three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional room type corresponding to the room; The first generation module is further configured to: generate three-dimensional layout diagrams respectively corresponding to the at least two images, including: generate two-dimensional layout diagrams respectively corresponding to the at least two images; determine the camera parameters corresponding to the at least two images, where the camera parameters include at least one of the following: a point where the two-dimensional projections of the camera converge and intersect during the projection process, a rotation matrix and / or a translation matrix between the world coordinate system and the camera coordinate system; based on the at least two images, the two-dimensional layout diagrams, and the camera parameters, generate three-dimensional layout diagrams respectively corresponding to the at least two images.

24. An electronic device, characterized in that, Include: A memory and a processor; wherein, the memory is used to store one or more computer instructions, and when the one or more computer instructions are executed by the processor, the method for generating a three-dimensional room type according to any one of claims 1 to 20 is implemented.

25. A generating device for a three-dimensional house type, characterized in that, Include: A second acquisition module, configured to acquire at least two images of different perspectives of the room, where the total shooting perspective range corresponding to the at least two images is greater than 360°; A second generation module, configured to generate two-dimensional layout diagrams respectively corresponding to the at least two images; A second determination module, configured to determine the camera parameters corresponding to the at least two images, where the camera parameters include at least one of the following: a point where the two-dimensional projections of the camera converge and intersect during the projection process, a rotation matrix and / or a translation matrix between the world coordinate system and the camera coordinate system; A second processing module, configured to generate a three-dimensional room type corresponding to the at least two images based on the at least two images, the two-dimensional layout diagrams, and the camera parameters.

26. An electronic device, characterized in that, Include: A memory and a processor; wherein, the memory is used to store one or more computer instructions, and when the one or more computer instructions are executed by the processor, the method for generating a three-dimensional room type according to claim 21 is implemented.

27. A generating device for a three-dimensional house type, characterized in that, Including: A third acquisition module, configured to acquire at least two three-dimensional layout diagrams of different perspectives of a room, wherein the sum of the shooting perspective ranges corresponding to the at least two three-dimensional layout diagrams is greater than 360°; A third determination module, configured to determine splicing positions corresponding to the at least two three-dimensional layout diagrams; A third processing module, configured to perform splicing processing on the at least two three-dimensional layout diagrams according to the splicing positions to generate a three-dimensional room type corresponding to the room; The third acquisition module is further configured to: acquire at least two three-dimensional layout diagrams of different perspectives of a room, including: generating two-dimensional layout diagrams respectively corresponding to different perspectives of the room; determining camera parameters corresponding to different perspectives of the room, where the camera parameters include at least one of the following: a point where the two-dimensional projections of the camera converge during the projection process, a rotation matrix and / or a translation matrix between the world coordinate system and the camera coordinate system; generating three-dimensional layout diagrams respectively corresponding to different perspectives of the room based on the different perspectives of the room, the two-dimensional layout diagrams, and the camera parameters.

28. An electronic device, characterized in that, Including: A memory and a processor; wherein the memory is configured to store one or more computer instructions, and when the one or more computer instructions are executed by the processor, the method for generating a three-dimensional room type as described in claim 22 is implemented.

Images