Floor plan rendering method, device and product based on image recognition

Abstract

The present disclosure provides a floor plan rendering method, device, and product based on image recognition. The method includes: obtaining a floor plan and converting the floor plan into a pixel array; loading a Wasm module containing a floor plan feature extraction submodule and a topology relationship reasoning submodule in a Web editor; passing the pixel array to the Wasm module and extracting floor plan features based on the floor plan feature extraction submodule to obtain floor plan features of the floor plan; performing topology relationship reasoning on the floor plan features using the topology relationship reasoning submodule of the Wasm module to obtain a floor plan recognition result, wherein the floor plan recognition result includes: a room module and a floor plan element module having a topological relationship, wherein the topological relationship includes at least one of a topological relationship between different rooms, a topological relationship between a room and a floor plan element, and a topological relationship between different floor plan elements; rendering is performed based on the floor plan recognition result to obtain a rendered target floor plan.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a floor plan rendering method, device, and product based on image recognition. Background Art

[0002] With the development of computer technology, the use of computer technology for image recognition and image rendering is becoming more and more mature.

[0003] In existing technologies, users input images to be processed through a front-end (client device), which then sends them to a back-end server. The back-end server then performs image recognition and rendering. This back-and-forth image transmission process is time-consuming, and especially when the image data volume is large, image recognition and rendering efficiency is relatively low. For example, in real estate applications, users often require floor plan recognition and rendering. Leaving this processing to the server severely impacts efficiency. Summary of the Invention

[0004] The present disclosure provides a floor plan rendering method, device, and product based on image recognition.

[0005] According to the first aspect of the present disclosure, a method for rendering a floor plan based on image recognition is provided. The method specifically includes: obtaining a floor plan and converting the floor plan into a pixel array; loading a Wasm module containing a floor plan feature extraction submodule and a topological relationship reasoning submodule in a Web editor; passing the pixel array to the Wasm module and extracting floor plan features based on the floor plan feature extraction submodule to obtain floor plan features of the floor plan; performing topological relationship reasoning on the floor plan features through the topological relationship reasoning submodule of the Wasm module to obtain a floor plan recognition result, the floor plan recognition result including: a room module and a floor plan element module with a topological relationship, the topological relationship including at least one of a topological relationship between different rooms, a topological relationship between a room and a floor plan element, and a topological relationship between different floor plan elements; rendering based on the floor plan recognition result to obtain a rendered target floor plan.

[0006] Based on the above, it can be seen that during the floor plan recognition process, the Wasm module is loaded into the Web editor. The floor plan to be recognized is converted into a pixel array, and after the pixel array is passed to the Wasm module, the floor plan features are extracted. After the floor plan features are extracted, the Wasm module further performs topological relationship reasoning, and finally obtains the floor plan recognition result. The floor plan recognition result includes room modules and floor plan element modules with topological relationships. Through the above solution, by loading the Wasm module with efficient image processing capabilities into the Web editor, efficient floor plan recognition and accurate reasoning of room modules and floor plan element modules in the floor plan can be achieved.

[0007] According to at least one embodiment of the present disclosure, a pixel array is passed to a Wasm module and apartment features are extracted based on an apartment feature extraction submodule to obtain apartment features of a floor plan, including: calling a Wasm module pre-set in a Web editor; wherein the Wasm module includes a feature extraction submodule and an element recognition submodule; utilizing the element recognition submodule to identify apartment elements from a pixel array; wherein the apartment elements include walls, windows, and doors; utilizing the feature extraction submodule to extract apartment features including element features and room features from the apartment elements.

[0008] According to at least one embodiment of the present disclosure, the topological relationship reasoning submodule of the Wasm module is used to perform topological relationship reasoning on the apartment type features to obtain the apartment type recognition result, including: performing topological relationship reasoning on the element features and the room features through the topological relationship reasoning submodule to generate a room module and an apartment type element module; wherein the room module includes the room size, the topological relationship between the room and the apartment type elements, and the topological relationship between different rooms; the apartment type element module includes: the wall size, different wall topological relationships, the wall and room topological relationship, the wall and window topological relationship, and the wall and door topological relationship; the window size, different window topological relationships, the window and room topological relationship, the window and wall topological relationship, and the window and door topological relationship, the door size, different door topological relationships, the door and room topological relationship, the door and window topological relationship, and the door and wall topological relationship.

[0009] According to at least one embodiment of the present disclosure, element features and room features are calculated and processed by a topological relationship reasoning submodule to generate room modules and apartment element modules, including: constructing a topological relationship graph using element features and room features; analyzing the topological relationship graph by a topological relationship reasoning submodule to determine the topological relationship between room and apartment elements; determining dimension statistics results using element features and room features; wherein the dimension statistics results include: room dimensions, wall dimensions, window dimensions, and door dimensions; and constructing room modules and apartment element modules based on the topological relationship analysis results and the dimension statistics results.

[0010] According to at least one embodiment of the present disclosure, after calculating and processing the apartment type features through the Wasm module to obtain the apartment type recognition result, it also includes: determining a reference room module with significant features in the room module; wherein the reference room module includes a bathroom module and a kitchen module; and correcting the apartment type recognition result based on the reference room module, topological relationship and correction rules.

[0011] According to at least one embodiment of the present disclosure, the apartment type recognition result is corrected based on the reference room module and the topological relationship, including: when the reference room module is a bathroom module, determining that the bathroom module is connected to the target room module; judging whether the target room module is the largest room in the apartment plan according to the topological relationship between different rooms; or, judging whether the target room module is located in the center of the apartment plan and has multiple doors facing other room modules in the same direction according to the topological relationship between different rooms; or, judging whether the target room module is connected to the kitchen module according to the topological relationship between different rooms; if so, the target room module is the living room; if not, the target room module is the bedroom.

[0012] According to at least one embodiment of the present disclosure, rendering is performed based on the apartment type recognition result to obtain a rendered target apartment type diagram, including: a Web editor calls a WebGL interface, and renders furniture decoration according to the room size and room type corresponding to the room module corresponding to the apartment type recognition result to obtain a rendered target apartment type diagram.

[0013] According to at least one embodiment of the present disclosure, it also includes: responding to a user's request to modify the apartment type recognition result, or responding to a user's request to modify the target apartment type diagram, determining the target room module and / or target apartment type element module corresponding to the modification request; obtaining the modification coordinates and modification type contained in the modification request; sending the modification coordinates and modification type to the Wasm module; and using the Wasm module to correct the target room module and / or target apartment type element module.

[0014] According to a second aspect of the present disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, so that the processor executes the method described in the first aspect of any embodiment of the present disclosure.

[0015] According to a third aspect of the present disclosure, a readable storage medium is provided, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the method described in the first aspect of any embodiment of the present disclosure.

[0016] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method according to the first aspect of any embodiment of the present disclosure is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0018] Figure 1 The present invention is a flowchart of a floor plan rendering method based on image recognition according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of a floor plan recognition result according to an embodiment of the present disclosure.

[0020] Figure 3 The figure is a flowchart of a method for correcting apartment type recognition results according to an embodiment of the present disclosure.

[0021] Figure 4 The present invention is a block diagram showing the structure of a floor plan rendering device based on image recognition according to an embodiment of the present invention.

[0022] Figure 5 The present invention is a block diagram showing the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present disclosure is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are intended only to illustrate the relevant content and are not intended to limit the present disclosure. It should also be noted that, for ease of description, only the portions relevant to the present disclosure are shown in the accompanying drawings.

[0024] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] In real estate floor plan recognition and rendering applications, floor plan recognition and rendering are frequently used and require processing of large amounts of floor plan data. Conventional technology typically sends the floor plans to be processed to a backend server. Backend servers possess significant computing power, which translates to superior image recognition and rendering capabilities. When users interact with the backend server using a web browser, frequent data exchange occurs between the two servers. For example, a user uploads a floor plan to be recognized and rendered to a web browser, which then sends the plan to the backend server. The backend server then uses a pre-defined algorithm to perform image recognition and rendering, returning the recognition and rendering results to the web browser. When large amounts of data are transmitted, data transmission is time-consuming, directly impacting the efficiency of floor plan recognition and rendering. Therefore, a simple and efficient solution for floor plan recognition and rendering is urgently needed.

[0026] For the convenience of description and to make the technical solutions of the specific embodiments of the present disclosure easier to understand, before describing the various embodiments of the present disclosure, the technical terms involved in the specific embodiments of the present disclosure are explained as follows.

[0027] Wasm: WebAssembly (Wasm) is a low-level, efficient bytecode format designed for browser environments. It aims to provide near-native computing capabilities while maintaining interoperability with JavaScript. Wasm is a compilation target that allows developers to write code in a variety of languages ​​(such as C, C++, and Rust), which is then compiled into Wasm modules and run in web browsers.

[0028] WebGL: A JavaScript API based on OpenGL ES for rendering 2D and 3D graphics in browsers. It allows developers to leverage GPU-accelerated drawing and rendering. WebGL provides low-level access to hardware acceleration, making it suitable for high-performance graphics rendering.

[0029] Figure 1 This is a flow chart of a floor plan rendering method based on image recognition provided by the present disclosure. Figure 1 The method shown includes steps 101 to 105. The method can be executed by an electronic device such as a server (a local server or a cloud server).

[0030] In step 101, a floor plan is obtained and converted into a pixel array. The floor plan here can be understood as a line drawing created by a user using graphics software (such as CAD software) or PowerPoint software. Different types of straight lines or line segments are used in this floor plan to represent different floor plan elements. For example, thick solid lines represent walls, while double thin lines represent windows.

[0031] After receiving the floor plan provided by the user, the web browser will further read the image file through JavaScript and draw it into a Canvas element to extract the pixel data. This pixel data is then organized into a pixel array. The pixel array can be received and processed by the Wasm module.

[0032] In step 102, a Wasm module containing a house type feature extraction submodule and a topology relationship reasoning submodule is loaded into the web editor. The Wasm module contains submodules that can implement various functions. These submodules cover a wide range of functions, including image preprocessing, image recognition, and image feature extraction.

[0033] During the actual development process, each functional algorithm is written in its own programming language. For example, complex numerical calculations can be written in efficient C++, while web page interaction logic can be written in JavaScript. Once written, this code undergoes a unified compilation process and is ultimately converted into a binary Wasm module. Wasm modules can be loaded smoothly into web browsers. Whether it's a desktop or mobile browser, as long as it supports Wasm technology, these Wasm modules can be easily loaded and run. Furthermore, once loaded, the Wasm module can efficiently perform the relevant computing tasks within the browser environment, significantly improving the performance and user experience of web applications.

[0034] In step 103, the pixel array is passed to the Wasm module and the apartment features are extracted based on the apartment feature extraction submodule to obtain the apartment features of the apartment plan.

[0035] In practical applications, after obtaining the pixel array in the manner described above, the pixel array of the floor plan can be further passed to the Wasm module. It should be noted that since the Wasm module is loaded in the Web browser, there is no need to send the pixel array to the server side, which is equivalent to extracting floor plan features locally in the browser. This not only ensures the security of data such as floor plan images, but also saves data transmission time, and can effectively improve the efficiency of floor plan recognition and rendering. The implementation scheme of floor plan feature extraction disclosed in this disclosure will be described in subsequent embodiments, so it will not be repeated here.

[0036] In step 104, the topological relationship reasoning submodule of the Wasm module performs topological relationship reasoning on the apartment features to obtain an apartment recognition result. The apartment recognition result includes: a room module and an apartment element module with a topological relationship. The topological relationship includes at least one of the topological relationship between different rooms, the topological relationship between rooms and apartment elements, and the topological relationship between different apartment elements. After extracting the apartment features in the manner described above, the Wasm module can be further used to perform topological relationship reasoning. In this topological relationship reasoning process, it is necessary to comprehensively consider the difficulty of identifying each room, the topological relationship between different rooms, the topological relationship between rooms and apartment elements, and the topological relationship between different apartment elements.

[0037] The difficulty of identifying rooms here can be understood as follows: bathrooms and kitchens each have their own unique identifiers. For example, bathrooms are small and typically have small amenities like toilets and showers marked on the floor plan. Kitchens are slightly larger than bathrooms, but significantly smaller than the living room and bedroom. Floor plans will also have stoves and other features, and they are adjacent to the dining room.

[0038] The topological relationships mentioned here can be understood as the connectivity and adjacency relationships between rooms, as well as the connectivity and adjacency relationships between floor plan elements (walls, windows, and doors). The room module and floor plan element module store their respective topological relationships. The topological relationships stored in the room module can be referred to as room topological relationships (including topological relationships between different rooms and between rooms and floor plan elements), while the topological relationships stored in the floor plan element module can be referred to as floor plan element topological relationships (including topological relationships between different floor plan elements and between floor plan elements and rooms). In general, topological relationships include at least one of the following: topological relationships between different rooms, topological relationships between rooms and floor plan elements, and topological relationships between different floor plan elements.

[0039] When using Wasm modules to infer topological relationships between apartment features, the resulting apartment recognition results are stored and managed by module. Specifically, the results are stored as multiple room modules and multiple apartment element modules. Each module is assigned a module identifier to distinguish between them. The module identifiers can be used to adjust and manage the room modules and apartment element modules.

[0040] For example, after the living room, master bedroom, second bedroom, kitchen, dining room, and bathroom are identified, they will be stored as independent room modules, that is, as living room module, master bedroom module, second bedroom module, dining room module, kitchen module, and bathroom module. In order to facilitate distinction and calling, a corresponding room module identifier will be added to each room module. For example, the living room module is identified as A1, and the master bedroom module is identified as A2. When apartment elements such as walls, windows, and doors are identified, they will be marked as master bedroom windows and master bedroom doors, and module identifiers will be added to each. For example, master bedroom window BA1, master bedroom door BD1. In actual applications, the marking method and marking rules can be set according to actual needs. This is only used as an example and does not constitute a limitation to the disclosed solution.

[0041] In step 105, rendering is performed based on the apartment type recognition result to obtain a rendered target apartment type diagram.

[0042] like Figure 2A schematic diagram of the floor plan recognition results provided by the disclosed solution. The floor plan recognition results accurately identify not only the rooms in the floor plan, but also the individual floor plan elements (including walls, windows, doors, etc.). After obtaining the floor plan recognition results corresponding to the floor plan in the above manner, the floor plan recognition results can be further rendered. During rendering, WebGL is integrated with the web browser, primarily achieved by calling the WebGL API via JavaScript. WebGL is a low-level graphics API based on OpenGL ES, designed specifically for browser environments. It allows developers to leverage GPU acceleration for high-performance 2D and 3D graphics rendering. Therefore, the rendered target floor plan can be either a 2D or 3D image. The target floor plan can include rendered furniture, decorations, etc. For example, a sofa and TV can be rendered in the living room, and a bed and wardrobe can be rendered in the bedroom. Furthermore, targeted rendering can be performed based on desired rendering styles, for example, furniture can be rendered in Chinese or European styles.

[0043] In one or more embodiments of the present disclosure, a pixel array is passed to a Wasm module, and apartment features are extracted based on an apartment feature extraction submodule to obtain apartment features for the floor plan. This process specifically includes calling a Wasm module pre-configured in a web editor; wherein the Wasm module includes a feature extraction submodule and an element recognition submodule; and using the element recognition submodule to identify apartment elements from the pixel array. Apartment elements include walls, windows, and doors; and using the feature extraction submodule to extract apartment features, including element features and room features, from the apartment elements.

[0044] The pre-set Wasm modules mentioned here refer to various pre-written algorithms compiled into Wasm modules. Specifically, when using the geometric feature recognition algorithm as the core of the element recognition sub-module, the geometric feature recognition algorithm is compiled into the element recognition sub-module. In addition, to improve the recognition capability and recognition effect of the element recognition sub-module, a trained classification model can be used to identify household elements. The trained classification model can be embedded in the code (for example, C / C++ code), and a compiler (for example, Emscripten) can be used to compile the classification model into a Wasm sub-module (that is, the element recognition sub-module). This can then realize the recognition of household elements.

[0045] After identifying the apartment elements, the feature extraction submodule further extracts element features and room features from the apartment elements. The feature extraction submodule mentioned here is also a Wasm submodule. Specifically, feature extraction algorithms can include edge detection algorithms (such as the Canny algorithm) and corner detection algorithms (such as the Harris algorithm). These algorithms can be written in a specific format (such as C / C++). A compiler compiles these algorithms into a Wasm-formatted feature extraction submodule.

[0046] After identifying basic apartment elements such as walls, windows, and doors through the element recognition submodule, the feature extraction submodule can further extract element features and room features from the apartment elements. Element features can be the geometric properties and spatial information of individual building structures (such as walls, windows, and doors). Geometric attributes include: dimensions (e.g., wall length and thickness, window height and width, door opening and closing radius, etc.), location (e.g., element coordinates in the floor plan (e.g., starting and ending coordinates)), and orientation (e.g., door opening and closing direction (left / right opening), window installation direction). It also includes the connection relationships between elements (e.g., the attachment relationship between doors and walls) and the association relationship between elements and rooms.

[0047] Room features can be understood as the properties of an enclosed space, defined by multiple elements (such as walls). These properties include geometry, functional type, and relationships with other rooms. Geometric attributes include area (e.g., the floor area of ​​a room, such as a 20-square-meter living room), perimeter (the total length of the room's boundaries), shape (e.g., rectangular, L-shaped), and boundary coordinates (the set of vertex coordinates that define the room's outline).

[0048] Based on the above solution, the floor plan elements are first identified, and then features are extracted from them. The element recognition submodule focuses on locating and classifying floor plan elements from the pixel array (e.g., through rule matching or lightweight models), which is a coarse-grained processing. The feature extraction submodule performs refined analysis of the identified elements (e.g., calculating size, position, and topological relationships), which is a fine-grained processing. Feature extraction is performed only on the identified element areas, avoiding calculations on invalid areas of the entire image (such as the background), which significantly reduces the amount of computation. Element recognition and feature extraction can be executed in parallel in stages (for example, feature extraction can be started immediately after element recognition), fully leveraging Wasm's multi-threading capabilities.

[0049] Furthermore, element recognition filters out noise through specific algorithms (such as shape rules or classification models), ensuring the reliability of recognition results. Feature extraction is based on confirmed element regions, avoiding feature misjudgment due to noise interference. For blurry or low-resolution floor plans, element recognition can incorporate contextual information (e.g., doors and windows are often attached to walls) to enhance feature extraction accuracy.

[0050] In one or more embodiments of the present disclosure, a topological relationship reasoning submodule of a Wasm module is used to perform topological relationship reasoning on apartment features to obtain an apartment recognition result, including: performing topological relationship reasoning on element features and room features through a topological relationship reasoning submodule to generate a room module and an apartment element module; wherein the room module includes the room size, the topological relationship between the room and the apartment elements, and the topological relationship between different rooms; the apartment element module includes: the wall size, the topological relationship between different walls, the topological relationship between the wall and the room, the topological relationship between the wall and the window, and the topological relationship between the wall and the door; the window size, the topological relationship between different windows, the topological relationship between the window and the room, the topological relationship between the window and the wall, and the topological relationship between the window and the door, the door size, the topological relationship between different doors, the topological relationship between the door and the room, the topological relationship between the door and the window, and the topological relationship between the door and the wall.

[0051] In practical applications, after obtaining element features and room features through the method described above, we can further infer the apartment element module and room module respectively.

[0052] Specifically, the reasoning process for the apartment element module includes reasoning about wall dimensions and topological relationships, window dimensions and window topological relationships, and door dimensions and door topological relationships. Wall dimension and topological relationship reasoning involves dimension calculation, calculating length using the Euclidean distance between the wall's start and end coordinates, and calculating thickness using the grayscale gradient of pixels on both sides of the wall (combined with the original pixel array). When reasoning about topological relationships, the intersection type (e.g., T-connection, L-connection) and connection method of the wall endpoints with other walls are analyzed, as well as determining the room boundaries to which the wall belongs. Furthermore, reasoning is required about the connection relationship and method between the wall and the window, as well as the connection relationship and method between the wall and the door.

[0053] Reasoning about window size and topology involves calculating the window's width and height using the coordinates of the identified vertices in the rectangular area. When reasoning about topology, the system determines the wall segment in which the window is located. If a window is located on a wall shared by two rooms, it may be inferred to be a "partition window." Furthermore, the system infers the connection between windows and doors, and how they are connected.

[0054] Door size and topology reasoning include calculating door width based on the pixel span of the door frame area and inferring the door opening and closing radius based on the door's arc trajectory or symbolic markings. When reasoning about topology, door location determines accessibility and connectivity between rooms, and directionality is determined based on how left- or right-opening doors may affect furniture placement.

[0055] As mentioned above, apartment type recognition results are stored and managed in modules. Furthermore, room modules and apartment element modules contain not only their respective dimensions but also their topological relationships. This topological relationship can be fully utilized to improve the accuracy of apartment type recognition.

[0056] Based on the solution described above, we can see that fully leveraging element size and topological relationships, as well as room size and topological relationships, during the apartment type recognition process can effectively improve apartment type recognition accuracy. Specifically, topological analysis determines whether walls form a closed area. If not, missing walls are automatically filled in or endpoints are adjusted. Using topological relationships, closure is effectively verified. Furthermore, doors and windows must be attached to walls. If a door's location is not associated with any wall, re-detection is triggered or it is marked as an anomaly.

[0057] Based on topological relationships, room location (such as central area or edge) and adjacency relationship (such as adjacent to the kitchen) inference functions, room modules can be accurately labeled with functions (for example, marked as living room, bedroom, dining room).

[0058] In one or more embodiments of the present disclosure, element features and room features are calculated and processed by a topological relationship reasoning submodule to generate room modules and apartment element modules, including: constructing a topological relationship graph using element features and room features; analyzing the topological relationship graph by a topological relationship reasoning submodule to determine the topological relationship between room and apartment elements; determining dimension statistics results using element features and room features; wherein the dimension statistics results include: room dimensions, wall dimensions, window dimensions, and door dimensions; and constructing room modules and apartment element modules based on the topological relationship and dimension statistics results.

[0059] First, when constructing a topological graph, the system maps element features (such as the coordinates and dimensions of walls, doors, and windows) and room features (such as boundary vertices and area) into nodes and edges within the graph structure. Specifically, rooms serve as core nodes, while walls, doors, and windows serve as connecting edges or subsidiary nodes. Associations are established through geometric coordinates and adjacency relationships. For example, the start and end coordinates of a wall determine the room it connects to, while the location and orientation of doors and windows are marked as attributes on the edge. Furthermore, the set of boundary vertices within a room is formed into polygons through a closure check to ensure the physical plausibility of the graph structure.

[0060] After obtaining the topological relationship diagram, topological relationship analysis can be performed. Based on the topological relationship diagram, graph theory algorithms (such as depth-first search and connectivity analysis) are used to infer the adjacency relationships between rooms (e.g., the living room and bedroom are connected by a door), the ownership of walls (e.g., a wall belongs to the shared boundary of two rooms), and the functional associations of doors and windows (e.g., windows are attached to walls and may connect different rooms). Simultaneously, the rationality of the topological relationships is verified by combining dimensional statistics (e.g., room area and wall length). For example, closure errors can be detected by the consistency of wall length with the room perimeter, or misjudgments can be corrected by matching door and window positions with room functions (e.g., a small windowless room is marked as a storage room). Ultimately, the topological relationship diagram and dimensional data together form a structured room module (including dimensions and topological relationships) and a floor plan element module (including the dimensions and connectivity of each element), supporting subsequent rendering and interactive optimization.

[0061] Based on this approach, we fully utilize element and room features to build a topological relationship graph. Furthermore, through topological relationship graph modeling and a multi-dimensional verification mechanism, we transform discrete geometric features into a semantic spatial relationship network, addressing the shortcomings of traditional methods in terms of accuracy, efficiency, and adaptability to complex scenarios.

[0062] like Figure 3 A flowchart of the method for correcting apartment type recognition results provided by the present disclosure. In one or more embodiments of the present disclosure, after calculating and processing apartment type features using a Wasm module to obtain an apartment type recognition result, the method further includes: Step 301: Determining reference room modules with significant features among the room modules; the reference room modules include bathroom modules and kitchen modules. Step 302: Correcting the apartment type recognition result based on the reference room modules, topological relationships, and correction rules.

[0063] Step 302 specifically includes the following: Step 3021: When the reference room module is a bathroom module, determining the target room module to which the bathroom module is connected. Step 3022: Based on the topological relationship between different rooms, determining whether the target room module is the largest room in the floor plan; or, based on the topological relationship between different rooms, determining whether the target room module is located at the center of the floor plan and has multiple doors facing other room modules; or, based on the topological relationship between different rooms, determining whether the target room module is connected to the kitchen module. Step 3023: If so, the target room module is the living room; if not, the target room module is the bedroom.

[0064] It is easy to understand that in the actual process of household type recognition, recognition errors are inevitable. In this case, the present disclosure proposes a correction method. That is, compiling the correction algorithm into a Wasm correction module.

[0065] It's important to note that bathrooms and kitchens have more distinct characteristics than other rooms (living rooms and bedrooms), such as smaller areas, dark, windowless bathrooms, and the presence of toilets, showers, and sinks. This means bathrooms and kitchens have higher recognition accuracy.

[0066] The kitchen and bathroom can be used as reference room modules, and then the recognition results of other rooms can be inferred to be correct based on the size, topological relationship, etc. contained in the reference room modules.

[0067] First, a reference room with clear functional characteristics is located (e.g., a small, windowless bathroom module, or a kitchen module near the edge of the unit). Once a bathroom module is detected, the correction module analyzes its adjacent target room modules. If the target room is the largest area in the unit, centrally located, connected to multiple rooms via doors facing the same direction, or directly connected to the kitchen module, it is corrected to a living room. Otherwise, if the target room does not meet these criteria (e.g., it is small and has an unclear function), it is corrected to a bedroom.

[0068] This process combines spatial location rules (such as the living room is usually located in the center), functional association rules (such as the kitchen and living room are adjacent to each other) and geometric feature rules (such as the largest room is preferentially identified as the living room), and corrects misjudgments in preliminary identification (such as misidentifying the living room as a bedroom or storage room) through logical reasoning.

[0069] Based on the publicly available solution, we can see that the strong association rules for "bathroom-living room" and "kitchen-living room" effectively improve room classification accuracy, addressing misclassifications caused by fuzzy local features in traditional methods (e.g., misclassifying a bedroom with a balcony as a living room). This reduces manual corrections (such as dragging room labels) and improves the interactive efficiency of the WebPPT editor.

[0070] The correction results meet the "shortest traffic flow" principle (e.g., living room close to entrance door) and "functional zoning" principle (e.g., bedroom and bathroom adjacent but private), enhancing the credibility of the recognition results. Furthermore, it supports complex structures such as lofts and split-levels, achieving robust judgment through multi-criteria weighted scoring (e.g., area weighting 40%, location weighting 30%, and door direction weighting 30%).

[0071] In one or more embodiments of the present disclosure, rendering is performed based on the apartment type recognition result to obtain a rendered apartment type diagram, including: a Web editor calls a WebGL interface, and renders furniture decoration according to the room size and room type corresponding to the room module in the apartment type recognition result to obtain a rendered target apartment type diagram.

[0072] The implementation method of calling WebGL to render floor plans in the web editor includes: creating a Canvas element through JavaScript and obtaining its WebGL context as the target canvas for graphics rendering.

[0073] After the apartment type recognition results (such as room size, type, and topological relationship) are passed from the Wasm module to the JavaScript layer, WebGL binds the preset rendering style according to the room type (such as light-colored fill for the living room and warm-colored texture for the bedroom), and converts the room's geometric data (such as vertex coordinates and dimensions) into the screen coordinate system through the vertex shader. The fragment shader then generates pixel colors based on the material library (such as floor maps and wall textures).

[0074] For complex structures (such as L-shaped rooms or polygonal areas), the system uses a triangulation algorithm to decompose polygons into multiple triangle primitives. Vertex data is submitted in batches using WebGL's VBO (Video Buffer Object) and depth testing and anti-aliasing techniques are enabled to optimize rendering quality. Furthermore, layered rendering is implemented using the WebGL framebuffer, allowing for dynamic highlighting of room boundaries or overlaying of furniture models during user interaction.

[0075] Based on the above solution, WebGL directly calls the GPU for parallel computing, improving 2D rendering performance by 3-5 times compared to traditional Canvas, and supporting millisecond-level redrawing of floor plans of 1,000 square meters. The mapping between room types and rendering styles is dynamically configured through the material library, allowing for the expansion of new styles (such as modern and Nordic) without modifying the core algorithm, enhancing design flexibility. Incremental rendering technology only updates the user's operating area (such as partial redrawing when dragging a wall), combined with WebGL's requestAnimationFrame to achieve smooth interaction at 60FPS (frames per second). Spatial information is encoded through color, texture, and transparency (such as red marking load-bearing walls and dotted lines indicating removable walls), helping users intuitively understand the floor plan structure and reducing cognitive overhead.

[0076] In one or more embodiments of the present disclosure, it also includes: responding to a user's request to modify the apartment type recognition result, or responding to a user's request to modify the target apartment type diagram, determining the target room module and / or target apartment type element module corresponding to the modification request; obtaining the modification coordinates and modification type contained in the modification request; sending the modification coordinates and modification type to the Wasm module; and using the Wasm module to correct the target room module and / or target apartment type element module.

[0077] As mentioned above, when recognizing floor plans, not only does it identify the specific floor plans contained within, but it also breaks down the results into different modules, including both element modules and room modules. This is different from traditional solutions, which treat floor plan recognition as a whole. If a user modifies a floor plan (for example, adjusting a wall length or door position), traditional solutions require a complete re-recognition process, resulting in performance degradation.

[0078] When a user initiates a modification request for the apartment recognition results or renderings (such as dragging a wall or adjusting the position of doors and windows), the web editor captures user interaction events (such as mouse drag coordinates and operation type), parses the modification request into structured instructions (such as "move wall X to coordinate (x, y)"), and passes the target room module ID (such as the living room) or apartment element ID (such as a door) along with the modification parameters to the Wasm module through JavaScript.

[0079] The Wasm module uses a modular correction mechanism, performing local calculations only on affected submodules (such as wall length and room area). For example, if the user expands the size of a room, the coordinates of its boundary vertices are recalculated, and the closure of adjacent walls is verified through the topological relationship reasoning submodule; if the user moves the position of a door, the coordinate properties of the door are updated and its attachment relationship with the wall is checked.

[0080] This process avoids recalculating the entire image through an incremental update algorithm, while leveraging Wasm's parallel computing capabilities (such as SIMD instructions) to accelerate geometric verification (such as whether the new coordinates cause room overlap).

[0081] In the disclosed solution, the apartment type recognition and rendering algorithm is divided into multiple sub-modules with different functions. During the modification process, Wasm sub-modules can be called on demand. When a modification task is triggered, only one Wasm sub-module needs to be called to complete the modification of the corresponding room module or apartment type element module. Overall, the amount of modified data is minimized. No other Wasm sub-modules will be called, and no other room modules or apartment type element modules will be modified, thus achieving functional decoupling of Wasm algorithm modules. This effectively reduces the amount of modification calculations and improves modification efficiency.

[0082] Wasm's sub-modular design, through functional decoupling, data sharing, and parallel computing, enables modular remediation solutions with both high performance and flexibility in a web environment. This architecture not only addresses the performance bottlenecks of traditional front-end remediation solutions but also provides a scalable technical framework for dynamic interaction with complex user environments, enabling real-time remediation and a WYSIWYG user experience.

[0083] Based on any of the above embodiments, the present disclosure also provides a floor plan rendering device based on image recognition. Figure 4FIG1 is a schematic block diagram of a floor plan rendering device based on image recognition according to an embodiment of the present disclosure. Figure 4 As shown, the image recognition-based floor plan rendering method and apparatus includes an acquisition module 41, a loading module 42, an extraction module 43, an identification module 44, and an image rendering module 45. Acquisition module 41 is used to acquire a floor plan and convert it into a pixel array. Loading module 42 is used to load a Wasm module containing a floor plan feature extraction submodule and a topology relationship inference submodule into a web editor. Extraction module 43 is used to pass the pixel array to the Wasm module and extract floor plan features based on the floor plan feature extraction submodule to obtain floor plan features. Identification module 44 is used to perform topology relationship inference on the floor plan features using the topology relationship inference submodule of the Wasm module to obtain floor plan recognition results. The floor plan recognition results include room modules and floor plan element modules with topological relationships. The topological relationships include at least one of the following: topological relationships between different rooms, topological relationships between rooms and floor plan elements, and topological relationships between different floor plan elements. Image rendering module 45 is used to render based on the floor plan recognition results to obtain a rendered target floor plan.

[0084] Optionally, the extraction module 43 in the device (including a feature extraction submodule and an element identification submodule within a Wasm module) is configured to call a Wasm module pre-configured in the web editor. The Wasm module includes a feature extraction submodule and an element identification submodule; the element identification submodule is used to identify floor plan elements from the pixel array. Floor plan elements include walls, windows, and doors; the feature extraction submodule is used to extract floor plan features, including element features and room features, from the floor plan elements.

[0085] Optionally, the Wasm module also includes a topological relationship reasoning submodule. The recognition module 44 in the device is also used to perform topological relationship reasoning on element features and room features through the topological relationship reasoning submodule to generate a room module and a unit element module. Among them, the room module includes room size, the topological relationship between the room and the unit element, and the topological relationship between different rooms; the unit element module includes: wall size, the topological relationship between different walls, the topological relationship between the wall and the room, the topological relationship between the wall and the window, and the topological relationship between the wall and the door; window size, the topological relationship between different windows, the topological relationship between the window and the room, the topological relationship between the window and the wall, and the topological relationship between the window and the door; door size, the topological relationship between different doors, the topological relationship between the door and the room, the topological relationship between the door and the window, and the topological relationship between the door and the wall.

[0086] The recognition module 44 in the device is also used to construct a topological relationship diagram using element features and room features; analyze the topological relationship diagram through the topological relationship reasoning submodule to determine the topological relationship between room and apartment elements; use the element features and room features to determine the dimension statistics results; wherein the dimension statistics results include: room dimensions, wall dimensions, window dimensions and door dimensions; based on the topological relationship and dimension statistics results, room modules and apartment element modules are constructed.

[0087] The device also includes a correction module 46 (including a correction module in the Wasm module) for determining reference room modules with significant features among the room modules. The reference room modules include bathroom modules and kitchen modules. The room type recognition results are corrected based on the reference room modules, topological relationships, and correction rules.

[0088] The correction module 46 is also used to determine the target room module connected to the bathroom module when the reference room module is a bathroom module; determine whether the target room module is the largest room in the floor plan based on the topological relationship between different rooms; or, determine whether the target room module is located in the center of the floor plan and has multiple doors facing other room modules in the same direction based on the topological relationship between different rooms; or, determine whether the target room module is connected to the kitchen module based on the topological relationship between different rooms; if so, the target room module is the living room; if not, the target room module is the bedroom.

[0089] The image rendering module 45 (including the WebGL rendering module) in the device is also used for the Web editor to call the WebGL interface, and render the furniture decoration according to the room size and room type corresponding to the room module in the apartment recognition result to obtain the rendered target apartment plan.

[0090] The correction module 46 (including the correction module in the Wasm module) is used to respond to the user's request to modify the apartment type recognition result, or respond to the user's request to modify the target apartment type diagram, determine the target room module and / or target apartment type element module corresponding to the modification request; obtain the modification coordinates and modification type contained in the modification request; send the modification coordinates and modification type to the Wasm module; and use the Wasm module to modify the target room module and / or target apartment type element module.

[0091] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0092] The execution subject of the floor plan rendering method based on image recognition in the specific embodiment of the present disclosure can be an electronic device such as a server (including a local server or a cloud server).

[0093] Therefore, based on any of the above embodiments, the present disclosure further provides an electronic device, which can execute the floor plan rendering method based on image recognition of any of the embodiments described above in the present disclosure.

[0094] Figure 5 1 is a schematic block diagram of the structure of an electronic device according to one embodiment of the present disclosure. The hardware structure of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various circuits including one or more processors 1200, memory 1300, and / or hardware modules. The bus 1100 can also connect various other circuits 1400 such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0095] Bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component Architecture (EISA) bus. Buses can be classified as address buses, data buses, control buses, and the like. For ease of illustration, this figure shows only one connecting line, but this does not imply that there is only one bus or only one type of bus.

[0096] The present disclosure also provides a readable storage medium having a computer program stored therein, which is used to implement the above-mentioned method when the computer program is executed by a processor. "Readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use in an instruction execution system, device or equipment or in combination with these instruction execution systems, devices or equipment. More specific examples of readable storage media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM), etc.

[0097] The present disclosure also provides a computer program product. The method of the present disclosure can be implemented in whole or in part using software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, the process or function of the present disclosure is performed in whole or in part.

[0098] A computer program or instruction can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instruction can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any accessible medium or a data storage device such as a server or data center that integrates one or more accessible media. The accessible medium can be a magnetic medium such as a floppy disk, hard disk, or magnetic tape; an optical medium such as a digital video disk; or a semiconductor medium such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or can include both volatile and non-volatile types of storage media.

[0099] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing method device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing method device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0103] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" means that the specific features, structures, or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present disclosure. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment / method or example. Moreover, the specific features, structures, or characteristics described may be combined in a suitable manner in any one or more embodiments / methods or examples. In addition, those skilled in the art may combine and combine different embodiments / methods or examples described in this specification and the features of different embodiments / methods or examples, unless they are contradictory.

[0104] Those skilled in the art will appreciate that the above embodiments are merely intended to clearly illustrate the present disclosure and are not intended to limit the scope of the present disclosure. Other changes or modifications may be made based on the above disclosure, and such changes or modifications are still within the scope of the present disclosure.

Claims

1. A floor plan rendering method based on image recognition, characterized in that: The method comprises: Obtaining a floor plan, and converting the floor plan into a pixel array; Load the Wasm module containing the house type feature extraction submodule and the topology relationship reasoning submodule in the web editor; Passing the pixel array to the Wasm module and performing apartment features extraction based on the apartment features extraction submodule to obtain apartment features of the apartment plan; Performing topological relationship reasoning on the apartment features by the topological relationship reasoning submodule of the Wasm module to obtain an apartment type recognition result, wherein the apartment type recognition result includes: a room module and an apartment type element module having a topological relationship, wherein the topological relationship includes at least one of a topological relationship between different rooms, a topological relationship between a room and an apartment type element, and a topological relationship between different apartment type elements; Rendering is performed based on the apartment type recognition result to obtain a rendered target apartment type diagram; The topological relationship reasoning submodule of the Wasm module performs topological relationship reasoning on the apartment type features to obtain an apartment type recognition result, including: performing topological relationship reasoning on the element features and room features of the apartment type features to generate the room module and the apartment type element module through the topological relationship reasoning submodule of the Wasm module; The room module includes room dimensions, topological relationships between rooms and apartment elements, and topological relationships between different rooms; The apartment element module includes: wall size, topological relationship between different walls, topological relationship between walls and rooms, topological relationship between walls and windows, and topological relationship between walls and doors; window size, topological relationship between different windows, topological relationship between windows and rooms, topological relationship between windows and walls, and topological relationship between windows and doors; door size, topological relationship between different doors, topological relationship between doors and rooms, topological relationship between doors and windows, and topological relationship between doors and walls.

2. The method according to claim 1, characterized in that The pixel array is passed to the Wasm module and the apartment features are extracted based on the apartment feature extraction submodule to obtain the apartment features of the apartment plan, including: Calling a Wasm module pre-set in the Web editor, wherein the Wasm module includes the feature extraction submodule and the element recognition submodule; Using the element recognition submodule to recognize apartment elements from the pixel array, wherein the apartment elements include walls, windows, and doors; The feature extraction submodule is used to extract the apartment features including element features and room features from the apartment elements.

3. The method according to claim 1, characterized in that Performing topological relationship reasoning on the element features and room features of the apartment features by the topological relationship reasoning submodule to generate the room module and the apartment element module includes: constructing a topological relationship graph using the element features and the room features; Analyzing the topological relationship graph by the topological relationship reasoning submodule to determine the topological relationship between the rooms and apartment elements; Determine a size statistical result using the element feature and the room feature, wherein the size statistical result includes: the room size, the wall size, the window size, and the door size; The room module and the apartment element module are constructed based on the topological relationship and the size statistical results.

4. The method according to claim 1, wherein After calculating and processing the apartment type features through the Wasm module to obtain an apartment type recognition result, the method further includes: Determining a reference room module having significant features among the room modules; wherein the reference room module includes a bathroom module and a kitchen module; The apartment type recognition result is corrected based on the reference room module, the topological relationship and the correction rule.

5. The method according to claim 4, characterized in that The correcting of the apartment type recognition result based on the reference room module and the topological relationship includes: When the reference room module is a bathroom module, determining a target room module connected to the bathroom module; Based on the topological relationship between the different rooms, determine whether the target room module is the largest room in the floor plan; or, based on the topological relationship between the different rooms, determine whether the target room module is located at the center of the floor plan and has multiple doors facing other room modules in the same direction; or, based on the topological relationship between the different rooms, determine whether the target room module is connected to the kitchen module; if so, the target room module is the living room; if not, the target room module is the bedroom.

6. The method according to claim 1, wherein Rendering is performed based on the apartment type recognition result to obtain a rendered target apartment type diagram, including: The Web editor calls the WebGL interface to perform furniture decoration rendering for the room size and room type corresponding to the room module corresponding to the apartment recognition result, and obtains the rendered target apartment plan.

7. The method according to claim 1, characterized in that Also includes: In response to a user's request to modify the apartment type recognition result, or in response to a user's request to modify the target apartment type diagram, determining a target room module and / or target apartment type element module corresponding to the modification request; Obtaining the modification coordinates and modification type included in the modification request; Sending the modification coordinates and the modification type to the Wasm module; The target room module and / or the target apartment element module are modified using the Wasm module.

8. An electronic device, characterized in that: include: a memory storing execution instructions; as well as A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1 to 7.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images