Indoor area dynamic measurement method and device based on depth camera

Abstract

The invention is suitable for the technical field of computer vision, and relates to a depth camera-based indoor area dynamic measurement method and device, and the method comprises the steps: obtaining first depth data shot indoors by a depth camera at a current time period; mapping the first depth data to a three-dimensional coordinate to obtain point cloud data; performing object identification on the point cloud data to obtain target point cloud data representing the same target object; calculating the distance value between the target objects and the surface area corresponding to the target objects according to the target point cloud data; calculating the indoor area according to the distance value between the target objects and the surface area corresponding to the target objects; and acquiring second depth data shot by the depth camera indoors in the next time period, and updating the indoor area according to the second depth data. Therefore, compared with an existing measurement method, the method does not need to use any manual measurement tool, achieves the automatic measurement of the indoor area, and saves the time and manpower cost.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, and particularly relates to a method and device for dynamically measuring indoor area based on a depth camera. Background Art

[0002] The dynamic measurement of indoor area is of great significance in many application scenarios, such as architectural design, indoor navigation, virtual reality, etc. Existing indoor area measurement methods usually require the use of measurement tools, such as tape measures or laser rangefinders, and are manually measured, which is time-consuming, laborious and error-prone. Therefore, an automated indoor area measurement method without manual intervention is needed. Summary of the Invention

[0003] The embodiments of this application provide a method and device for dynamically measuring indoor area based on a depth camera, which can solve the problem of an automated indoor area measurement method without manual intervention.

[0004] In a first aspect, the embodiments of this application provide a method for dynamically measuring indoor area based on a depth camera, including: Obtain first depth data captured by the depth camera indoors in the current time period; wherein, the first depth data represents the distance from the depth point to the object being photographed; Map the first depth data to three-dimensional coordinates to obtain point cloud data; Perform object recognition on the point cloud data to obtain target point cloud data representing the same target object; Calculate the distance value between the target objects and the surface area corresponding to the target objects according to the target point cloud data; Calculate the indoor area according to the distance value between the target objects and the surface area corresponding to the target objects; Obtain second depth data captured by the depth camera indoors in the next time period, and update the indoor area according to the second depth data.

[0005] In a possible implementation manner of the first aspect, after obtaining the first depth data captured by the depth camera indoors in the current time period, it further includes: Obtain the pose data of the depth camera when shooting indoors in the current time period; Correct the first depth data through the pose data.

[0006] In a possible implementation manner of the first aspect, after obtaining the first depth data captured by the depth camera indoors in the current time period, it further includes: Preprocess the first depth data through a preset noise removal algorithm.

[0007] In a possible implementation of the first aspect, object recognition is performed on the point cloud data to obtain target point cloud data representing the same target object, including: Extracting a first feature map corresponding to the point cloud data through a preset feature extraction network; Dividing the first feature map into multiple feature regions through a preset context integration network, and performing a pooling operation in each feature region to obtain a second feature map; Performing a deconvolution operation on the second feature map through a preset upsampling network to obtain a third feature map; Performing a classification operation on the third feature map according to a preset classification network to obtain a class label corresponding to the point cloud data; Taking the point cloud data with the same class label as the target point cloud data representing the same target object.

[0008] In a possible implementation of the first aspect, performing a classification operation on the third feature map according to a preset classification network to obtain a class label corresponding to the point cloud data, including: Determining the class label corresponding to the point cloud data according to the following formula: ; Wherein, Represents the class label corresponding to the point cloud data, Represents the classification operation of the preset classification network, Represents the deconvolution operation corresponding to the preset upsampling network, Represents the pooling operation of the i-th feature region in the first feature map by the preset context integration network, Represents the extraction operation corresponding to the preset feature extraction network, Represents the size of the i-th feature region in the first feature map, Represents the point cloud data input to the preset feature extraction network.

[0009] In a possible implementation of the first aspect, calculating a distance value between the target objects and a surface area corresponding to the target object according to the target point cloud data, including: Performing surface reconstruction on the target point cloud data representing the same target object to obtain a three-dimensional model of the target object; Based on a preset area calculation algorithm, determining the surface area corresponding to the target object according to the three-dimensional model of the target object; Based on a preset distance calculation algorithm, determining the distance value between the target objects according to the three-dimensional model of the target object.

[0010] In a possible implementation of the first aspect, calculating the indoor area according to the distance value between the target objects and the surface area corresponding to the target objects includes: Based on a preset division rule, dividing the indoor space into a plurality of indoor sub-regions according to the distance value between the target objects and the surface area corresponding to the target objects, and calculating the area corresponding to the indoor sub-regions; For each pair of adjacent indoor sub-regions, calculating the overlapping area between the adjacent indoor sub-regions; Determining the indoor area according to the overlapping area between the adjacent indoor sub-regions and the area corresponding to the indoor sub-regions.

[0011] In a second aspect, an embodiment of the present application provides an indoor area dynamic measurement device based on a depth camera, including: An acquisition module, configured to acquire first depth data captured by the depth camera indoors during a current time period; wherein, the first depth data represents the distance from a depth point to an object being photographed; A mapping module, configured to map the first depth data to three-dimensional coordinates to obtain point cloud data; An identification module, configured to perform object identification on the point cloud data to obtain target point cloud data representing the same target object; A first calculation module, configured to calculate the distance value between the target objects and the surface area corresponding to the target objects according to the target point cloud data; A second calculation module, configured to calculate the indoor area according to the distance value between the target objects and the surface area corresponding to the target objects; An update module, configured to acquire second depth data captured by the depth camera indoors during the next time period, and update the indoor area according to the second depth data.

[0012] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where when the processor executes the computer program, the method described in the first aspect above is implemented.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0014] The beneficial effects of the embodiments of the present application compared with the prior art are: Embodiments of the present application can obtain depth data from a depth camera, map it to three-dimensional coordinates to obtain point cloud data. Then, by performing object recognition on the point cloud data, the target point cloud data of the same target object can be extracted. Based on the target point cloud data, the distance value between target objects and the surface area corresponding to the target object can be calculated. Through these calculation results, an estimated value of the indoor area can be obtained, and in subsequent time periods, by obtaining new depth data, the measurement result of the indoor area can be updated in real time. It can be seen that compared with the existing measurement methods, this method of the embodiments of the present application does not require the use of any manual measurement tools, such as a tape measure or a laser rangefinder, realizes automatic indoor area measurement, and saves time and labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0016] Figure 1 is a schematic flowchart of a method for dynamically measuring indoor area based on a depth camera provided by an embodiment of the present application; Figure 2 is a structural block diagram of a device for dynamically measuring indoor area based on a depth camera provided by an embodiment of the present application; Figure 3 is a schematic structural diagram of a terminal device provided by an embodiment of the present application. EMBODIMENTS

[0017] In the following description, specific details such as specific system structures and technologies are proposed for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0018] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0019] It should also be understood that the term "and / or" as used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0020] As used in the specification and appended claims of this application, the term "if" may be construed, depending on the context, as "when", "once", "in response to determining", or "in response to detecting". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0021] In addition, in the description of the specification and appended claims of this application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0022] Reference to "one embodiment" or "some embodiments" etc. described in the specification of this application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0023] The technical solutions of the embodiments of this application will be introduced below.

[0024] See Figure 1 , which is a schematic flowchart of the indoor area dynamic measurement method based on a depth camera provided by the embodiments of this application. By way of example and not limitation, this method can be applied to a terminal device, and this method includes the following steps: Step S101, obtain first depth data captured by the depth camera indoors during the current time period.

[0025] Wherein, the first depth data represents the distance from the depth point to the object being photographed.

[0026] Exemplarily, the depth camera may be a laser depth camera. A laser depth camera is a camera that uses lidar technology for depth perception. It emits a laser beam using a laser beam emitter and then receives the return signal of the laser beam through a receiver to measure the distance between the object and the camera.

[0027] In an alternative implementation, after obtaining the first depth data captured by the depth camera indoors during the current time period, the following steps are further included: Obtain the pose data of the depth camera when it is capturing indoors during the current time period; Correct the first depth data based on the pose data.

[0028] Exemplarily, the pose data is position data and attitude data. Obtaining the pose data of the depth camera when it is capturing indoors during the current time period includes: Using a GPS positioning system to obtain the position data of the depth camera in the geographical coordinate system; Using the inertial measurement unit set in the depth camera to obtain the attitude data of the depth camera.

[0029] Exemplarily, correcting the first depth data based on the pose data includes: Calibrating parameters such as the focal length and principal point coordinates of the depth camera to calculate the internal parameters of the depth camera; Based on the pose data, calculating the pose of the depth camera in the world coordinate system as the external parameters of the depth camera through the conversion between the camera and the world coordinate system; According to the internal parameters and external parameters of the depth camera, performing a reprojection operation to map each pixel point in the depth data to the corresponding world coordinate, thereby correcting the first depth data.

[0030] In an alternative implementation, after obtaining the first depth data captured by the depth camera indoors during the current time period, the following steps are further included: Preprocess the first depth data through a preset noise removal algorithm.

[0031] Among them, the preset noise removal algorithm can be mean filtering, median filtering, or Gaussian filtering.

[0032] It can be understood that using a preset noise removal algorithm can effectively reduce the noise interference in the depth camera data in the preprocessing stage, improving the accuracy and precision of subsequent object recognition and area measurement.

[0033] Step S102: Map the first depth data to three-dimensional coordinates to obtain point cloud data.

[0034] Exemplarily, according to the camera projection model corresponding to the depth camera, project the pixel points in the first depth data from the pixel coordinate system to the three-dimensional coordinate system (i.e., the world coordinate system) to obtain point cloud data. Step S103: Perform object recognition on the point cloud data to obtain target point cloud data representing the same target object.

[0035] Among them, the target object refers to objects such as indoor walls, doors, windows, columns, tables, chairs, etc.

[0036] Exemplarily, the object recognition model can be used to perform object recognition on the point cloud data to obtain target point cloud data representing the same target object.

[0037] Among them, the preset object recognition model includes a preset feature extraction network, a preset context integration network, a preset upsampling network, and a preset classification network.

[0038] The preset feature extraction network can be a convolutional neural network, which is used to extract high-level semantic features from the input point cloud data to form a first feature map; the preset context integration network is used to divide the first feature map into multiple regions of different sizes by using multi-scale feature information, and perform a pooling operation (such as max pooling) in each region to obtain a second feature map, so as to capture context information at multiple scales; the preset upsampling network is used to restore the second feature map obtained by the pooling operation to the same resolution as the original input point cloud data through deconvolution operation. After restoring the resolution, the object segmentation result in the point cloud data can be predicted more accurately; the preset classification network can be a fully connected layer, which is used to classify the second feature map and generate the final object segmentation result.

[0039] In a specific application, performing object recognition on the point cloud data to obtain target point cloud data representing the same target object includes: Extracting the first feature map corresponding to the point cloud data through the preset feature extraction network; Dividing the first feature map into multiple feature regions through the preset context integration network, and performing a pooling operation in each feature region to obtain a second feature map; Performing a deconvolution operation on the second feature map through the preset upsampling network to obtain a third feature map; Performing a classification operation on the third feature map according to the preset classification network to obtain the class label corresponding to the point cloud data; Taking the point cloud data with the same class label as the target point cloud data representing the same target object.

[0040] Exemplarily, performing a classification operation on the third feature map according to the preset classification network to obtain the class label corresponding to the point cloud data includes: Determining the class label corresponding to the point cloud data according to the following formula: ; Among them, represents the class label corresponding to the point cloud data, represents the classification operation of the preset classification network, represents the deconvolution operation corresponding to the preset upsampling network, represents the pooling operation of the preset context integration network on the th feature region in the first feature map, represents the extraction operation corresponding to the preset feature extraction network, represents the size of the \(i\)-th feature region in the first feature map, represents the point cloud data input to the preset feature extraction network.

[0041] In the embodiments of the present application, through the preset feature extraction network, high-level semantic features can be extracted from the point cloud data, which helps to accurately capture the feature information of the target object and improve the accuracy of object recognition; the preset context integration network can utilize multi-scale feature information, divide the first feature map into multiple regions of different sizes, and perform pooling operations to capture context information at different scales, so as to more comprehensively understand the target object in the point cloud data and improve the robustness of recognition; by performing a deconvolution operation on the second feature map obtained by the pooling operation through the preset upsampling network, the resolution of the feature map can be restored, and the object segmentation result in the point cloud data can be predicted more accurately, improving the accuracy of object segmentation; the preset classification network uses a fully connected layer to classify the third feature map, and can obtain the class label corresponding to the point cloud data.

[0042] Step S104, calculate the distance value between the target objects and the surface area corresponding to the target object according to the target point cloud data.

[0043] In a specific application, calculating the distance value between the target objects and the surface area corresponding to the target object according to the target point cloud data includes: Performing surface reconstruction on the target point cloud data representing the same target object to obtain the three-dimensional object model of the target object; Based on a preset area calculation algorithm, determining the surface area corresponding to the target object according to the three-dimensional model of the target object; Based on a preset distance calculation algorithm, determining the distance value between the target objects according to the three-dimensional model of the target object.

[0044] Exemplarily, performing surface reconstruction on the target point cloud data representing the same target object to obtain the three-dimensional object model of the target object includes: using the Poisson reconstruction algorithm to perform surface reconstruction on the target point cloud data representing the same target object to obtain the three-dimensional object model of the target object composed of triangular meshes.

[0045] Exemplarily, based on a preset area calculation algorithm, determining the surface area corresponding to the target object according to the three-dimensional model of the target object includes: using Heron's formula or vector cross product to calculate the area of each triangle, and summing the areas of all triangles to obtain the surface area of the target object.

[0046] Exemplarily, based on a preset distance calculation algorithm, a distance value between the target objects is determined according to the three-dimensional models of the target objects, including: calculating a normal vector of the three-dimensional model of the target object according to the three-dimensional coordinates of the target point cloud data in the three-dimensional model of the target object, and based on the normal vector of the three-dimensional model of the target object, calculating a model distance value between the three-dimensional models of the respective target objects according to the nearest neighbor search algorithm or the distance transformation method, and taking the distance value between the three-dimensional models of the target objects as the distance value between the target objects.

[0047] Step S105, calculating the indoor area according to the distance value between the target objects and the surface area corresponding to the target objects.

[0048] In a specific application, calculating the indoor area according to the distance value between the target objects and the surface area corresponding to the target objects includes: Based on a preset partitioning rule, dividing the indoor space into multiple indoor sub-regions according to the distance value between the target objects and the surface area corresponding to the target objects, and calculating the area of the indoor sub-region corresponding thereto; For each pair of adjacent indoor sub-regions, calculating the overlapping area between the adjacent indoor sub-regions; Determining the indoor area according to the overlapping area between the adjacent indoor sub-regions and the area of the indoor sub-region corresponding thereto.

[0049] Exemplarily, based on a preset partitioning rule, dividing the indoor space into multiple indoor sub-regions according to the distance value between the target objects and the surface area corresponding to the target objects includes: the indoor space can be divided into different indoor sub-regions according to a pre-determined rule. For example, the space can be divided according to the positions and shapes of objects such as walls, doors, windows, columns, etc.

[0050] Exemplarily, calculating the area of the indoor sub-region corresponding thereto includes: calculating the area of the indoor sub-region corresponding thereto according to the area calculation formula of a polygon, such as Green's theorem or the Shoelace formula.

[0051] Exemplarily, for each pair of adjacent indoor sub-regions, calculating the overlapping area between the adjacent indoor sub-regions includes: processing and registering the point cloud data of the two regions to ensure that they are in the same coordinate system, then converting the point cloud data into a mesh representation using voxel calculation or other methods, and finally calculating the overlapping area on the mesh representation. By calculating the number of voxels of the intersection between the meshes or the area between the voxels, the area of the overlapping area can be obtained.

[0052] Step S106: Obtain the second depth data captured by the depth camera indoors in the next time period, and update the indoor area according to the second depth data.

[0053] It can be understood that after the second depth data is collected in the next time period, the processes of the above steps 101 to step S105 are repeated, and the indoor area measured according to the first depth data before will be updated according to the obtained latest indoor area.

[0054] In the embodiment of the present application, depth data can be obtained from the depth camera and mapped to three-dimensional coordinates to obtain point cloud data. Then, by performing object recognition on the point cloud data, the target point cloud data of the same target object can be extracted. According to the target point cloud data, the distance value between the target objects and the surface area corresponding to the target objects can be calculated. Through these calculation results, an estimated value of the indoor area can be obtained, and in subsequent time periods, by obtaining new depth data, the measurement result of the indoor area can be updated in real time. It can be seen that compared with the existing measurement methods, this method in the embodiment of the present application does not require the use of any manual measurement tools, such as a tape measure or a laser rangefinder, realizes automatic measurement of the indoor area, and saves time and labor costs.

[0055] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0056] Corresponding to the indoor area dynamic measurement method based on the depth camera described in the above embodiments, Figure 2 The structural block diagram of the indoor area dynamic measurement device based on the depth camera provided by the embodiment of the present application is shown. For the sake of convenience of description, only the parts related to the embodiment of the present application are shown.

[0057] Refer to Figure 2 , the device includes: An acquisition module 21, configured to acquire first depth data captured by the depth camera indoors in the current time period; wherein, the first depth data represents the distance from the depth point to the object being photographed; A mapping module 22, configured to map the first depth data to three-dimensional coordinates to obtain point cloud data; An identification module 23, configured to perform object recognition on the point cloud data to obtain target point cloud data representing the same target object; A first calculation module 24, configured to calculate the distance value between the target objects and the surface area corresponding to the target objects according to the target point cloud data; A second calculation module 25, configured to calculate the indoor area according to the distance value between the target objects and the surface area corresponding to the target objects; An update module 26, configured to obtain second depth data captured by a depth camera indoors in a next time period, and update the indoor area according to the second depth data.

[0058] It should be noted that for the information interaction, execution process, etc. between the above-mentioned devices / units, since they are based on the same concept as the method embodiments of the present application, their specific functions and the technical effects brought thereby can be specifically referred to in the method embodiment section, and will not be elaborated herein.

[0059] Figure 3 It is a schematic structural diagram of a terminal device provided by an embodiment of the present application. As Figure 3 shown, the terminal device 3 of this embodiment includes: at least one processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30. When the processor 30 executes the computer program 32, the steps in any of the above method embodiments are implemented.

[0060] The terminal device 3 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art can understand that Figure 3 this is only an example of the terminal device 3, and does not constitute a limitation on the terminal device 3. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0061] The so-called processor 30 may be a central processing unit (CPU), and the processor 30 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0062] In some embodiments, the memory 31 may be an internal storage unit of the terminal device 3, such as a hard disk or memory of the terminal device 3. In other embodiments, the memory 31 may also be an external storage device of the terminal device 3, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal device 3. Further, the memory 31 may also include both the internal storage unit and the external storage device of the terminal device 3. The memory 31 is used to store an operating system, application programs, a BootLoader, data, and other programs, such as the program code of the computer program. The memory 31 may also be used to temporarily store data that has been output or will be output.

[0063] Those skilled in the art can clearly understand that for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0064] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

[0065] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned embodiment methods of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, USB flash drive, mobile hard disk, magnetic disk or optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be electrical carrier signal and telecommunication signal.

[0066] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0067] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in this text can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0068] In the embodiments provided in this application, it should be understood that the disclosed device / network device and method can be implemented in other ways. For example, the device / network device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0069] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0070] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An indoor area dynamic measurement method based on a depth camera, characterized in that, including: Obtain first depth data captured by a depth camera indoors during the current time period; wherein, the first depth data represents the distance from a depth point to the object being captured; Map the first depth data to three-dimensional coordinates to obtain point cloud data; Perform object recognition on the point cloud data to obtain target point cloud data representing the same target object; Calculate the distance value between the target objects and the surface area corresponding to the target object according to the target point cloud data; Calculate the indoor area according to the distance value between the target objects and the surface area corresponding to the target object; Obtain second depth data captured by a depth camera indoors during the next time period, and update the indoor area according to the second depth data.

2. The indoor area dynamic measurement method based on a depth camera according to claim 1, wherein After obtaining the first depth data captured by a depth camera indoors during the current time period, it further includes: Obtain the pose data of the depth camera when capturing indoors during the current time period; Correct the first depth data through the pose data.

3. The indoor area dynamic measurement method based on a depth camera according to claim 1, characterized in that After obtaining the first depth data captured by a depth camera indoors during the current time period, it further includes: Preprocess the first depth data through a preset noise removal algorithm.

4. The indoor area dynamic measurement method based on a depth camera according to claim 1, wherein, Performing object recognition on the point cloud data to obtain target point cloud data representing the same target object includes: Extract a first feature map corresponding to the point cloud data through a preset feature extraction network; Divide the first feature map into multiple feature regions through a preset context integration network, and perform a pooling operation in each feature region to obtain a second feature map; Perform a deconvolution operation on the second feature map through a preset upsampling network to obtain a third feature map; Perform a classification operation on the third feature map according to a preset classification network to obtain the class label corresponding to the point cloud data; Use the point cloud data with the same class label as the target point cloud data representing the same target object.

5. The indoor area dynamic measurement method based on a depth camera according to claim 4, wherein Performing a classification operation on the third feature map according to a preset classification network to obtain the class label corresponding to the point cloud data includes: Determine the class label corresponding to the point cloud data according to the following formula: ； Among them, represents the class label corresponding to the point cloud data, represents the classification operation of the preset classification network, represents the deconvolution operation corresponding to the preset upsampling network, represents the pooling operation of the preset context integration network on the $i$-th feature region in the first feature map, represents the extraction operation corresponding to the preset feature extraction network, represents the size of the $i$-th feature region in the first feature map, represents the point cloud data input to the preset feature extraction network.

6. The indoor area dynamic measurement method based on a depth camera according to claim 1, wherein Calculating the distance value between the target objects and the surface area corresponding to the target object according to the target point cloud data includes: Perform surface reconstruction on the target point cloud data representing the same target object to obtain a three-dimensional model of the object of the target object; Based on a preset area calculation algorithm, determine the surface area corresponding to the target object according to the three-dimensional model of the target object; Based on a preset distance calculation algorithm, determine the distance value between the target objects according to the three-dimensional model of the target object.

7. The indoor area dynamic measurement method based on a depth camera according to claim 6, wherein Calculating the indoor area according to the distance value between the target objects and the surface area corresponding to the target object includes: Based on a preset division rule, divide the indoor space into multiple indoor sub-regions according to the distance value between the target objects and the surface area corresponding to the target object, and calculate the area of the indoor sub-region corresponding to it; For each adjacent indoor sub-region, calculate the overlapping area between the adjacent indoor sub-regions; Determine the indoor area according to the overlapping area between the adjacent indoor sub-regions and the area of the indoor sub-region corresponding to it.

8. An indoor area dynamic measurement device based on a depth camera, characterized in that, including: An acquisition module, configured to acquire first depth data captured by a depth camera indoors during a current time period; wherein the first depth data represents the distance from a depth point to an object to be captured; A mapping module, configured to map the first depth data to three-dimensional coordinates to obtain point cloud data; An identification module, configured to perform object identification on the point cloud data to obtain target point cloud data representing the same target object; A first calculation module, configured to calculate a distance value between the target objects and a surface area corresponding to the target objects according to the target point cloud data; A second calculation module, configured to calculate an indoor area according to the distance value between the target objects and the surface area corresponding to the target objects; An update module, configured to acquire second depth data captured by the depth camera indoors during a next time period, and update the indoor area according to the second depth data.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images