Construction monitoring method, device, electronic equipment and storage medium

Abstract

The disclosed embodiments disclose a construction monitoring method, device, electronic device and storage medium. The construction monitoring method comprises: obtaining a construction site image captured by an image acquisition device and standard construction information matching the construction site image; extracting actual construction information contained in the construction site image; comparing the actual construction information with the standard construction information to obtain a comparison result; and generating construction situation information of the construction site indicated by the construction site image based on the comparison result. The disclosed embodiments can improve the efficiency of construction monitoring and help ensure construction quality through construction monitoring.

Description

Technical Field

[0001] The present disclosure relates to image processing technology, and in particular to a construction monitoring method, device, electronic equipment and storage medium. Background Art

[0002] Construction monitoring is a means to ensure that the project fully reflects the design ideas and ensures safety during the construction or use phase. For example, in the process of house decoration, the rationality of the decoration design and the quality of the decoration are often closely related to the owner of the house. Hidden works are a pain point in the decoration process. If the hidden works are done well, it can avoid a lot of unnecessary troubles for the owner of the house. It is also an important link to test the decoration company. However, the understanding and skills of the construction workers are uneven. If professional personnel come to the site for acceptance one by one, they also need to refer to specific hidden engineering drawings. Because the construction sites are scattered, this is not only inefficient, but also each household is customized according to the needs of the hidden engineering design, it is easy to overlook some details.

[0003] It can be seen that how to improve the efficiency of construction monitoring and ensure construction quality through construction monitoring is an issue worthy of attention. Summary of the invention

[0004] The embodiments of the present disclosure provide a construction monitoring method, device, electronic device and storage medium to improve the efficiency of construction monitoring and help ensure construction quality through construction monitoring.

[0005] According to a first aspect of an embodiment of the present disclosure, a construction monitoring method is provided, comprising:

[0006] Acquire a construction site image captured by an image acquisition device, and standard construction information matching the construction site image;

[0007] extracting actual construction information contained in the construction site image;

[0008] Comparing the actual construction information with the standard construction information to obtain a comparison result;

[0009] Based on the comparison result, construction situation information of the construction site indicated by the construction site image is generated.

[0010] Optionally, in the method of any embodiment of the present disclosure, generating the construction situation information of the construction site indicated by the construction site image based on the comparison result includes at least one of the following:

[0011] In response to the comparison result indicating that the actual construction information matches the standard construction information, generating first construction situation information, wherein the first construction situation information indicates that the construction site indicated by the construction site image is under normal construction;

[0012] In response to the comparison result indicating that the actual construction information does not match the standard construction information, determining whether the construction condition of the construction site indicated by the construction site image is within a preset deviation range; and, in response to being within the preset deviation range, generating second construction condition information, wherein the second construction condition information indicates that the construction condition of the construction site indicated by the construction site image is within the preset deviation range;

[0013] In response to the comparison result indicating that the actual construction information does not match the standard construction information, and the construction conditions at the construction site indicated by the construction site image are outside a preset deviation range, third construction condition information is generated, wherein the third construction condition information indicates that the construction at the construction site indicated by the construction site image is abnormal.

[0014] Optionally, in the method of any embodiment of the present disclosure, the actual construction information includes actual wiring location information and actual wiring type information, and the standard construction information includes a predetermined building information model containing water and electricity wiring information; and

[0015] The actual construction information is compared with the standard construction information to obtain a comparison result, including:

[0016] Extracting standard wiring location information and standard wiring type information from the building information model;

[0017] The actual wiring position information is compared with the standard wiring position information, and the actual wiring type information is compared with the standard wiring type information to obtain a comparison result.

[0018] Optionally, in the method of any embodiment of the present disclosure, the method further includes:

[0019] According to the pipeline position indicated by the predetermined standard wiring position information and the pipeline type indicated by the predetermined standard wiring type information, the pipeline and the three-dimensional space indicated by the preset construction site image are three-dimensionally fused to obtain a building information model.

[0020] Optionally, in the method of any embodiment of the present disclosure, the three-dimensional fusion of the pipeline and the three-dimensional space indicated by the preset construction site image to obtain the building information model includes:

[0021] The pipeline and the three-dimensional space indicated by the preset construction site image are three-dimensionally fused to obtain a fusion result;

[0022] The pipeline trends in the fusion results are linearized to obtain a building information model.

[0023] Optionally, in the method of any embodiment of the present disclosure, extracting actual construction information contained in the construction site image includes:

[0024] The construction site image is input into a pre-trained deep model, and edges of the construction site image are extracted via the deep model to obtain actual wiring position information contained in the construction site image, and the actual wiring position information is used as actual construction information.

[0025] Optionally, in the method of any embodiment of the present disclosure, extracting actual construction information contained in the construction site image includes:

[0026] The construction site image is input into a pre-trained deep model, actual wiring type information contained in the construction site image is extracted via the deep model, and the actual wiring type information is used as actual construction information.

[0027] Optionally, in the method of any embodiment of the present disclosure, the method further includes at least one of the following:

[0028] Presenting the construction status information;

[0029] In response to the construction situation information indicating that the construction at the construction site indicated by the construction site image is abnormal, an alarm message is generated.

[0030] Optionally, in the method of any embodiment of the present disclosure, the wiring of the wiring position indicated by the standard wiring position information consists of straight lines and curves, and the curvature of the curve is less than or equal to a preset curvature threshold.

[0031] According to a second aspect of an embodiment of the present disclosure, a construction monitoring device is provided, comprising:

[0032] an acquisition unit configured to acquire a construction site image captured by an image acquisition device and standard construction information matching the construction site image;

[0033] an extraction unit configured to extract actual construction information contained in the construction site image;

[0034] A comparison unit is configured to compare the actual construction information with the standard construction information to obtain a comparison result;

[0035] The first generating unit is configured to generate construction situation information of the construction site indicated by the construction site image based on the comparison result.

[0036] Optionally, in the device of any embodiment of the present disclosure, the first generating unit includes at least one of the following:

[0037] A first generating subunit is configured to generate first construction situation information in response to the comparison result indicating that the actual construction information matches the standard construction information, wherein the first construction situation information indicates that the construction site indicated by the construction site image is under normal construction;

[0038] The second generating subunit is configured to, in response to the comparison result indicating that the actual construction information does not match the standard construction information, determine whether the construction situation of the construction site indicated by the construction site image is within a preset deviation range; and, in response to being within the preset deviation range, generate second construction situation information, wherein the second construction situation information indicates that the construction situation of the construction site indicated by the construction site image is within the preset deviation range;

[0039] The third generating sub-unit is configured to generate third construction condition information in response to the comparison result indicating that the actual construction information does not match the standard construction information, and the construction condition of the construction site indicated by the construction site image is outside a preset deviation range, wherein the third construction condition information indicates that the construction at the construction site indicated by the construction site image is abnormal.

[0040] Optionally, in the device of any embodiment of the present disclosure, the actual construction information includes actual wiring position information and actual wiring type information, and the standard construction information includes a predetermined building information model containing water and electricity wiring information; and

[0041] The comparison unit comprises:

[0042] an extraction subunit configured to extract standard wiring location information and standard wiring type information from the building information model;

[0043] The comparison subunit is configured to compare the actual wiring position information with the standard wiring position information, and to compare the actual wiring type information with the standard wiring type information to obtain a comparison result.

[0044] Optionally, in the device of any embodiment of the present disclosure, the device further includes:

[0045] The fusion unit is configured to perform three-dimensional fusion of the pipeline and the three-dimensional space indicated by the preset construction site image according to the pipeline position indicated by the predetermined standard wiring position information and the pipeline type indicated by the predetermined standard wiring type information to obtain a building information model.

[0046] Optionally, in the device of any embodiment of the present disclosure, the fusion unit includes:

[0047] The fusion subunit is configured to three-dimensionally fuse the pipeline and the three-dimensional space indicated by the preset construction site image to obtain a fusion result;

[0048] The linearization processing subunit is configured to perform linearization processing on the pipeline trends in the fusion result to obtain a building information model.

[0049] Optionally, in the device of any embodiment of the present disclosure, the extraction unit includes:

[0050] The first input sub-unit is configured to input the construction site image into a pre-trained deep model, perform edge extraction on the construction site image via the deep model, obtain actual wiring position information contained in the construction site image, and use the actual wiring position information as actual construction information.

[0051] Optionally, in the device of any embodiment of the present disclosure, the extraction unit includes:

[0052] The second input subunit is configured to input the construction site image into a pre-trained deep model, extract actual wiring type information contained in the construction site image via the deep model, and use the actual wiring type information as actual construction information.

[0053] Optionally, in the device of any embodiment of the present disclosure, the device further includes at least one of the following:

[0054] A presentation unit, configured to present the construction status information;

[0055] The second generating unit is configured to generate alarm information in response to the construction situation information indicating that the construction at the construction site indicated by the construction site image is abnormal.

[0056] Optionally, in the device of any embodiment of the present disclosure, the wiring of the wiring position indicated by the standard wiring position information consists of straight lines and curves, and the curvature of the curve is less than or equal to a preset curvature threshold.

[0057] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including:

[0058] Memory for storing computer programs;

[0059] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method of any embodiment of the construction monitoring method of the first aspect of the present disclosure is implemented.

[0060] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable medium is provided, and when the computer program is executed by a processor, the method of any embodiment of the construction monitoring method of the first aspect is implemented.

[0061] According to a fifth aspect of an embodiment of the present disclosure, a computer program is provided, which includes a computer-readable code. When the computer-readable code is run on a device, a processor in the device executes instructions for implementing each step in the method of any embodiment of the construction monitoring method of the first aspect above.

[0062] Based on the construction monitoring method, device, electronic device and storage medium provided by the above-mentioned embodiments of the present disclosure, it is possible to obtain the construction site image taken by the image acquisition device and the standard construction information matching the above-mentioned construction site image, and then extract the actual construction information contained in the above-mentioned construction site image, and then compare the above-mentioned actual construction information with the above-mentioned standard construction information to obtain the comparison result, and finally, based on the above-mentioned comparison result, generate the construction situation information of the construction site indicated by the above-mentioned construction site image. In the embodiment of the present disclosure, the standard construction information can be set in advance, and the construction situation can be determined by comparing the actual construction information in the construction site image with the set standard construction information. As a result, there is no need for professional personnel to conduct on-site supervision and acceptance, which improves the efficiency of construction monitoring and acceptance, and to a certain extent avoids the uneven construction quality caused by the inconsistent standards of constructors, supervisors, and acceptors.

[0063] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0065] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0066] Figure 1 This is a flow chart of the first embodiment of the construction monitoring method disclosed in the present invention.

[0067] Figure 2 This is a flow chart of a second embodiment of the construction monitoring method disclosed herein.

[0068] Figure 3A-3C The present invention is a schematic diagram of an application scenario of an embodiment of the construction monitoring method disclosed herein.

[0069] Figure 4 It is a structural schematic diagram of an embodiment of the construction monitoring device disclosed in the present invention.

[0070] Figure 5 A structural diagram of an electronic device provided as an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0071] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0072] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0073] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0074] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0075] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0076] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0077] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0078] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0079] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0080] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0081] The disclosed embodiments may be applied to at least one electronic device in a terminal device, a computer system, and a server, and may operate with numerous other general or special computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with at least one electronic device in a terminal device, a computer system, and a server include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0082] At least one of the terminal device, computer system, and server may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules may include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. The computer system / server may be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located on local or remote computing system storage media including storage devices.

[0083] Please refer to Figure 1 , shows a process 100 of a first embodiment of a construction monitoring method according to the present disclosure. The construction monitoring method comprises:

[0084] 101, obtaining a construction site image captured by an image acquisition device and standard construction information matching the construction site image.

[0085] In this embodiment, the executor of the construction monitoring method (such as a server, a terminal device, a construction monitoring device, etc.) can obtain a construction site image of the construction site taken by an image acquisition device set at the construction site from other electronic devices or locally through a wired connection or a wireless connection, and obtain standard construction information matching the above-mentioned construction site image.

[0086] As an example, a fisheye camera can be selected as an image acquisition device. The reason for choosing a fisheye camera is that it has a wide field of view and there is no need to worry about angle issues, especially when it is installed at the top, both the ground and the wall can be observed. If it is a non-fisheye camera, multiple cameras are required to achieve the field of view of the fisheye camera, and the images captured by multiple cameras must be calibrated, which increases the amount of calculation and affects the accuracy. If a non-fisheye camera is used, it still depends on its angle and position to obtain a better viewing angle. Fisheye cameras do not have this problem.

[0087] The construction site image may be an image obtained by photographing the construction site using the above-mentioned image acquisition device during or after the construction.

[0088] The standard construction information matched with the above-mentioned construction site image may include the location information, height information, width information, angle information, type information, etc. of the construction object (such as a bridge, a building, etc.) of the project to be carried out at the construction site indicated by the construction site image. That is, during the construction process or the inspection stage, the standard construction information can be used as a standard to measure whether the construction is qualified.

[0089] It should be noted that the execution subject may first obtain the construction site image captured by the image acquisition device, and then obtain the standard construction information matching the construction site image; or first obtain the standard construction information matching the construction site image, and then obtain the construction site image captured by the image acquisition device; or simultaneously obtain the construction site image captured by the image acquisition device and the standard construction information matching the construction site image. This is not limited here.

[0090] 102, extracting actual construction information contained in the above-mentioned construction site image.

[0091] In this embodiment, the execution entity may extract actual construction information contained in the construction site image from the construction site image.

[0092] The actual construction information may include location information, height information, width information, angle information, type information, etc. of the construction objects (such as bridges, buildings, indoor wiring, etc.) of the projects actually being carried out at the above-mentioned construction sites.

[0093] 103, comparing the actual construction information with the standard construction information to obtain a comparison result.

[0094] In this embodiment, the execution entity may compare the actual construction information obtained in 102 with the standard construction information obtained in 101 to obtain a comparison result.

[0095] Among them, the comparison result can indicate whether the actual construction information is consistent with the standard construction information, and the error size between the actual construction information and the standard construction information.

[0096] 104. Based on the comparison result, generate construction situation information of the construction site indicated by the construction site image.

[0097] In this embodiment, the execution entity may generate the construction situation information of the construction site indicated by the construction site image based on the comparison result obtained in 103 .

[0098] The above construction situation information can represent the actual construction situation of the construction site, and the construction situation information can be used to evaluate the construction quality. For example, the construction situation information may include: whether the construction progress meets expectations, whether the difference between the actual height of the construction object and the expected height meets expectations, etc.

[0099] The construction monitoring method provided by the above-mentioned embodiment of the present disclosure can obtain the construction site image taken by the image acquisition device, and the standard construction information matching the above-mentioned construction site image, and then extract the actual construction information contained in the above-mentioned construction site image, and then compare the above-mentioned actual construction information with the above-mentioned standard construction information to obtain the comparison result, and finally, based on the above-mentioned comparison result, generate the construction situation information of the construction site indicated by the above-mentioned construction site image. In the embodiment of the present disclosure, the standard construction information can be set in advance, and the construction situation can be determined by comparing the actual construction information in the construction site image with the set standard construction information. As a result, there is no need for professional personnel to conduct on-site supervision and acceptance, which improves the efficiency of construction monitoring and acceptance, and to a certain extent avoids the uneven construction quality caused by the inconsistent standards of constructors, supervisors, and acceptors.

[0100] In some optional implementations of this embodiment, the actual construction information includes actual wiring position information and actual wiring type information. The wiring position information can indicate the position of the pipeline in the construction site image, and can also indicate the position of the pipeline in the world coordinates. The wiring type information (including actual wiring type information and standard wiring type information) can be used to indicate the type of the pipeline. For example, the color of the pipeline; or whether the pipeline is a water pipe, an electric wire, a heating pipe, a gas pipe, etc.

[0101] The standard construction information includes a predetermined building information model (BIM) containing standard water and electricity wiring information. Here, the building information model can be formulated and generated by pipeline layout designers.

[0102] On this basis, the execution subject may perform step 103 in the following manner to compare the actual construction information with the standard construction information to obtain a comparison result:

[0103] Firstly, standard wiring location information and standard wiring type information are extracted from the building information model.

[0104] Here, according to the construction wiring rules, the three-dimensional apartment space information in the building information model can be read and parsed to obtain standard wiring position information (such as coordinate information) and standard wiring type information (such as color information, different wiring types will be drawn in different colors).

[0105] Afterwards, the actual wiring position information is compared with the standard wiring position information, and the actual wiring type information is compared with the standard wiring type information to obtain a comparison result.

[0106] Here, the comparison result can represent: whether the position indicated by the above-mentioned actual wiring position information is consistent with the position indicated by the above-mentioned standard wiring position information; whether the type indicated by the above-mentioned actual wiring type information is consistent with the type indicated by the above-mentioned standard wiring type information; the error between the position indicated by the above-mentioned actual wiring position information and the position indicated by the above-mentioned standard wiring position information; the error between the type indicated by the above-mentioned actual wiring type information and the type indicated by the above-mentioned standard wiring type information, and so on.

[0107] It can be understood that in the above optional implementation, standard water and electricity wiring information can be pre-set, and the wiring situation can be determined by comparing the actual wiring position information with the standard wiring position information, and comparing the actual wiring type information with the standard wiring type information. As a result, there is no need for professional personnel to conduct on-site supervision and acceptance, which improves the efficiency of wiring monitoring and acceptance, and to a certain extent avoids the uneven wiring quality caused by the inconsistent standards of wiring workers, supervisors, and acceptors.

[0108] In some application scenarios of the above-mentioned optional implementation methods, the above-mentioned execution entity can also three-dimensionally fuse the pipeline and the three-dimensional space indicated by the preset construction site image according to the pipeline position indicated by the predetermined standard wiring position information and the pipeline type indicated by the predetermined standard wiring type information to obtain a building information model.

[0109] Here, the three-dimensional coordinates indicated by the pipeline position can be converted into world coordinates to obtain the coordinates of the pipeline in the three-dimensional fusion space. The coordinates of the pixels in the preset construction site image are converted into world coordinates to obtain the coordinates of each point in the construction site in the three-dimensional fusion space. In addition, the pipeline position can be characterized based on multiple points. According to the rules and trends of construction wiring, the various points are connected, and different colors are used to represent the wire pipes of different pipeline types. The pipeline and the three-dimensional space indicated by the preset construction site image can be three-dimensionally fused to obtain a building information model.

[0110] It is understood that in the above optional implementation, the building information model can be obtained by three-dimensionally fusing the pipeline and the three-dimensional space indicated by the preset construction site image, which is convenient for subsequent comparison of the actual wiring type information with the standard wiring type information, and helps to obtain more accurate comparison results.

[0111] In some cases of the above application scenarios, the above execution subject may also obtain the building information model in the following manner:

[0112] Firstly, the three-dimensional space indicated by the pipeline and the preset construction site image is fused in three dimensions to obtain the fusion result.

[0113] Then, the pipeline trends in the above fusion results are linearized to obtain the building information model.

[0114] It can be understood that in the above situation, the pipeline trends of all or part of the pipelines in the fusion results can be linearized, so that the linearization degree of the actual construction wiring can be used as a standard for wiring monitoring and acceptance, which helps to further improve the quality of construction wiring.

[0115] In some optional implementations of this embodiment, the execution subject may perform the above 102 in the following manner to extract the actual construction information contained in the above construction site image:

[0116] The above-mentioned construction site image is input into a pre-trained deep model, and edge extraction (such as sub-pixel edge extraction) is performed on the above-mentioned construction site image through the above-mentioned deep model to obtain the actual wiring position information contained in the above-mentioned construction site image, and the above-mentioned actual wiring position information is used as the actual construction information.

[0117] Among them, the above-mentioned deep model can be trained in a supervised manner.

[0118] Exemplarily, the above-mentioned deep model can use HRnet-18-small-v2 as the basis, add 4 times Pixel Shuffle (for upsampling) at the position of 1 / 4 downsampling output, so as to realize edge extraction, and use the logistic regression function (sigmoid) to output the probability that each pixel or sub-pixel belongs to the pipeline, so as to obtain more accurate edge information of each pipeline. The information of each pipeline uses Quantity Focal Loss (quantity focal loss function) and Dice Loss (dice coefficient loss function) as loss functions. The edge information of the pipeline is Gaussian distributed, and Quantity Focal Loss is used as the loss function. During the test, the pipeline information is subtracted from the edge information to find a suitable threshold (the threshold is used to determine whether it is background, pipeline, and then realize semantic segmentation), and the obtained information is binarized, and then the distance transform is performed to extract the center line position of each pipeline, and the center line position is used as the actual wiring position.

[0119] Here, sub-pixel is an image element that is a subdivided pixel, a basic unit, and is a unit smaller than a pixel. By performing sub-pixel edge extraction, the image resolution can be improved. Usually, sub-pixel edge points exist in areas where the image gradually changes excessively. Sub-pixel edge extraction can be understood as a method of using software algorithms to improve edge detection accuracy while keeping the hardware conditions of the camera system unchanged, or an image processing technology that makes the resolution less than one pixel.

[0120] It can be understood that in the above optional implementation, more accurate actual wiring position information can be obtained by performing edge extraction on the construction site image.

[0121] Optionally, the positions of edge points may be obtained by using various methods such as polynomial fitting.

[0122] In some optional implementations of this embodiment, the execution subject may perform the above 102 in the following manner to extract the actual construction information contained in the above construction site image:

[0123] The construction site image is input into a pre-trained deep model, and the actual wiring type information contained in the construction site image is extracted through the deep model, and the actual wiring type information is used as actual construction information.

[0124] Among them, the above-mentioned deep model can be trained in a supervised manner.

[0125] The model structure of the depth model in this optional implementation may be the same as or different from the model structure of the depth model in the previous optional implementation.

[0126] It can be understood that in the above optional implementation, a deep model can be used to extract the actual wiring type information contained in the construction site image, thereby obtaining more accurate actual wiring type information.

[0127] In some optional implementations of this embodiment, the method further includes at least one of the following:

[0128] The first item is to present the above-mentioned construction status information.

[0129] Second, when the construction situation information indicates that the construction at the construction site indicated by the construction site image is abnormal, an alarm message is generated.

[0130] It can be understood that the above optional implementation method can provide construction monitors with a reference to the construction status by presenting construction status information or generating alarm information, and promptly alarm when construction abnormalities occur, which helps to discover safety hazards more promptly and improve the safety of wiring construction.

[0131] In some optional implementations of this embodiment, the wiring at the wiring position indicated by the standard wiring position information is composed of straight lines and curves, and the curvature of the curve is less than or equal to a preset curvature threshold.

[0132] It can be understood that in the above optional implementation, by setting the value range of the wiring curvature, the occurrence of excessive wiring curvature can be avoided to a certain extent, thereby achieving more refined wiring monitoring and helping to improve the safety of wiring construction.

[0133] Further references Figure 2 , Figure 2 2 is a flow chart of a second embodiment of the construction monitoring method disclosed herein. The process 200 of the construction monitoring method includes:

[0134] 201, obtain the construction site image captured by the image acquisition device and the standard construction information matching the construction site image. Then, continue to execute 202.

[0135] In this embodiment, the executor of the construction monitoring method (such as a server, a terminal device, a construction monitoring device, etc.) can obtain the construction site images taken by the image acquisition device and the standard construction information matching the above-mentioned construction site images from other electronic devices or locally through a wired connection or a wireless connection.

[0136] In this embodiment, 201 and Figure 1 The 101 in the corresponding embodiment is basically the same and will not be described again here.

[0137] 202, extracting the actual construction information contained in the construction site image. Then, proceed to 203.

[0138] In this embodiment, the execution entity may extract actual construction information contained in the construction site image.

[0139] In this embodiment, 202 and Figure 1 The 102 in the corresponding embodiment is basically the same and will not be described again here.

[0140] 203, compare the actual construction information with the standard construction information to obtain a comparison result. Then, continue to execute 204.

[0141] In this embodiment, the execution entity may compare the actual construction information with the standard construction information to obtain a comparison result.

[0142] In this embodiment, step 203 and Figure 1 Step 103 in the corresponding embodiment is basically the same and will not be described again here.

[0143] 204 , determining whether the comparison result indicates that the actual construction information matches the standard construction information. If so, proceed to 206 ; if not, proceed to 205 .

[0144] In this embodiment, the execution entity may determine whether the comparison result indicates that the actual construction information matches the standard construction information. For example, the execution entity may determine whether the comparison result indicates that the actual construction situation indicated by the actual construction information is the same as the standard construction situation indicated by the standard construction information.

[0145] 205 , determining whether the construction situation of the construction site indicated by the construction site image is within a preset deviation range. If yes, proceed to 207 ; if no, proceed to 208 .

[0146] In this embodiment, the execution entity may determine whether the construction situation of the construction site indicated by the construction site image is within a preset deviation range.

[0147] Here, a variety of methods can be used to determine whether the construction situation is within the preset deviation range. For example, if the Euclidean distance between the actual construction information and the standard construction information is less than or equal to the preset distance threshold, it can be determined that the construction situation is within the preset deviation range; if the similarity between the actual construction information and the standard construction information is less than or equal to the preset similarity threshold, it can be determined that the construction situation is within the preset deviation range.

[0148] 206. Generate first construction status information.

[0149] In this embodiment, the execution subject may generate first construction situation information, wherein the first construction situation information indicates that the construction site indicated by the construction site image is in normal construction.

[0150] 207, generating second construction status information.

[0151] In this embodiment, the execution subject may generate second construction situation information, wherein the second construction situation information indicates that the construction situation of the construction site indicated by the construction site image is within a preset deviation range.

[0152] 208, generating third construction status information.

[0153] In this embodiment, the execution subject may generate third construction situation information, wherein the third construction situation information indicates that the construction at the construction site indicated by the construction site image is abnormal.

[0154] Here, the first construction status information, the second construction status information, and the third construction status information may be in different texts, or different colors, or different shapes, or different sounds.

[0155] It should be noted that, in addition to the above contents, the embodiments of the present application may also include Figure 1 The same or similar features and effects as those in the corresponding embodiments will not be described in detail here.

[0156] from Figure 2 It can be seen that the process 200 of the construction monitoring method in this embodiment can generate corresponding construction situation information by distinguishing different construction situations, thereby providing a more refined construction situation reference for the construction monitor, which is helpful to timely evaluate the construction quality and prevent construction risks.

[0157] As an example, please refer to Figure 3A-3C , Figure 3A-3C It is a schematic diagram of an application scenario of an embodiment of the construction monitoring method disclosed in the present invention.

[0158] exist Figure 3AIn this application scenario, the intelligent design of water and electricity wiring, the construction of water and electricity workers, and 3D video fusion are involved. Based on this, in the smart decoration scenario, the water and electricity wiring design is digitized, and the construction site images after the water and electricity wiring construction are obtained using a fisheye camera. Deep learning is performed to analyze the position information of each pipeline in the construction site image (that is, the actual construction information), and the world coordinate information of each pipe (that is, the position information in the construction site image) is obtained using 3D video fusion technology. According to the digital information of the design, a comparison is performed to obtain the result of whether the actual construction information is consistent with the standard actual construction information, and the mismatched positions are marked, and professionals analyze whether they are reasonable.

[0159] Specifically, in Figure 3A The following modules are included:

[0160] Digitalization module: convert the three-dimensional coordinates of the water and electricity wiring (i.e. the xyz coordinates determined during wiring) into the world coordinates in the three-dimensional fusion space, and connect the points according to the wiring trend, using different colors to represent different types of wire pipes. According to the rules of construction wiring, and reading and parsing the three-dimensional apartment space information in the building information model, output the coordinate point information, and draw the pipeline shape at each coordinate point through the front end, and add elbows at the turning points. Among them, the curvature of the curve at the above elbow is less than or equal to the preset curvature threshold. As an example, the generated pipeline trend can be as follows Figure 3B shown.

[0161] Deep learning analysis: First, collect the actual data of the relevant water and electricity pipe layout as training data, and perform data annotation. When annotating, distinguish the water and electricity pipes according to their functions. For example, blue electric pipes represent weak electricity, red electric pipes represent strong electricity, and water pipes are uniformly green. However, depending on the function, the outside of the water pipe will be wrapped with red or blue foam tubes, so this data can also be used as training data. After the training data is prepared, deep learning training can be performed, using HRnet-18-small-v2 as the basis. Finally, add 4 times Pixel Shuffle at the position of 1 / 4 downsampling output to perform sub-pixel extraction, and use the sigmoid function to obtain the information of each pipeline and its edge information. The information of each pipeline uses Quantity FocalLoss and Dice Loss as loss functions. The edge information of the pipeline is Gaussian distributed, and Quantity FocalLoss is used as the loss function. During testing, the pipeline information is subtracted from the edge information, and a suitable threshold is found. The obtained information is binarized, and then the distance transform is performed to extract the centerline position of each pipeline.

[0162] 3D fusion: Based on the line pipe type and centerline position obtained through deep learning analysis, it can be converted into a 3D fusion space to obtain the 3D position information of the electric pipe. In addition, fisheye video can be fused into the 3D house to achieve a fusion effect.

[0163] Line simplification: The 3D position information of the electric pipe obtained by the 3D fusion module is processed by line detection to obtain the straight line segment information, and then the straight line segment information is visualized in the 3D fusion space. After that, the difference between the water and electricity pipe layout design and the corresponding construction can be observed by human eyes.

[0164] Comparison: Although the difference between the water and electricity pipe layout design and the corresponding construction can be observed by human eyes in the 3D fusion space, it will consume a lot of manpower and time. This process is that the machine automatically compares the difference between the water and electricity pipe layout design and the corresponding construction, and marks the location of the difference. Compare the type and coordinates of the water and electricity pipe layout design in the 3D fusion space, and the pipeline type and coordinate information of the water and electricity pipe layout after construction extracted from the fisheye image in the 3D fusion space, analyze the construction error information, and find the construction location greater than the error threshold according to the set error threshold, and mark it in the 3D space. As an example, please refer to Figure 3C ,exist Figure 3C In the figure, different image shapes indicated by different labels are used to indicate the problems of water and electricity wiring detection. Label 301 indicates normal, which is consistent with the system recommendation; label 302 indicates partial normal; label 303 indicates abnormal. In addition, for abnormal places, alarm information can be generated for information prompts.

[0165] In the above application scenarios, it can be ensured that workers carry out construction according to the design plan and drawings. It can determine whether the construction process meets the specifications, whether the construction results are consistent with the design, and whether the inconsistencies in the construction are reasonable. In this way, there is no need for professional personnel to conduct on-site supervision and acceptance, which improves the efficiency of wiring monitoring and acceptance, and to a certain extent avoids the uneven wiring quality caused by the inconsistent standards of wiring workers, supervisors, and acceptance personnel.

[0166] Further references Figure 4 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a construction monitoring device. Figure 1 Corresponding to the method embodiment shown in FIG. 3 , in addition to the features described below, the device embodiment may also include Figure 1 The same or corresponding features as those of the method embodiment shown in FIG. 3 , and the Figure 1 The method embodiment shown in FIG3 has the same or corresponding effects. The device can be specifically applied to various electronic devices.

[0167] like Figure 4As shown, the construction monitoring device 400 of this embodiment includes: an acquisition unit 401, an extraction unit 402, a comparison unit 403 and a first generation unit 404. The acquisition unit 401 is configured to acquire the construction site image captured by the image acquisition device and the standard construction information matching the above construction site image; the extraction unit 402 is configured to extract the actual construction information contained in the above construction site image; the comparison unit 403 is configured to compare the above actual construction information with the above standard construction information to obtain a comparison result; the first generation unit 404 is configured to generate the construction situation information of the construction site indicated by the above construction site image based on the above comparison result.

[0168] In this embodiment, the acquisition unit 401 of the construction monitoring device 400 can acquire the construction site image taken by the image acquisition device and the standard construction information matching the construction site image.

[0169] In this embodiment, the extraction unit 402 may extract actual construction information contained in the construction site image.

[0170] In this embodiment, the comparison unit 403 may compare the actual construction information with the standard construction information to obtain a comparison result.

[0171] In this embodiment, the first generating unit 404 may generate the construction situation information of the construction site indicated by the construction site image based on the comparison result.

[0172] In some optional implementations of this embodiment, the first generating unit 404 includes at least one of the following:

[0173] A first generating subunit (not shown in the figure) is configured to generate first construction situation information in response to the comparison result indicating that the actual construction information matches the standard construction information, wherein the first construction situation information indicates that the construction site indicated by the construction site image is in normal construction;

[0174] The second generating subunit (not shown in the figure) is configured to, in response to the comparison result indicating that the actual construction information does not match the standard construction information, determine whether the construction situation of the construction site indicated by the construction site image is within a preset deviation range; and, in response to being within the preset deviation range, generate second construction situation information, wherein the second construction situation information indicates that the construction situation of the construction site indicated by the construction site image is within the preset deviation range;

[0175] The third generating subunit (not shown in the figure) is configured to generate third construction condition information in response to the above-mentioned comparison result indicating that the above-mentioned actual construction information does not match the above-mentioned standard construction information, and the construction condition of the construction site indicated by the above-mentioned construction site image is outside the preset deviation range, wherein the above-mentioned third construction condition information indicates that the construction at the construction site indicated by the above-mentioned construction site image is abnormal.

[0176] In some optional implementations of this embodiment, the actual construction information includes actual wiring location information and actual wiring type information, and the standard construction information includes a predetermined building information model including water and electricity wiring information; and

[0177] The comparison unit 403 includes:

[0178] An extraction subunit (not shown in the figure), configured to extract standard wiring location information and standard wiring type information from the building information model;

[0179] The comparison subunit (not shown in the figure) is configured to compare the actual wiring position information with the standard wiring position information, and to compare the actual wiring type information with the standard wiring type information to obtain a comparison result.

[0180] In some optional implementations of this embodiment, the apparatus 400 further includes:

[0181] A fusion unit (not shown in the figure) is configured to perform three-dimensional fusion of the pipeline and the three-dimensional space indicated by the preset construction site image according to the pipeline position indicated by the predetermined standard wiring position information and the pipeline type indicated by the predetermined standard wiring type information to obtain a building information model.

[0182] In some optional implementations of this embodiment, the fusion unit includes:

[0183] A fusion subunit (not shown in the figure) is configured to perform three-dimensional fusion of the pipeline and the three-dimensional space indicated by the preset construction site image to obtain a fusion result;

[0184] The linearization processing subunit (not shown in the figure) is configured to perform linearization processing on the pipeline trends in the above fusion results to obtain a building information model.

[0185] In some optional implementations of this embodiment, the extracting unit 402 includes:

[0186] The first input sub-unit (not shown in the figure) is configured to input the above-mentioned construction site image into a pre-trained deep model, perform edge extraction on the above-mentioned construction site image through the above-mentioned deep model, obtain the actual wiring position information contained in the above-mentioned construction site image, and use the above-mentioned actual wiring position information as actual construction information.

[0187] In some optional implementations of this embodiment, the extracting unit 402 includes:

[0188] The second input sub-unit (not shown in the figure) is configured to input the above-mentioned construction site image into a pre-trained deep model, extract the actual wiring type information contained in the above-mentioned construction site image through the above-mentioned deep model, and use the above-mentioned actual wiring type information as the actual construction information.

[0189] In some optional implementations of this embodiment, the apparatus 400 further includes at least one of the following:

[0190] A presentation unit (not shown in the figure), configured to present the above-mentioned construction situation information;

[0191] The second generating unit (not shown in the figure) is configured to generate alarm information in response to the construction situation information indicating that the construction site indicated by the construction site image is abnormal.

[0192] In some optional implementations of this embodiment, the wiring at the wiring position indicated by the standard wiring position information is composed of straight lines and curves, and the curvature of the curve is less than or equal to a preset curvature threshold.

[0193] In the construction monitoring device 400 provided by the above embodiment of the present disclosure, the acquisition unit 401 can acquire the construction site image taken by the image acquisition device, and the standard construction information matching the above construction site image, and then the extraction unit 402 can extract the actual construction information contained in the above construction site image, and then the comparison unit 403 can compare the above actual construction information with the above standard construction information to obtain the comparison result, and finally, the first generation unit 404 can generate the construction situation information of the construction site indicated by the above construction site image based on the above comparison result. In the embodiment of the present disclosure, the standard construction information can be set in advance, and the construction situation can be determined by comparing the actual construction information in the construction site image with the set standard construction information. As a result, there is no need for professional personnel to conduct on-site supervision and acceptance, which improves the efficiency of construction monitoring and acceptance, and to a certain extent avoids the uneven construction quality caused by the inconsistent standards of constructors, supervisors, and acceptors.

[0194] Below, reference Figure 5The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.

[0195] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is illustrated.

[0196] like Figure 5 As shown, the electronic device 5 includes one or more processors 501 and a memory 502 .

[0197] The processor 501 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0198] The memory 502 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 501 may run the program instructions to implement the construction monitoring method of each embodiment of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0199] In one example, the electronic device may further include: an input device 503 and an output device 504, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0200] For example, when the electronic device is the first device or the second device, the input device 503 may be the microphone or microphone array described above, for capturing input signals from a sound source. When the electronic device is a stand-alone device, the input device 503 may be a communication network connector, for receiving collected input signals from the first device and the second device.

[0201] In addition, the input device 503 may also include, for example, a keyboard, a mouse, etc. The output device 504 may output various information to the outside, including determined distance information, direction information, etc. The output device 504 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0202] Of course, to simplify, Figure 5 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0203] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the construction monitoring method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0204] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0205] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0206] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0207] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0208] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0209] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0210] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0211] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A construction monitoring method, characterized in that: The method comprises: Acquire a construction site image captured by an image acquisition device, which is a fisheye camera, and standard construction information matching the construction site image; extracting actual construction information contained in the construction site image; Comparing the actual construction information with the standard construction information to obtain a comparison result; Based on the comparison result, generating construction situation information of the construction site indicated by the construction site image; The actual construction information includes actual wiring location information and actual wiring type information, and the standard construction information includes a predetermined building information model including water and electricity wiring information; and The actual construction information is compared with the standard construction information to obtain a comparison result, including: Extracting standard wiring position information and standard wiring type information from the building information model, wherein the pipeline trends in the building information model are linearized; The actual wiring position information is compared with the standard wiring position information, and the actual wiring type information is compared with the standard wiring type information to obtain a comparison result.

2. The method according to claim 1, characterized in that The method further comprises: According to the pipeline position indicated by the predetermined standard wiring position information and the pipeline type indicated by the predetermined standard wiring type information, the pipeline and the three-dimensional space indicated by the preset construction site image are three-dimensionally fused to obtain a building information model.

3. The method according to claim 2, characterized in that The three-dimensional fusion of the pipeline and the three-dimensional space indicated by the preset construction site image to obtain the building information model includes: The pipeline and the three-dimensional space indicated by the preset construction site image are three-dimensionally fused to obtain a fusion result; The pipeline trends in the fusion results are linearized to obtain a building information model.

4. The method according to any one of claims 1 to 3, characterized in that: The extracting the actual construction information contained in the construction site image includes: Inputting the construction site image into a pre-trained depth model, performing edge extraction on the construction site image via the depth model, obtaining actual wiring position information contained in the construction site image, and using the actual wiring position information as actual construction information; and / or The construction site image is input into a pre-trained deep model, actual wiring type information contained in the construction site image is extracted via the deep model, and the actual wiring type information is used as actual construction information.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises at least one of the following: Presenting the construction status information; In response to the construction situation information indicating that the construction at the construction site indicated by the construction site image is abnormal, an alarm message is generated.

6. The method according to any one of claims 1 to 3, characterized in that: Generating the construction situation information of the construction site indicated by the construction site image based on the comparison result includes at least one of the following: In response to the comparison result indicating that the actual construction information matches the standard construction information, generating first construction situation information, wherein the first construction situation information indicates that the construction site indicated by the construction site image is under normal construction; In response to the comparison result indicating that the actual construction information does not match the standard construction information, determining whether the construction condition of the construction site indicated by the construction site image is within a preset deviation range; and, in response to being within the preset deviation range, generating second construction condition information, wherein the second construction condition information indicates that the construction condition of the construction site indicated by the construction site image is within the preset deviation range; In response to the comparison result indicating that the actual construction information does not match the standard construction information, and the construction conditions at the construction site indicated by the construction site image are outside a preset deviation range, third construction condition information is generated, wherein the third construction condition information indicates that the construction at the construction site indicated by the construction site image is abnormal.

7. A construction monitoring device, characterized in that: The device comprises: an acquisition unit configured to acquire a construction site image captured by an image acquisition device and standard construction information matching the construction site image, wherein the image acquisition device is a fisheye camera; an extraction unit configured to extract actual construction information contained in the construction site image; A comparison unit is configured to compare the actual construction information with the standard construction information to obtain a comparison result; A first generating unit is configured to generate construction situation information of the construction site indicated by the construction site image based on the comparison result; The actual construction information includes actual wiring location information and actual wiring type information, and the standard construction information includes a predetermined building information model including water and electricity wiring information; and The comparison unit comprises: An extraction subunit is configured to extract standard wiring position information and standard wiring type information from the building information model, and the pipeline trend in the building information model is linearized; The comparison subunit is configured to compare the actual wiring position information with the standard wiring position information, and to compare the actual wiring type information with the standard wiring type information to obtain a comparison result.

8. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute a computer program stored in the memory, and when the computer program is executed, implement the method described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method described in any one of claims 1 to 6 is implemented.

Images