Automatic modeling method based on space perception intelligent fusion

Abstract

The invention relates to an automatic modeling method based on spatial perception intelligent fusion, and relates to the technical field of spatial modeling, and the method comprises the steps: collecting laser detection information and image detection information; calling laser point cloud data corresponding to each detection position point based on the laser detection information; generating a laser space contour according to the laser point cloud data; generating space image information in combination with the laser point cloud data and the image detection information; generating modeling comprehensive information in combination with the laser space contour and the space image information; and performing modeling based on the modeling comprehensive information to generate modeling result information, and outputting the modeling result information. The method has the effect of improving the spatial modeling precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of space modeling, in particular to an automatic modeling method based on space perception intelligent fusion. BACKGROUND

[0002] Space modeling refers to a technology and method for constructing an abstract model based on geographic space data through mathematical, geometric, logical or statistical methods to describe, analyze, simulate or predict the shape, distribution, relationship and dynamic change process of geographic space entities, and is widely used in the fields of geographic information science (GIS), remote sensing, urban planning, environmental science, etc.

[0003] At present, when a home room is refitted, the length, width, height, wall thickness, door and window opening size and other sizes of the home room are generally measured by manual measurement to obtain space size data, and the collected space size data is processed to form a space model.

[0004] Since the space size data is generally obtained by manual measurement at present, the manual measurement is easily affected by reading deviation, improper measurement angle and other human operations, resulting in low collection accuracy and insufficient space modeling accuracy. SUMMARY

[0005] In order to improve the space modeling accuracy, the present application provides an automatic modeling method based on space perception intelligent fusion.

[0006] The present application provides an automatic modeling method based on space perception intelligent fusion, which adopts the following technical scheme: An automatic modeling method based on space perception intelligent fusion, comprising: S1: collecting laser detection information and image detection information; S2: based on the laser detection information, calling the laser point cloud data corresponding to each detection position point; S3: generating a laser space contour according to the laser point cloud data; S4: generating space image information in combination with the laser point cloud data and the image detection information; S5: generating modeling comprehensive information in combination with the laser space contour and the space image information; S6: modeling based on the modeling comprehensive information to generate modeling result information, and outputting the modeling result information.

[0007] By adopting the technical scheme, the laser detection information and the image detection information are collected in sequence, the laser point cloud data is called to generate the laser space contour, the laser point cloud data and the image detection information are fused to form the space image information, the modeling comprehensive information is generated based on the laser space contour and the space image information, and the modeling output is completed, the laser detection and the image detection are cooperatively linked, the complementary advantages of the two types of detection information are fully given, the integrity and the accuracy of the modeling comprehensive information are effectively improved, the modeling result information can comprehensively and accurately reflect the target space characteristics, the modeling deviation caused by single detection information is avoided, and the space modeling accuracy is improved.

[0008] Optionally, the space image information generation method comprises: S41: calling unit position image information corresponding to each detection position point based on the image detection information; S42: performing image recognition based on the unit position image information to obtain a unit position image type; S43: determining a detection distance value corresponding to each detection position point according to the laser point cloud data; S44: calculating a proportion value of adjacent detection distance values as an adjacent distance proportion value; S45: combining the unit position image information according to the adjacent distance proportion value and the unit position image type to form image comprehensive information, and taking the image comprehensive information as the space image information.

[0009] By adopting the technical scheme, the unit position image information is called and the unit position image type is recognized, the adjacent distance proportion value is calculated according to the detection distance value determined by the laser point cloud data, and then the unit position image information is combined to form the space image information, so that the distortion problem in the image collection process is effectively corrected, the adaptability of the space image information to the actual space is improved, and high-quality image data support is provided for the generation of subsequent modeling comprehensive information.

[0010] Optionally, the image comprehensive information formation method comprises: S451: adjusting the unit position image information according to the adjacent distance proportion value to obtain proportionally adjusted image information; S452: extracting the proportionally adjusted image information according to the unit position image type to obtain a same-type region image and a remaining region image; S453: calling an image actual brightness value based on the same-type region image; S454: selecting the same-type region image based on the image actual brightness value as a selected region image; S455: combining the selected region image and the remaining region image as the image comprehensive information.

[0011] By adopting the technical scheme, the proportional adjustment image information is obtained by adjusting the unit image information according to the adjacent distance proportional value, the same kind region image and the residual region image are extracted, the selected region image is selected based on the actual brightness value of the image and combined with the residual region image to form the image comprehensive information, the accurate screening and optimization of the image information are realized, the brightness consistency and effectiveness of the same kind region image in the image comprehensive information are ensured, the invalid or poor quality image part is eliminated, and the definition and reliability of the space image information are improved, thereby laying a high-quality image foundation for modeling comprehensive information generation.

[0012] Optionally, the selection method of the selected region image comprises: S4541: collecting an ambient light value; S4542: determining an image reference brightness value based on the ambient light value and the detection distance value; S4543: determining whether the actual brightness value of the image is greater than the image reference brightness value; S4544: if yes, calculating a difference value between the actual brightness value of the image and the image reference brightness value as a brightness deviation value; S4545: determining a deviation selection reference value based on the brightness deviation value and the unit image type; S4546: selecting the same kind region image corresponding to the larger value of the deviation selection reference value as the selected region image; S4547: if no, selecting the same kind region image corresponding to the larger value of the actual brightness value of the image as the selected region image.

[0013] By adopting the technical scheme, the proportional adjustment image information is obtained by adjusting the unit image information according to the adjacent distance proportional value, the same kind region image and the residual region image are extracted, the selected region image is selected based on the actual brightness value of the image and combined with the residual region image to form the image comprehensive information, the accurate screening and optimization of the image information are realized, the brightness consistency and effectiveness of the same kind region image in the image comprehensive information are ensured, the invalid or poor quality image part is eliminated, and the definition and reliability of the space image information are improved, thereby laying a high-quality image foundation for modeling comprehensive information generation.

[0014] Optionally, the determination method of the image reference brightness value comprises: S45421: collecting a device specification of an image acquisition device; S45422: determining a specification adaptive brightness interval according to the device specification; S45423: determining a distance light influence coefficient according to the detection distance value; S45424: calculating a product value between the ambient light value and the light influence coefficient as a distance adjustment brightness value; S45425: determining whether the distance-adjusted brightness value is located in the specification-adaptive brightness interval; S45426: if yes, taking the distance-adjusted brightness value as the image reference brightness value; S45427: if no, selecting the end value of the specification-adaptive brightness interval based on the image reference brightness value and taking the end value as the image reference brightness value.

[0015] By adopting the technical solution, the specification-adaptive brightness interval is determined through the specification of the acquisition device, the distance-adjusted brightness value is calculated in combination with the ambient light value and the distance light influence coefficient, the image reference brightness value is determined according to the relationship between the distance-adjusted brightness value and the specification-adaptive brightness interval, the image reference brightness value matches the performance parameters of the image acquisition device and fully considers the influence of the detection distance on the light, which provides a reliable basis for the accurate selection of the selected area image and further guarantees the accuracy of the spatial image information.

[0016] Optionally, the determination method of the deviation selection reference value comprises: S45451: determining the category deviation tolerance interval according to the unit position image category; S45452: determining whether the brightness deviation value falls into the category deviation tolerance interval; S45453: if yes, determining the category deviation coefficient according to the unit position image category; S45454: calculating the product value between the category deviation coefficient and the brightness deviation value and taking the product value as the deviation selection reference value; S45455: if no, selecting the end value of the category deviation tolerance interval based on the brightness deviation value to obtain an interval selection end value; S45456: calculating the ratio value between the brightness deviation value and the interval selection end value and taking the ratio value as the deviation selection reference value.

[0017] By adopting the technical solution, the category deviation tolerance interval is determined through the unit position image category, it is judged whether the brightness deviation value falls into the interval and the deviation selection reference value is calculated correspondingly, the determination of the deviation selection reference value is matched with the characteristic requirements of the unit position image category, the problem of insufficient adaptability of the selected area image caused by the uniform standard selection is avoided, and it is ensured that the selected selected area image meets the brightness deviation requirements and conforms to the characteristics of the corresponding image category, thereby improving the pertinence and effectiveness of the spatial image information.

[0018] Optionally, the generation method of the modeling comprehensive information comprises: S51: determining the image space contour and the image texture information according to the spatial image information; S52: determining the contour similarity in combination with the image space contour and the laser space contour; S53: determining whether the contour similarity is greater than a preset contour similarity reference value; S54: If yes, the image texture information is fused with the laser spatial profile to form modeling comprehensive information; S55: If no, the profile difference region and the profile consistent region are determined in combination with the image spatial profile and the laser spatial profile; S56: The difference region image and the adjacent region image are retrieved from the spatial image information based on the profile difference region; S57: The difference adjustment modeling information is generated in combination with the difference region image, the adjacent region image and the profile consistent region, and the difference adjustment modeling information is taken as the modeling comprehensive information.

[0019] By adopting the above technical solution, the image spatial profile and the image texture information are determined through the spatial image information, the profile similarity is calculated and the case is handled, the image texture information is fused with the laser spatial profile when the similarity meets the standard, the profile difference region and the profile consistent region are identified and the difference adjustment modeling information is generated when the similarity does not meet the standard, the intelligent adaptive fusion of the laser spatial profile and the image spatial profile is realized, the geometric precision advantage of the laser spatial profile is retained, the rich details of the image texture information are integrated, the problem of inconsistency between the two types of profiles is effectively solved, and the integrity and precision of the modeling comprehensive information are improved.

[0020] Optionally, the generation method of the difference adjustment modeling information comprises: S571: The image texture information is fused with the profile consistent region to obtain profile consistent modeling information; S572: The difference region category is identified based on the difference region image, and the adjacent region category is identified based on the adjacent region image; S573: The category deviation value is determined in combination with the difference region category and the adjacent region category; S574: When the category deviation value is less than a preset category reference deviation value, the category texture adjustment information is determined according to the adjacent region category; S575: The category texture adjustment information is fused with the profile difference region to obtain profile difference modeling information; S576: The profile consistent modeling information and the profile difference modeling information are combined to serve as the difference adjustment modeling information.

[0021] By adopting the above technical solution, when the category deviation value is small, the category texture adjustment information is determined based on the adjacent region category, the information is fused with the profile difference region to form the profile difference modeling information, and the profile consistent modeling information is combined to generate the difference adjustment modeling information, the texture coordination and adaptation of the profile difference region and the profile consistent region are realized, the modeling information is prevented from being broken or incoordination due to the profile difference, the coherence and rationality of the difference adjustment modeling information are ensured, and the overall quality of the modeling comprehensive information is improved.

[0022] Optionally, the method for generating the difference adjustment modeling information further comprises: S5741: When the category deviation value is not less than the preset category reference deviation value, determining the image distance value and the laser distance value according to the difference region image; S5742: Determining whether the image distance value is greater than the laser distance value; S5743: If yes, fusing the image texture information and the image space contour to form the difference adjustment modeling information; S5744: If no, calling the difference image brightness value based on the difference region image; S5745: Determining the contour change region in combination with the difference region category, the difference image brightness value and the category deviation value; S5746: Combining the image texture information with the contour change region as the difference adjustment modeling information.

[0023] By using the above technical solution, when the category deviation value is large, the image distance value is compared with the laser distance value, or the contour change region is determined in combination with the difference region category, the difference image brightness value and the category deviation value, the difference adjustment modeling information is generated in a targeted manner, the accurate processing of different category deviation scenes is realized, the modeling distortion problem caused by too large category deviation is effectively avoided, and it is ensured that the difference adjustment modeling information can accurately reflect the actual characteristics of the target space, and the reliability of the modeling result information is further improved.

[0024] In summary, the present application has at least one of the following beneficial technical effects: 1. By sequentially collecting laser detection information and image detection information, calling laser point cloud data to generate a laser space contour, fusing the laser point cloud data and the image detection information to form space image information, and then generating modeling comprehensive information based on the laser space contour and the space image information and completing modeling output, the laser detection and image detection are cooperatively linked, the complementary advantages of the two types of detection information are fully utilized, the integrity and accuracy of the modeling comprehensive information are effectively improved, and it is ensured that the modeling result information can fully and accurately reflect the characteristics of the target space, avoid the modeling deviation caused by single detection information, and improve the spatial modeling precision; 2. By calling unit image information and identifying the unit image category, calculating the adjacent distance proportion value in combination with the detection distance value determined by the laser point cloud data, and then combining the unit image information to form the space image information, the distortion problem in the image acquisition process is effectively corrected, the adaptability of the space image information to the actual space is improved, and high-quality image data support is provided for the generation of subsequent modeling comprehensive information; 3. Determine the image space contour and image texture information through the space image information, calculate the contour similarity and handle it in different cases, fuse the image texture information and the laser space contour when the similarity meets the standard, and identify the contour difference area and the contour consistent area and generate the difference adjustment modeling information when it does not meet the standard, realize the intelligent adaptive fusion of the laser space contour and the image space contour, retain the geometric precision advantage of the laser space contour, and also integrate the rich details of the image texture information, effectively solve the inconsistency problem of the two types of contours, and improve the integrity and precision of the modeling comprehensive information. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a method flow chart of automatic modeling based on spatial perception intelligent fusion; Figure 2 is a method flow chart of generating space image information; Figure 3 is a method flow chart of generating modeling comprehensive information. DETAILED DESCRIPTION

[0026] The application will be further described in detail below in combination with the drawings and examples.

[0027] An automatic modeling method based on spatial perception intelligent fusion, by first collecting laser detection information and image detection information, generating a laser space contour through laser point cloud data, and combining detection distance, image type, brightness, and device specifications, environmental light value to optimize the generation of space image information; then, according to the contour similarity, the laser space contour and the image space contour, and the texture information are fused, the contour difference area is adapted and processed according to the type deviation and distance, the modeling comprehensive information is generated and the modeling result is output, so as to realize the collaborative linkage of laser detection and image detection, and improve the spatial modeling precision.

[0028] Reference Figure 1 , the embodiment of the application discloses an automatic modeling method based on spatial perception intelligent fusion, which comprises: S1: collecting laser detection information and image detection information.

[0029] Among them, the laser detection information refers to the collection of data containing three-dimensional geometric position (such as X, Y, Z coordinates), reflection intensity, etc. through the active emission of laser beams by laser acquisition equipment at different positions, and the reflection of target space object surface. The image detection information refers to the image information collected by the image acquisition equipment at different positions.

[0030] The laser acquisition equipment can be a modeling handheld instrument, a laser module of a depth perception camera, etc. The image acquisition equipment can be a vision module of a depth perception camera. The laser acquisition equipment and the image acquisition equipment can be integrated on the same handheld instrument, or can be integrated on a device such as a unmanned aerial vehicle, so as to facilitate synchronous detection.

[0031] S2: retrieve the laser point cloud data corresponding to each detection position point based on the laser detection information.

[0032] The detection position point refers to a specific sampling point position corresponding to the scanning process of the target space by the laser collection device and the image collection device. The laser point cloud data refers to the geometric position, reflection intensity, and other data collected by the laser collection device. The laser detection information includes the detection position point and the corresponding laser point cloud data.

[0033] The detection position point and the laser point cloud data are retrieved through the laser detection information, which facilitates subsequent use.

[0034] S3: generate a laser space contour based on the laser point cloud data.

[0035] The laser space contour refers to the three-dimensional geometric boundary form of the target space and the object formed according to the laser collection situation.

[0036] After preprocessing the laser point cloud data by removing noise points and filling sparse areas, the point cloud data is divided into different point cloud subsets corresponding to different objects / regions (such as point cloud groups of walls, furniture, and shelves) by a point cloud segmentation algorithm according to spatial continuity and reflection intensity differences. Then, the three-dimensional boundary of the object is constructed by edge extraction and contour fitting for each point cloud subset, forming a contour and serving as the laser space contour.

[0037] S4: generate space image information by combining the laser point cloud data and the image detection information.

[0038] The space image information refers to the image data set of the entire target space.

[0039] The space image information is obtained by stitching each image in the image detection information based on the laser point cloud data, which facilitates subsequent use.

[0040] In order to further ensure the rationality of the space image information, it is necessary to make a further separate analysis and calculation of the space image information. The specific steps are as follows.

[0041] Referring to Figure 2 , the method for generating space image information includes the following steps: S41: retrieve the unit position image information corresponding to each detection position point based on the image detection information.

[0042] The unit position image information refers to the image information corresponding to each detection position point. The image detection information includes the detection position point and the corresponding unit position image information.

[0043] The image detection information is used to retrieve the single-location image information corresponding to the detection location point, which is convenient for subsequent use.

[0044] S42: Perform image recognition based on single-location image information to obtain single-location image types.

[0045] Among them, single-location image category refers to the category to which an object belongs in an image at a single detection location point.

[0046] By performing image recognition on single-location image information and using the identified item types as single-location image types, it is convenient for subsequent use.

[0047] S43: Determine the detection distance value corresponding to each detection location point based on the laser point cloud data.

[0048] The detection distance value refers to the distance between the spatial location detected by the laser and the detection location point.

[0049] The spatial location is retrieved by laser point cloud data, and the average distance between the location and the corresponding detection location is calculated as the detection distance value for subsequent use.

[0050] S44: Calculate the ratio of adjacent detection distance values ​​and use it as the adjacent distance ratio.

[0051] The adjacent distance ratio refers to the ratio between the detection distance values ​​corresponding to adjacent detection location points.

[0052] The ratio between the detection distance values ​​corresponding to adjacent detection locations is calculated, and the calculation result is used as the adjacent distance ratio for convenient subsequent use.

[0053] S45: Based on the ratio of adjacent distances and the type of single-location image, combine the single-location image information to form comprehensive image information, and use the comprehensive image information as spatial image information.

[0054] Image composite information refers to the image set data corresponding to the composite stitching of images from various locations.

[0055] By combining the ratio of adjacent distances with the types of single-location images, the individual images in the single-location image information are stitched together, and the stitched image is used as comprehensive image information. This comprehensive image information is then used as spatial image information, thereby improving the accuracy of the acquired spatial image information.

[0056] To further ensure the rationality of the image synthesis information, it is necessary to perform further separate analysis and calculation on the image synthesis information, which will be explained in detail through the following steps.

[0057] The method for forming image composite information includes the following steps: S451: Adjust the image information at a single location based on the ratio of adjacent distances to obtain proportionally adjusted image information.

[0058] Among them, the scale-adjusted image information refers to the image information corresponding to the scale adjustment of the single-position image information.

[0059] By adjusting the image information at a single location according to the ratio of adjacent distances, proportionally adjusted image information with consistent image proportions is obtained, which facilitates subsequent use.

[0060] S452: Based on the image type at a single location, adjust the image information proportionally to extract the image information to obtain images of the same type of region and the remaining region images.

[0061] Among them, images of the same type of region refer to images of the corresponding region of the same single location (such as walls, doors and windows, shelves, goods, etc.). Remaining region images refer to the corresponding images remaining after removing images of the same type of region from the scaled image information.

[0062] By extracting images of the same type from the scaled image information according to the single-location image type, images of the same type of region are obtained, and the remaining images are used as the remaining region images for convenient subsequent use.

[0063] S453: Retrieve the actual brightness value of the image based on images of the same type of region.

[0064] The actual brightness value of an image refers to the quantized data of the true brightness of each pixel or region in an image of the same type.

[0065] The brightness conversion formula is used to calculate the brightness value of each pixel in the same type of image region, and then the average brightness of the region is calculated as the actual brightness value of the image. The brightness conversion formula is existing technology and will not be described in detail.

[0066] S454: Select images of the same type based on the actual brightness value of the image and use them as the selected area images.

[0067] Among them, the selected region image refers to the image corresponding to the selected region images of the same type.

[0068] By selecting images of the same type of region according to their actual brightness values, and using the selected images as the selected region images, it is convenient for subsequent use.

[0069] To further ensure the rationality of the selected region image, it is necessary to perform further separate analysis and calculation on the selected region image, which will be explained in detail through the steps shown below.

[0070] The selection method of the selected area image comprises the following steps: S4541: Collect the ambient light value.

[0071] The ambient light value refers to the quantitative data of the light intensity actually existing in the environment of the target space.

[0072] The ambient light value is detected and obtained by a light sensor pre-installed on the laser collection device or the image collection device.

[0073] S4542: Determine the image reference brightness value in combination with the ambient light value and the detection distance value.

[0074] The image reference brightness value refers to the reference standard for judging whether the brightness of the same type of area image meets the standard.

[0075] The image reference brightness value is determined by combining and analyzing the ambient light value and the detection distance value, which facilitates subsequent use.

[0076] In order to further ensure the rationality of the image reference brightness value, it is necessary to make further separate analysis and calculation on the image reference brightness value, which will be described in detail through the following steps.

[0077] The determination method of the image reference brightness value comprises the following steps: S45421: Collect the device specifications of the image collection device.

[0078] The device specifications refer to the core technical parameter set of the image collection device. The device specifications include sensor size, ISO range, aperture size, exposure time adjustment range, image resolution, supported image format, etc.

[0079] The device specifications can be obtained by pre-input by the operator, or by querying the nameplate or built-in storage data of the image collection device.

[0080] S45422: Determine the specification adaptive brightness interval according to the device specifications.

[0081] The specification adaptive brightness interval refers to the brightness numerical range that allows the image collection device to stably collect clear, low-noise, and non-overexposure / underexposure images.

[0082] The specification adaptive brightness interval is matched by inputting the device specifications into the pre-set specification database, which facilitates subsequent use.

[0083] The specification database pre-stores a comparison table of different device specifications and corresponding specification adaptive brightness intervals. The specification database is obtained by pre-input by the operator by querying the manufacturer's technical manual of different device specifications.

[0084] S45423: determining the distance illumination influence coefficient according to the detection distance value.

[0085] The distance illumination influence coefficient refers to a dimensionless coefficient quantifying the effect of the detection distance value on the ambient light intensity decay.

[0086] The detection distance value is input into a preset distance illumination influence database to match the distance illumination influence coefficient, which is convenient for subsequent use.

[0087] The distance illumination influence database pre-stores a control table of different detection distance values and corresponding distance illumination influence coefficients, and the distance illumination influence database is obtained by an operator detecting actual illumination values of different detection distance values and calculating coefficients between the preset theoretical output illumination values.

[0088] S45424: calculating the product value between the ambient illumination value and the illumination influence coefficient as the distance-adjusted brightness value.

[0089] The distance-adjusted brightness value refers to the brightness value corresponding to the illumination intensity decay of the detection distance value.

[0090] The product value between the ambient illumination value and the illumination influence coefficient is calculated, and the calculation result is used as the distance-adjusted brightness value, which is convenient for subsequent use.

[0091] S45425: determining whether the distance-adjusted brightness value is within the specification-adaptive brightness interval. If yes, S45426 is executed; if no, S45427 is executed.

[0092] The distance-adjusted brightness value is determined to be within the specification-adaptive brightness interval, so as to determine whether the distance-adjusted brightness value can be directly used.

[0093] S45426: taking the distance-adjusted brightness value as the image reference brightness value.

[0094] When the distance-adjusted brightness value is within the specification-adaptive brightness interval, it means that the distance-adjusted brightness value can be directly used, so the distance-adjusted brightness value is taken as the image reference brightness value, improving the accuracy of the obtained image reference brightness value.

[0095] S45427: selecting the end value of the specification-adaptive brightness interval based on the image reference brightness value as the image reference brightness value.

[0096] When the distance-adjusted brightness value is not located in the specification-adapted brightness interval, it is not directly used, and the difference between the two end values of the specification-adapted brightness interval and the image reference brightness value is calculated, and the end value with smaller difference is selected as the image reference brightness value for subsequent use.

[0097] S4543: Determine whether the image actual brightness values are all greater than the image reference brightness value. If yes, perform S4544; if no, perform S4547.

[0098] The image actual brightness values are all greater than the image reference brightness value, and whether the brightness condition can be directly selected is determined by judging whether the image actual brightness values are all greater than the image reference brightness value.

[0099] S4544: Calculate the difference between the image actual brightness value and the image reference brightness value as the brightness deviation value.

[0100] The brightness deviation value refers to the deviation value corresponding to the brightness deviation.

[0101] When the image actual brightness values are all greater than the image reference brightness value, the brightness condition cannot be directly selected, and the difference between the image actual brightness value and the image reference brightness value is calculated, and the calculation result is used as the brightness deviation value for subsequent use.

[0102] S4545: Determine the deviation selection reference value by combining the brightness deviation value and the unit image category.

[0103] The deviation selection reference value refers to the reference value corresponding to the selection according to the brightness and the category.

[0104] The brightness deviation value and the unit image category are combined and analyzed to determine the deviation selection reference value for subsequent use.

[0105] To further ensure the rationality of the deviation selection reference value, the deviation selection reference value needs to be further analyzed and calculated separately, which is specifically described as follows.

[0106] The determination method of the deviation selection reference value includes the following steps: S45451: Determine the category deviation tolerance interval according to the unit image category.

[0107] The category deviation tolerance interval refers to the allowed range of brightness deviation for a specific unit image category (such as wall, glass, furniture, shelf, etc.).

[0108] The unit image category is input into a preset category database to match the category deviation allowable range, facilitating subsequent use.

[0109] The category database pre-stores a comparison table of different unit image categories and corresponding category deviation allowable ranges, which is obtained by an operator combining industry modeling accuracy standards and a large number of collected experimental statistics reasonable deviation ranges.

[0110] For example, the category database can be set as: when the unit image category is a wall, the corresponding category deviation allowable range is [-20, +20]; when the unit image category is glass, the corresponding category deviation allowable range is [-50, +50]; and when the unit image category is a decorative picture, the corresponding category deviation allowable range is [-30, +30].

[0111] S45452: Determine whether the luminance deviation value falls within the category deviation allowable range. If yes, perform S45453; if no, perform S45455.

[0112] The luminance deviation value is determined by judging whether it falls within the category deviation allowable range, so as to select different ways to determine the deviation selection reference value.

[0113] S45453: Determine the category deviation coefficient according to the unit image category.

[0114] The category deviation coefficient is a dimensionless weighting coefficient set for a specific unit image category (such as a wall, glass, a shelf, and goods). Different unit image categories correspond to different light reflection characteristics, so the sensitivity of the corresponding image to luminance deviation is different, and therefore different unit image categories correspond to different category deviation coefficients.

[0115] When the luminance deviation value falls within the category deviation allowable range, the unit image category is input into a preset category database to match the category deviation coefficient, facilitating subsequent use.

[0116] The category database pre-stores a comparison table of different unit image categories and corresponding category deviation coefficients, which is pre-set by an operator according to requirements.

[0117] For example, when the unit image category is a wall, the corresponding category deviation coefficient is 1.3; when the unit image category is glass, the corresponding category deviation coefficient is 1.0; and when the unit image category is a decorative picture, the corresponding category deviation coefficient is 0.7.

[0118] S45454: Calculate the product value between the category deviation coefficient and the luminance deviation value as the deviation selection reference value.

[0119] The product value between the category deviation coefficient and the brightness deviation value is calculated, and the calculation result is taken as the deviation selection reference value, thereby improving the accuracy of the obtained deviation selection reference value.

[0120] S45455: The end value of the category deviation tolerance interval is selected based on the brightness deviation value, and an interval selection end value is obtained.

[0121] The interval selection end value refers to the end value corresponding to the selection of the end value of the category deviation tolerance interval.

[0122] When the brightness deviation value does not fall within the category deviation tolerance interval, the difference between the two end values of the category deviation tolerance interval and the brightness deviation value is calculated, and the end value corresponding to the smaller difference value is selected as the interval selection end value, which is convenient for subsequent use.

[0123] S45456: The ratio value between the brightness deviation value and the interval selection end value is calculated and taken as the deviation selection reference value.

[0124] The ratio value between the brightness deviation value and the interval selection end value is calculated, and the calculation result is taken as the deviation selection reference value, thereby improving the accuracy of the obtained deviation selection reference value.

[0125] S4546: The same category region image corresponding to the larger value of the deviation selection reference value is selected as the selected region image.

[0126] The same category region image corresponding to the larger value of the deviation selection reference value is selected, and the selected image is taken as the selected region image, thereby improving the accuracy of the obtained selected region image.

[0127] S4547: The same category region image corresponding to the larger value of the image actual brightness value is selected as the selected region image.

[0128] When the image actual brightness value is not greater than the image reference brightness value, it means that the selection can be directly based on the brightness at this time, so the same category region image corresponding to the larger value of the image actual brightness value is selected, and the selected image is taken as the selected region image, thereby improving the accuracy of the obtained selected region image.

[0129] S455: The selected region image and the remaining region image are combined as image comprehensive information.

[0130] The selected area image and the remaining area image are spliced and fused after space coordinate alignment and boundary smoothing processing, so as to form the final image comprehensive information and improve the accuracy of the obtained image comprehensive information.

[0131] S5: generating modeling comprehensive information by combining the laser space profile and the space image information.

[0132] The modeling comprehensive information refers to a standardized modeling core data set formed by fusing the three-dimensional geometric precision of the laser space profile and the space image.

[0133] The laser space profile and the space image information are aligned in the same three-dimensional space and fused, so as to generate the modeling comprehensive information for subsequent use.

[0134] In order to further ensure the rationality of the modeling comprehensive information, the modeling comprehensive information needs to be further analyzed and calculated separately. The specific steps are as follows.

[0135] Referring to Figure 3 , the modeling comprehensive information generation method comprises the following steps: S51: determining image space profile and image texture information according to space image information.

[0136] The image space profile refers to two-dimensional or three-dimensional profile data reflecting the boundaries of various objects (such as walls, shelves, goods, etc.) in the target space. The image texture information refers to a set of visual feature data of the object surface in the image. The image texture information includes texture structure (such as wall grain, shelf metal lines), color distribution (such as goods packaging color, wall paint color), light and dark details, etc.

[0137] The space image information is preprocessed, and a semantic segmentation algorithm (such as U-Net network) is used to label the object type label of each pixel in the space image. Then, an edge detection algorithm (such as Canny algorithm) is used to extract the boundary profile of the same label pixels. After coordinate calibration, the image space profile corresponding to the actual space position is formed. Then, for each object region divided by the image space profile, a feature extraction algorithm (such as gray level co-occurrence matrix, color histogram) is used to extract the texture structure, color distribution, etc. in the region. The image texture information is integrated by combining the pixel brightness value and the RGB channel information for subsequent use.

[0138] S52: determining profile similarity by combining the image space profile and the laser space profile.

[0139] The profile similarity refers to a dimensionless index quantifying the geometric matching degree of the image space profile and the laser space profile.

[0140] By putting the image space contour and the laser space contour in the same space reference, the coordinate deviation influence is eliminated, and then the key points (such as the vertex, the inflection point, the circular arc center) of the two types of contours, the contour line parameters (such as the length of the line segment, the curvature of the curve, the angle relationship), the region topological structure (such as the relative position between objects) and other core features are extracted respectively, and then the key point matching rate, the shape similarity and other indicators are calculated, and finally the contour similarity is calculated by weighting, which is convenient for subsequent use. The specific weight of the key point matching rate, the shape similarity and other indicators is set by the operator according to the actual demand in advance.

[0141] S53: Determine whether the contour similarity is greater than a preset contour similarity reference value. If yes, execute S54; if no, execute S55.

[0142] The contour similarity reference value refers to a threshold value for defining that the image space contour and the laser space contour are matched and qualified. The contour similarity reference value is obtained by pre-inputting by the operator.

[0143] By judging whether the contour similarity is greater than the preset contour similarity reference value, it is determined whether the image texture information or the laser space contour needs to be adjusted.

[0144] S54: Fuse the image texture information and the laser space contour to form modeling comprehensive information.

[0145] When the contour similarity is greater than the preset contour similarity reference value, it is indicated that the image texture information or the laser space contour does not need to be adjusted at this time, so the pixel coordinates of the image texture information are directly converted to the three-dimensional coordinate system of the laser space contour, and the texture is covered according to the contour, thereby forming the modeling comprehensive information, and improving the accuracy of the obtained modeling comprehensive information.

[0146] S55: Determine the contour difference region and the contour consistent region in combination with the image space contour and the laser space contour.

[0147] The contour difference region refers to the contour region corresponding to the difference between the image space contour and the laser space contour. The contour consistent region refers to the contour region corresponding to the consistency between the image space contour and the laser space contour.

[0148] When the contour similarity is greater than the preset contour similarity reference value, it is indicated that the image texture information or the laser space contour does not need to be adjusted at this time, so the image space contour and the laser space contour are compared, the region with the difference is taken as the contour difference region, and the region consistent is taken as the contour consistent region, which is convenient for subsequent use.

[0149] S56: retrieve a distinguished region image and a neighboring region image from the spatial image information based on the contour distinguished region.

[0150] The distinguished region image refers to an image corresponding to the contour distinguished region, and the neighboring region image refers to an image corresponding to a region adjacent to the contour distinguished region.

[0151] The image corresponding to the contour distinguished region is retrieved from the spatial image information as the distinguished region image, and the image adjacent to the distinguished region image is retrieved as the neighboring region image, facilitating subsequent use.

[0152] S57: generate distinguished adjustment modeling information by combining the distinguished region image, the neighboring region image, and the contour consistent region, and use the distinguished adjustment modeling information as modeling comprehensive information.

[0153] The distinguished adjustment modeling information refers to a modeling core data set corresponding to the contour after adjustment according to the distinguished situation.

[0154] The distinguished adjustment modeling information is generated by combining the distinguished region image, the neighboring region image, and the contour consistent region, thereby improving the accuracy of the modeling comprehensive information obtained.

[0155] In order to further ensure the rationality of the distinguished adjustment modeling information, it is necessary to make further separate analysis and calculation on the distinguished adjustment modeling information. The specific steps are as follows.

[0156] The generation method of the distinguished adjustment modeling information includes the following steps: S571: fuse the image texture information with the contour consistent region to obtain contour consistent modeling information.

[0157] The contour consistent modeling information refers to a modeling core data set corresponding to the contour consistent region after texture fusion.

[0158] The pixel coordinates of the image texture information are converted to the three-dimensional coordinate system of the contour consistent region, and the texture is covered according to the contour, thereby forming the contour consistent modeling information, facilitating subsequent use.

[0159] S572: identify the distinguished region category based on the distinguished region image, and identify the neighboring region category based on the neighboring region image.

[0160] The distinguished region category refers to the category of the object in the distinguished region image, and the neighboring region category refers to the category of the object in the neighboring region image.

[0161] The object category of the distinguished area image is recognized to obtain the distinguished area category, and the object category of the adjacent area image is recognized to obtain the adjacent area category, facilitating subsequent use.

[0162] S573: Determine the category deviation value by combining the distinguished area category and the adjacent area category.

[0163] The category deviation value is a quantitative index that quantifies the difference between the distinguished area category and the adjacent area category in core characteristics (reflectivity, light transmittance, texture complexity, material density).

[0164] The distinguished area category and the adjacent area category are input into the preset category deviation database to match the category deviation value, facilitating subsequent use.

[0165] The category deviation database pre-stores a comparison table of different distinguished area categories, adjacent area categories, and corresponding category deviation values, and the category deviation database is pre-set by an operator according to actual needs.

[0166] For example, the category deviation database can set the corresponding category deviation value when the distinguished area category and the adjacent area category are glass and wall respectively as 2, the corresponding category deviation value when they are decorative painting and wall respectively as 1, and the corresponding category deviation value when they are glass and decorative painting respectively as 3.

[0167] S574: When the category deviation value is less than the preset category reference deviation value, determine the category texture adjustment information according to the adjacent area category.

[0168] The category reference deviation value is a deviation value corresponding to only texture adjustment. The category reference deviation value is obtained by pre-input by an operator, and in this embodiment, the category reference deviation value can be set to 1.

[0169] When the category deviation value is less than the preset category reference deviation value, it means that the distinguished area category and the adjacent area category are of the same category at this time, so the adjacent area category is input into the preset category texture database to match the category texture adjustment information, facilitating subsequent use.

[0170] The category texture database pre-stores a comparison table of different adjacent area categories and corresponding category texture adjustment information, and the category texture database is obtained by pre-input by an operator.

[0171] S575: Fuse the category texture adjustment information with the contour distinguished area to obtain contour distinguished modeling information.

[0172] The contour distinguished modeling information is a set of modeling core data corresponding to the texture fusion of the contour distinguished area.

[0173] The contour-distinguishing modeling information is obtained by converting the pixel coordinates of the category texture adjustment information to a three-dimensional coordinate system of the contour-distinguishing region and performing texture overlay according to the contour, so as to facilitate subsequent use.

[0174] S576: The contour-consistent modeling information and the contour-distinguishing modeling information are combined to serve as the distinguishing adjustment modeling information.

[0175] The contour-consistent modeling information and the contour-distinguishing modeling information are combined to form the modeling core data set of the overall contour and serve as the distinguishing adjustment modeling information, thereby improving the accuracy of the obtained distinguishing adjustment modeling information.

[0176] In order to further ensure the rationality of the distinguishing adjustment modeling information, further separate analysis and calculation of the distinguishing adjustment modeling information are required, which are specifically described as follows.

[0177] The method for generating the distinguishing adjustment modeling information further includes the following steps: S5741: When the category deviation value is not less than the preset category reference deviation value, the image distance value and the laser distance value are determined according to the distinguishing region image.

[0178] The image distance value refers to the distance value between the position collected by the image and the detection position point. The laser distance value refers to the distance value between the position collected by the laser and the detection position point.

[0179] When the category deviation value is not less than the preset category reference deviation value, it indicates that the category of the distinguishing region and the category of the adjacent region are not the same category, so the corresponding laser point cloud data is retrieved from the distinguishing region image to obtain the laser distance value, and the relative distance between each pixel point and the device is calculated according to the perspective law of the image acquisition device (shooting posture), and the image distance value is taken as the average value of the region, thereby facilitating subsequent use.

[0180] S5742: It is determined whether the image distance value is greater than the laser distance value. If yes, S5743 is performed; if no, S5744 is performed.

[0181] It is determined whether the image distance value is greater than the laser distance value, so as to determine whether the image captures the wall / furniture behind the glass through the glass, and the laser captures the glass.

[0182] S5743: The image texture information is fused with the image space contour to form the distinguishing adjustment modeling information.

[0183] When the image distance value is greater than the laser distance value, it indicates that the image captures the wall / furniture behind the glass, while the laser captures the glass. The only difference at this time is caused by the laser being affected by the glass. By converting the pixel coordinates of the image texture information to the three-dimensional coordinate system of the image space contour and performing texture covering according to the contour, the difference adjustment modeling information is formed, and the accuracy of the obtained difference adjustment modeling information is improved.

[0184] S5744: Obtain a difference image brightness value based on the difference area image.

[0185] The difference image brightness value refers to the quantized brightness data in the difference area image.

[0186] When the image distance value is not greater than the laser distance value, it indicates that the image does not capture the wall / furniture behind the glass, while the laser captures the glass. Therefore, the difference image brightness value is obtained through the difference area image, which is convenient for subsequent use.

[0187] S5745: Determine a contour change area based on the difference area type, the difference image brightness value, and the type deviation value.

[0188] The contour change area refers to the area corresponding to the contour after the change adjustment.

[0189] The difference area type, the difference image brightness value, and the type deviation value are input into the contour expansion database to match the contour expansion value, and the contour difference area is expanded according to the contour expansion value. At this time, the contour consistent area is reduced, and the contour area corresponding to the overall contour after the adjustment is taken as the contour change area, which is convenient for subsequent use.

[0190] The contour expansion database pre-stores a comparison table of different difference area types, difference image brightness values, type deviation values, and corresponding contour expansion values. The contour expansion database obtains the average distance value that can expand the contour after testing different difference area types according to different difference image brightness values and different type deviation values, and takes it as the contour expansion value.

[0191] S5746: Combine the image texture information and the contour change area as difference adjustment modeling information.

[0192] The pixel coordinates of the image texture information are converted to the three-dimensional coordinate system of the contour change area, and texture covering is performed according to the contour, so as to form the difference adjustment modeling information, and improve the accuracy of the obtained difference adjustment modeling information.

[0193] S6: Model based on modeling comprehensive information to generate modeling result information, and output the modeling result information.

[0194] The modeling result information refers to a standardized three-dimensional model data set that completely restores the physical form of the target space and surface details.

[0195] The modeling comprehensive information is input into a preset three-dimensional modeling system to perform three-dimensional modeling, so as to obtain the modeling result information, and the modeling result information is output, thereby realizing the cooperative linkage of laser detection and image detection and improving the space modeling precision.

[0196] Based on the same inventive concept, the present application provides an automatic modeling system based on intelligent fusion of space perception, comprising: The acquisition module is configured to acquire laser detection information, image detection information, ambient light values, and device specifications. The memory stores a program for implementing the automatic modeling method based on intelligent fusion of space perception. The processor loads and executes the program stored in the memory.

[0197] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0198] The above is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solution falling within the concept of the present application shall be deemed to fall within the protection scope of the present application. It should be noted that, for ordinary skilled persons in the art, some improvements and refinements without departing from the principles of the present application shall also be deemed to fall within the protection scope of the present application.

Claims

1. An automated modeling method based on spatial perception intelligent fusion, characterized in that, The method comprises the following steps: S1: collecting laser detection information and image detection information; S2: calling laser point cloud data corresponding to each detection position point based on the laser detection information; S3: generating a laser space profile according to the laser point cloud data; S4: generating space image information by combining the laser point cloud data and the image detection information; S5: generating modeling comprehensive information by combining the laser space profile and the space image information; S6: modeling based on the modeling comprehensive information to generate modeling result information and output the modeling result information. 2.The automatic modeling method based on spatial perception intelligent fusion according to claim 1, wherein, The method for generating space image information comprises the following steps: S41: calling unit position image information corresponding to each detection position point based on the image detection information; S42: performing image recognition based on the unit position image information to obtain unit position image categories; S43: determining detection distance values corresponding to each detection position point according to the laser point cloud data; S44: calculating the proportion value of adjacent detection distance values as adjacent distance proportion values; S45: combining unit position image information according to adjacent distance proportion values and unit position image categories to form image comprehensive information, and taking the image comprehensive information as space image information.

3. The automated modeling method based on spatial perception intelligent fusion according to claim 2, characterized in that, The method for forming image comprehensive information comprises the following steps: S451: adjusting unit position image information according to adjacent distance proportion values to obtain proportionally adjusted image information; S452: extracting proportionally adjusted image information according to unit position image categories to obtain same category region image and remaining region image; S453: calling image actual brightness values based on the same category region image; S454: selecting the same category region image as selected region image based on the image actual brightness values; S455: combining the selected region image and the remaining region image as image comprehensive information.

4. The automatic modeling method based on spatial perception intelligent fusion according to claim 3, characterized in that, The method for selecting the selected region image comprises the following steps: S4541: collecting ambient light value; S4542: determining image reference brightness value by combining the ambient light value and the detection distance value; S4543: determining whether the image actual brightness value is greater than the image reference brightness value; S4544: if yes, calculating the difference value between the image actual brightness value and the image reference brightness value as brightness deviation value; S4545: determining deviation selection reference value by combining the brightness deviation value and the unit position image category; S4546: selecting the same category region image corresponding to the larger value of the deviation selection reference value as the selected region image; S4547: if no, selecting the same category region image corresponding to the larger value of the image actual brightness value as the selected region image.

5. The automated modeling method based on spatial perception intelligent fusion according to claim 4, characterized in that, The method for determining the image reference brightness value comprises the following steps: S45421: collecting the equipment specifications of the image acquisition equipment; S45422: determining specification adaptive brightness interval according to the equipment specifications; S45423: determining distance light influence coefficient according to the detection distance value; S45424: calculating the product value between the ambient light value and the light influence coefficient as distance adjusted brightness value; S45425: determining whether the distance adjusted brightness value is located in the specification adaptive brightness interval; S45426: if yes, taking the distance adjusted brightness value as the image reference brightness value; S45427: If no, then the end value of the specification adaptive luminance interval is selected based on the image reference luminance value and taken as the image reference luminance value.

6. The automated modeling method based on spatial perception intelligent fusion according to claim 4, characterized in that, The method for determining the deviation selection reference value comprises: S45451: determining a category deviation allowable interval according to the unit position image category; S45452: determining whether the luminance deviation value falls within the category deviation allowable interval; S45453: if yes, then determining a category deviation coefficient according to the unit position image category; S45454: calculating the product value between the category deviation coefficient and the luminance deviation value and taking the product value as the deviation selection reference value; S45455: if no, then selecting the end value of the category deviation allowable interval based on the luminance deviation value to obtain an interval selection end value; S45456: calculating the ratio value between the luminance deviation value and the interval selection end value and taking the ratio value as the deviation selection reference value.

7. The automated modeling method based on spatial perception intelligent fusion according to claim 1, characterized in that, The method for generating the modeling comprehensive information comprises: S51: determining image space contour and image texture information according to the space image information; S52: determining contour similarity by combining the image space contour and the laser space contour; S53: determining whether the contour similarity is greater than a preset contour similarity reference value; S54: if yes, then fusing the image texture information and the laser space contour to form the modeling comprehensive information; S55: if no, then determining contour difference region and contour consistent region by combining the image space contour and the laser space contour; S56: based on the contour difference region, calling the difference region image and the adjacent region image from the space image information; S57: generating the difference adjustment modeling information by combining the difference region image, the adjacent region image and the contour consistent region, and taking the difference adjustment modeling information as the modeling comprehensive information. 8.The automatic modeling method based on spatial perception intelligent fusion according to claim 7, wherein, The method for generating the difference adjustment modeling information comprises: S571: fusing the image texture information and the contour consistent region to obtain contour consistent modeling information; S572: identifying the difference region category based on the difference region image and identifying the adjacent region category based on the adjacent region image; S573: determining a category deviation value by combining the difference region category and the adjacent region category; S574: when the category deviation value is less than a preset category reference deviation value, determining category texture adjustment information according to the adjacent region category; S575: fusing the category texture adjustment information and the contour difference region to obtain contour difference modeling information; S576: combining the contour consistent modeling information and the contour difference modeling information to take the combination as the difference adjustment modeling information.

9. The automatic modeling method based on spatial perception intelligent fusion according to claim 8, characterized in that, The method for generating the difference adjustment modeling information further comprises: S5741: when the category deviation value is not less than the preset category reference deviation value, determining an image distance value and a laser distance value according to the difference region image; S5742: determining whether the image distance value is greater than the laser distance value; S5743: if yes, then fusing the image texture information and the image space contour to form the difference adjustment modeling information; S5744: if no, then calling a difference image luminance value based on the difference region image; S5745: determining a contour change region by combining the difference region category, the difference image luminance value and the category deviation value; S5746: combine image texture information with the contour change region and use as a distinct adjustment modeling information.

Images