Method, system and equipment for measuring available volume of refrigeration equipment and storage medium

Abstract

The invention discloses a method, a system and equipment for measuring the available volume of refrigeration equipment and a storage medium, and relates to the field of refrigeration equipment control, and the method comprises the following steps: obtaining ultrasonic measurement data and a visual image in the refrigeration equipment, and carrying out segmentation and feature extraction on the visual image. Then, constructing a preliminary ultrasonic three-dimensional point cloud model based on ultrasonic measurement data, and supplementing and correcting the preliminary ultrasonic three-dimensional point cloud model by using the extracted image feature data to obtain a target ultrasonic three-dimensional point cloud model; meanwhile, a target visual three-dimensional model is constructed based on image feature data. And finally, fusing and reconstructing the target visual three-dimensional model and the target ultrasonic three-dimensional point cloud model to obtain a target three-dimensional model which accurately reflects the internal structure of the refrigeration equipment, and accurately calculating the available volume of the refrigeration equipment based on the model. The available volume of the refrigeration equipment is accurately evaluated through the method.

Description

Technical Field

[0001] The present application relates to the field of refrigeration equipment control, and in particular to a method, system, device and storage medium for measuring the available volume of refrigeration equipment. Background Art

[0002] With the advancement of science and technology and the improvement of people's living standards, refrigeration equipment has become an indispensable household appliance in modern families. People's requirements for refrigeration equipment are no longer limited to basic food storage and preservation functions. Intelligent operation and convenience have also become important directions for the development of refrigeration equipment. In order to meet the growing needs of users, many refrigeration equipment manufacturers have made intelligence the focus of research and development, and are committed to providing more humane and intelligent refrigeration equipment products to improve users' experience and quality of life.

[0003] In the daily use of refrigeration equipment, users often need to understand the usage of the internal space of the refrigeration equipment in order to rationally plan the storage and purchase of food. The traditional way is to estimate the available volume of the refrigeration equipment by opening the door of the refrigeration equipment and observing it with the naked eye. However, this method has many inconveniences and limitations. First, the user needs to go to the vicinity of the refrigeration equipment in person to operate it, and it is impossible to obtain the volume information of the refrigeration equipment remotely. Secondly, frequent opening and closing of the refrigeration equipment door will cause the loss of cold air, which will have an adverse effect on the storage and preservation of food and reduce the energy efficiency of the refrigeration equipment. In addition, the results of human eye observation are often subjective and inaccurate, making it difficult to accurately evaluate the actual available space of the refrigeration equipment. Summary of the invention

[0004] The present application provides a method, system, device and storage medium for measuring the available volume of a refrigeration device, which are used to accurately evaluate the available volume of the refrigeration device.

[0005] In a first aspect, the present application provides a method for measuring the available volume of a refrigeration device, which is applied to a refrigeration device provided with an ultrasonic measuring device and a visual measuring device, and the method comprises: Acquiring ultrasonic measurement data measured by an ultrasonic measuring device and a visual image measured by a visual measuring device in the refrigeration equipment; Perform image segmentation on the visual image to obtain a number of segmented images, and perform feature extraction on the segmented images to obtain image feature data; Construct a preliminary ultrasonic 3D point cloud model of the interior of the refrigeration equipment based on ultrasonic measurement data; Extract features from the preliminary ultrasonic three-dimensional point cloud model to obtain three-dimensional point cloud graphic feature data, and integrate the three-dimensional point cloud graphic feature data with the three-dimensional point cloud graphic feature data to obtain target image feature data; A target visual three-dimensional model is constructed based on the target image feature data, and a preliminary ultrasonic three-dimensional point cloud model is corrected based on the image feature data to obtain a target ultrasonic three-dimensional point cloud model; The target visual 3D model and the target ultrasonic 3D point cloud model are fused and reconstructed to obtain the target 3D model; The available volume of the refrigeration equipment is determined based on the target three-dimensional model.

[0006] By adopting the above technical scheme, ultrasonic measurement data and visual images inside the refrigeration equipment are obtained by using ultrasonic measurement devices and visual measurement devices, which overcomes the shortcomings of a single measurement method and can obtain comprehensive and accurate internal information of the refrigeration equipment. The visual image is processed by image segmentation and feature extraction technology to obtain image feature data, which fully mines the effective information contained in the visual image. At the same time, a preliminary ultrasonic three-dimensional point cloud model is constructed based on the ultrasonic measurement data, and then feature extraction is performed to obtain three-dimensional point cloud graphic feature data, which is then integrated with the image feature data to obtain target image feature data, realizing the fusion of multi-source data and greatly improving the accuracy and reliability of the three-dimensional model. On this basis, the target visual three-dimensional model and the target ultrasonic three-dimensional point cloud model are constructed respectively, and the ultrasonic model is corrected using the image feature data to further optimize the accuracy of the model. Finally, by fusing and reconstructing the target visual three-dimensional model and the target ultrasonic three-dimensional point cloud model, a high-precision and high-fidelity target three-dimensional model is obtained, which provides reliable data support for the accurate calculation of the available volume of the refrigeration equipment.

[0007] In a second aspect of the present application, a system for measuring the available volume of a refrigeration device is provided, the system comprising: A data acquisition module, used to acquire ultrasonic measurement data measured by an ultrasonic measurement device in the refrigeration equipment and visual images measured by a visual measurement device; An image feature extraction module is used to perform image segmentation on the visual image to obtain a number of segmented images, and perform feature extraction on the segmented images to obtain image feature data; An ultrasonic three-dimensional point cloud model building module is used to build a preliminary ultrasonic three-dimensional point cloud model inside the refrigeration equipment based on ultrasonic measurement data; A visual model building module is used to extract features from the preliminary ultrasonic three-dimensional point cloud model to obtain three-dimensional point cloud graphic feature data, and integrate the three-dimensional point cloud graphic feature data with the three-dimensional point cloud graphic feature data to obtain target image feature data; A correction module is used to construct a target visual three-dimensional model based on the target image feature data, and to correct the preliminary ultrasonic three-dimensional point cloud model based on the image feature data to obtain a target ultrasonic three-dimensional point cloud model; A fusion reconstruction module is used to fuse and reconstruct the target visual 3D model and the target ultrasonic 3D point cloud model to obtain a target 3D model; The available volume calculation module of the refrigeration equipment is used to determine the available volume of the refrigeration equipment based on the target three-dimensional model.

[0008] In a third aspect of the present application, a computer storage medium is provided, wherein the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the above method steps.

[0009] In the fourth aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the above method.

[0010] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The final target 3D model of the internal space of the refrigeration equipment is obtained by fusing and reconstructing the target visual 3D model and the target ultrasonic 3D point cloud model. This fusion reconstruction process combines the advantages of both visual and ultrasonic measurement data to obtain more complete and accurate 3D space information inside the refrigeration equipment. Based on the target 3D model, the available volume of the refrigeration equipment can be accurately calculated, providing users with real and reliable volume information feedback.

[0011] 2. By acquiring ultrasonic measurement data and visual images inside the refrigeration equipment, the complementarity of the two measurement methods is fully utilized. Ultrasonic measurement can obtain distance information inside the refrigeration equipment, while visual images can capture the shape and texture characteristics of objects. By performing image segmentation and feature extraction on the visual image, rich image feature data is obtained, providing an important information basis for subsequent 3D reconstruction.

[0012] 3. This application constructs a preliminary ultrasonic 3D point cloud model of the interior of the refrigeration equipment based on ultrasonic measurement data, and obtains the preliminary geometric structure of the refrigeration equipment space. On this basis, by supplementing and correcting the image feature data with the preliminary ultrasonic 3D point cloud model, more complete and accurate target image feature data and target ultrasonic 3D point cloud model are obtained. This process makes full use of the complementarity of the two measurement data and effectively improves the accuracy and reliability of 3D reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of a flow chart of a method for measuring the available volume of a refrigeration device provided in an embodiment of the present application; Figure 2An architectural diagram of a refrigeration equipment available volume measurement system provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0014] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0015] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0016] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple devices refer to two or more devices, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0017] In order to facilitate understanding of the method and system provided by the embodiments of the present application, before introducing the embodiments of the present application, the background of the embodiments of the present application is first introduced.

[0018] With the rapid development of technology and the continuous improvement of people's quality of life, refrigeration equipment has become an indispensable appliance for modern families. People's expectations for refrigeration equipment are no longer limited to simple food storage and preservation functions. Intelligent control experience and convenience are also becoming an important trend in the development of refrigeration equipment. In order to meet the growing needs of users, many refrigeration equipment manufacturers have taken intelligence as the core direction of research and development, and have been tirelessly committed to providing more humane and intelligent refrigeration equipment products, in order to significantly improve the user experience and quality of life.

[0019] In the daily use of refrigeration equipment, users often need to understand the utilization status of the internal space of the refrigeration equipment in order to reasonably plan the storage and purchase of food. The traditional way is to roughly estimate the available volume of the refrigeration equipment by opening the door of the refrigeration equipment and visually observing it with the naked eye. However, this method has many inconveniences and limitations. First, the user must be near the refrigeration equipment to operate it, and it is impossible to obtain the convenience of remotely obtaining the volume information of the refrigeration equipment. Secondly, frequent opening and closing of the refrigeration equipment door will cause a large amount of cold air to be lost, which will have an adverse effect on the storage and preservation of food, and also reduce the energy efficiency of the refrigeration equipment. In addition, the results obtained by human eye observation alone are often difficult to avoid subjectivity and inaccuracy, and it is difficult to accurately evaluate the actual available space of the refrigeration equipment.

[0020] After the background introduction of the above content, those skilled in the art can understand the problems existing in the prior art. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0021] On this basis, the present application embodiment provides a method for measuring the available volume of a refrigeration device, see Figure 1 The method can be implemented by a computer program or run as an independent tool application. Specifically, the method is applied to a refrigeration device provided with an ultrasonic measuring device and a visual measuring device. The method includes: S101, acquiring ultrasonic measurement data measured by an ultrasonic measurement device and a visual image measured by a visual measurement device in a refrigeration device; Specifically, the ultrasonic measuring device is arranged at a specific position inside the refrigeration equipment, such as the top and side wall of the refrigeration equipment. The ultrasonic measuring device transmits ultrasonic signals and receives reflected echoes to measure the time it takes for ultrasonic waves to propagate inside the refrigeration equipment, and then calculates the distance the ultrasonic signal reaches the surface of the object. By performing multiple ultrasonic measurements at different positions inside the refrigeration equipment, a series of distance data can be obtained to form an ultrasonic measurement data set. These distance data contain the geometric structure information of the internal space of the refrigeration equipment, providing an important data basis for the subsequent construction of a three-dimensional point cloud model.

[0022] At the same time, visual measurement devices, usually high-resolution cameras, are also installed at specific locations inside the refrigeration equipment, such as the top and side walls of the refrigeration equipment. The visual measurement device captures the appearance information of objects inside the refrigeration equipment, such as the shape and texture of the objects, by taking images inside the refrigeration equipment. By taking multiple images at different angles and positions, a comprehensive visual image data set can be obtained. These visual images contain rich information about the appearance features of objects, providing key data support for subsequent image segmentation, feature extraction, and 3D reconstruction.

[0023] S102, performing image segmentation on the visual image to obtain a plurality of segmented images, and performing feature extraction on the segmented images to obtain image feature data; Specifically, the acquired visual image is first segmented. Image segmentation is a commonly used image processing technology, which aims to divide an image into a number of non-overlapping regions so that the pixels within each region have similar characteristics, while the pixels between different regions have obvious differences. In the application scenario of the present invention, image segmentation is mainly used to separate objects inside the refrigeration equipment from the background, thereby facilitating subsequent feature extraction and three-dimensional reconstruction.

[0024] The specific implementation of image segmentation can adopt a variety of algorithms, such as threshold-based segmentation, region-based segmentation, edge-based segmentation, etc. For example, the visual image can be preprocessed first, such as denoising, smoothing, etc., to improve the quality of the image. Then, an appropriate segmentation algorithm is selected according to the characteristics of the image, such as by setting a suitable threshold to divide the image into two regions: foreground (object) and background. Then, through morphological processing, connected domain analysis and other operations, the segmentation result is further optimized to obtain a number of segmented images, each of which corresponds to an object inside the refrigeration equipment. After obtaining the segmented image, the method of the present invention performs feature extraction on each segmented image. Feature extraction refers to extracting key information that can characterize the characteristics of an object from an image, such as the shape, texture, color, etc. of the object. In the application scenario of the present invention, feature extraction is mainly used to obtain the appearance characteristics of objects inside the refrigeration equipment, providing important reference information for subsequent three-dimensional reconstruction and volume measurement.

[0025] Based on the above embodiment, as an optional embodiment, the visual image is segmented to obtain a plurality of segmented images, and features are extracted from the segmented images to obtain image feature data, including: S201, performing image segmentation on the visual image based on an image segmentation algorithm to obtain a plurality of segmented images; Specifically, there are many specific implementation methods for image segmentation, such as threshold-based segmentation, region-based segmentation, edge-based segmentation, etc. In the present invention, it is preferred to use a semantic segmentation algorithm based on deep learning, such as Fully Convolutional Networks (FCN), U-Net, etc. These algorithms perform end-to-end pixel-level classification of images through convolutional neural networks, and can automatically learn high-level semantic information in the image to achieve accurate target segmentation. Specifically, the visual image is input into a pre-trained semantic segmentation network, and the network will classify each pixel of the image and predict which type of object or region it belongs to. The segmentation result is usually output in the form of a mask map, and different objects or regions are represented by different colors or grayscale values.

[0026] After image segmentation, the original visual image is divided into several segmented images, each of which corresponds to an independent object or region. In these segmented images, the interference of background and irrelevant objects is greatly reduced, while the target object of interest is highlighted, which is convenient for subsequent analysis and processing. At the same time, due to the use of advanced semantic segmentation algorithms, segmentation accuracy and semantic consistency are also guaranteed, providing a high-quality data foundation for subsequent feature extraction.

[0027] S202, preprocessing the segmented image region to obtain a preprocessed image region; Specifically, the segmented image area is preprocessed to obtain the preprocessed image area. The reason for preprocessing is that the segmented image area may still have some noise, distortion, uneven illumination and other problems, and direct feature extraction may affect the quality and accuracy of the features. By preprocessing the segmented image, these interference factors can be eliminated, the quality and availability of the image can be improved, and a more stable and reliable data basis can be provided for subsequent feature extraction. The specific steps of preprocessing include image denoising, image enhancement, image correction and image normalization. First, algorithms such as median filtering and bilateral filtering are used to remove noise points and outliers in the segmented image to make the image smoother and cleaner. Then, the contrast and clarity of the image are enhanced by methods such as histogram equalization and contrast adjustment, so that the texture and edge of the object are more obvious. Then, geometric transformation algorithms such as perspective transformation and affine transformation are used to perform geometric correction on the segmented image to eliminate distortion caused by shooting angle, lens distortion, etc., so that the image is more regular and easy to analyze. Finally, the segmented images are scaled and normalized to make the images in different regions have similar scale and brightness distribution, which is convenient for unified feature extraction and comparison. By processing the segmented images one by one, a set of pre-processed image regions with higher quality and more stable features can be obtained.

[0028] Compared with the original segmented image, the preprocessed image area has the following advantages: higher image quality, effective suppression of noise and distortion, clearer and more regular textures and edges of objects; more stable image features, more consistent scale, brightness and other aspects of images in different areas, which facilitates feature extraction and comparison; more concentrated image information, removal of irrelevant background and interference information, highlighting the target objects of interest, and improving the pertinence and efficiency of subsequent analysis.

[0029] S203, extracting texture features from the preprocessed image area to obtain texture image feature data; Specifically, the specific steps of texture feature extraction are as follows: First, one or more texture description algorithms, such as gray level co-occurrence matrix (GLCM), local binary pattern (LBP), Gabor filter, etc., are used to measure the texture of the preprocessed image area. These algorithms extract a set of numerical indicators that can characterize texture features, such as contrast, entropy, uniformity, directionality, etc., by analyzing the spatial relationship and grayscale distribution of pixels in the image area. Then, the extracted texture indicators are combined into a high-dimensional feature vector as the texture feature representation of the image area. In order to further improve the robustness and discriminability of the features, some feature selection and dimensionality reduction methods, such as principal component analysis (PCA) and linear discriminant analysis (LDA), can also be used to optimize and compress the texture feature vector to obtain a more compact and effective feature representation. Finally, the optimized texture feature vector is stored to form texture image feature data for subsequent analysis and application.

[0030] By extracting texture features, we can obtain a set of numerical features that can fully characterize the texture properties of the image area. These texture features are closely related to the material, shape, lighting and other properties of the object, and can effectively distinguish different categories of objects, such as metal, plastic, paper, etc. Compared with the original image data, texture features have the following advantages: First, texture features are a highly abstract and generalized representation, which is not affected by factors such as the scale, angle, and lighting of the object, and has strong robustness and adaptability. Secondly, texture features are a compact representation, and the amount of data is much smaller than the original image data, which is convenient for storage, transmission and processing. Finally, texture features are a semantically rich representation that can reflect the properties and characteristics of objects from multiple perspectives, and provide important discriminant information for tasks such as object recognition, classification, and retrieval.

[0031] S204, performing edge recognition on the processed image region to obtain an edge image of the image region, and calculating a geometric moment of the image region based on the edge image; Specifically, the specific steps of edge recognition and geometric moment calculation are as follows: First, one or more edge detection algorithms, such as the Canny algorithm, the Sobel algorithm, the Laplacian algorithm, etc., are used to extract the edge of the preprocessed image area. These algorithms identify the boundaries and contours of the object by analyzing the gradient changes and local structures of the pixels in the image area, and generate a binary edge image. In the edge image, the value of the edge pixel is 1, and the value of the non-edge pixel is 0, which clearly outlines the shape and position of the object. Then, based on the extracted edge image, the geometric moment of the image area is calculated. The geometric moment is a set of numerical indicators that can characterize the shape attributes of the image area. Commonly used geometric moments include area, perimeter, circularity, rectangularity, direction, etc. These geometric moments can be obtained by performing operations such as integration, distance transformation, and fitting on the edge image. For example, the area can be obtained by summing the edge image, the perimeter can be obtained by counting the 8-connectivity of the edge pixels, the circularity can be obtained by the ratio of the area to the perimeter, the rectangularity can be obtained by fitting the minimum circumscribed rectangle, and the direction can be obtained by calculating the second-order moment of the image. Finally, the calculated geometric moments are stored as shape feature data of the image area for subsequent analysis and application.

[0032] Through edge recognition and geometric moment calculation, a set of numerical features that can fully characterize the shape attributes of the image area can be obtained. These shape features are closely related to the attributes of the object such as the outline, size, and direction, and can effectively distinguish objects of different categories and states, such as beverage bottles, fruits and vegetables, and boxed foods. Compared with the original image data, shape features have the following advantages: First, shape features are a highly abstract and generalized representation that is not affected by factors such as the texture, color, and lighting of the object, and has strong robustness and adaptability. Secondly, shape features are a compact representation that is much smaller than the original image data, making it easier to store, transmit, and process. Finally, shape features are a semantically rich representation that can reflect the geometric attributes and spatial relationships of objects from multiple perspectives, providing important discriminant information for tasks such as object recognition, positioning, and three-dimensional reconstruction.

[0033] S205, calculating shape features based on geometric moments to obtain shape feature data, and using the texture image feature data and the shape feature data as image feature data.

[0034] Specifically, the specific steps of shape feature calculation and feature combination are as follows: First, based on the geometric moment obtained in step S204, a set of shape features are further extracted. Common shape features include: area ratio, aspect ratio, circularity, rectangularity, convexity, direction, etc. These shape features can be obtained by normalizing the geometric moment, calculating the ratio, statistical analysis and other mathematical operations. For example, the area ratio can be obtained by dividing the area of ​​the image area by the area of ​​the minimum circumscribed rectangle, the aspect ratio can be obtained by dividing the length of the minimum circumscribed rectangle by the width, the circularity can be obtained by dividing the area by the square of the perimeter and then multiplying it by 4π, the rectangularity can be obtained by dividing the area of ​​the image area by the area of ​​the minimum circumscribed rectangle, the convexity can be obtained by dividing the area of ​​the image area by the area of ​​its convex hull, and the direction can be obtained by the direction of the minimum inertia principal axis of the image area. These shape features are all scalar values, usually ranging from 0 to 1, and can quantitatively describe various geometric properties of the image area. The extracted shape features are combined into a vector as the shape feature data of the image area. Then, the texture image feature data obtained in step S203 and the shape feature data calculated above are combined into a complete image feature data. Specifically, the texture feature vector and the shape feature vector are concatenated in a certain order to form a higher-dimensional feature vector. This feature vector is the final feature representation of the image area, which combines the visual attributes of both texture and shape, and can fully and accurately characterize the characteristics and content of the image area. In order to balance the contribution of texture features and shape features, the feature vectors can also be normalized so that the feature values ​​of different dimensions are at similar scales and distributions. Finally, the combined image feature data is stored for subsequent tasks such as object recognition and attribute analysis.

[0035] Based on the above embodiment, as an optional embodiment, the geometric moment includes area moment, center distance moment, radial distance moment, composite moment and invariant moment; shape feature calculation is performed based on the geometric moment to obtain shape feature data, including: Calculate shape characteristics based on area moment and determine area characteristic data; Calculate shape features based on the central moment to determine feature data from the center to the boundary point; Calculate shape features based on radial distance moments to determine feature data from the shape center to the boundary points; Calculate the shape characteristics according to the composite moment to determine the overall morphological characteristic data of the shape; Calculate shape features based on invariant moments to determine feature data of the shape under geometric transformation; The area feature data, the feature data from the shape center to the boundary point, the overall morphological feature data of the shape and the feature data under geometric transformation are taken as the shape feature data.

[0036] Specifically, the steps of shape feature calculation are as follows: First, shape feature calculation is performed based on area moment to determine area feature data. Area moment describes the size and scale information of the image area and can be used to distinguish objects of different sizes. By normalizing the area moment, a scale-invariant area feature can be obtained, which reflects the size ratio of the image area relative to the entire image. Secondly, shape feature calculation is performed based on the center moment to determine the feature data from the center to the boundary point. The center moment describes the center of mass position and symmetry information of the image area and can be used to determine the position and direction of the object. By calculating the distance from the center of mass to the boundary point, a set of feature values ​​reflecting the symmetry and direction of the object shape can be obtained. Thirdly, shape feature calculation is performed based on radial distance moment to determine the feature data from the center of the shape to the boundary point. The radial distance moment describes the radial scale and compactness information of the image area and can be used to distinguish objects of different shapes. By calculating the distance from the center of the shape to the boundary point, a set of feature values ​​reflecting the compactness and regularity of the object shape can be obtained. Then, shape feature calculation is performed based on the composite moment to determine the overall morphological feature data of the shape. The composite moment is a high-order geometric moment that comprehensively considers multiple geometric properties such as the area, center, and direction of the image area and can characterize the overall shape characteristics of the object. By calculating various invariants of composite moments, such as Hu moments, Zernike moments, etc., a set of eigenvalues ​​reflecting the overall morphological characteristics of the object can be obtained.

[0037] Finally, the shape features are calculated based on the invariant moments to determine the feature data of the shape under geometric transformation. An invariant moment is a geometric moment that remains unchanged under geometric transformations such as translation, rotation, and scaling, and can characterize the transformation properties of the image area. By constructing various combinations of invariant moments, such as rotational invariant moments and affine invariant moments, a set of eigenvalues ​​that reflect the shape of the object remaining unchanged under geometric transformation can be obtained.

[0038] The shape feature data of the image region are obtained by combining the area feature data, feature data from the shape center to the boundary point, overall morphological feature data of the shape and feature data under geometric transformation. These shape feature data together with the texture image feature data constitute a complete description of the visual attributes of the image region, reflecting the geometric characteristics, structural characteristics and transformation properties of the image region from different angles, laying a solid feature foundation for subsequent tasks such as object recognition, shape matching, and 3D reconstruction.

[0039] Through the above shape feature extraction method, a comprehensive, detailed and robust shape feature representation can be obtained. This shape feature that integrates multiple geometric moments has the following advantages: First, the shape feature integrates geometric information at multiple levels, describes the attributes of the shape from multiple angles such as area, symmetry, compactness, and topological structure, and can accurately characterize the geometric characteristics of the object. Secondly, the shape feature is invariant to common geometric transformations and can adapt to changes such as translation, rotation, and scaling of the object, enhancing the robustness and adaptability of the system. Thirdly, the shape feature is efficient in calculation and compact in storage, which can realize real-time processing and massive data retrieval, meeting the performance requirements of practical applications.

[0040] S103, constructing a preliminary ultrasonic three-dimensional point cloud model inside the refrigeration equipment based on the ultrasonic measurement data; Specifically, the ultrasonic measurement data contains a series of distance information, each distance value corresponds to the distance from the ultrasonic sensor to the surface of the object inside the refrigeration equipment. By combining these distance values ​​with the position and direction of the ultrasonic sensor, the coordinates of each measurement point in three-dimensional space can be calculated. By summarizing and organizing the coordinates of all measurement points, a preliminary three-dimensional point cloud model is obtained, which describes the approximate geometric structure of the internal space of the refrigeration equipment.

[0041] The process of constructing a preliminary ultrasonic three-dimensional point cloud model can be divided into the following steps: First, the ultrasonic measurement data is preprocessed, such as removing noise and outliers, to improve the data quality. Specifically, algorithms such as median filtering and Kalman filtering are used to remove random noise and pulse interference in the ultrasonic measurement data to ensure the accuracy and stability of the data. Then, according to the position and direction of the ultrasonic sensor and the distance value of each measurement point, the coordinates of each measurement point in three-dimensional space are calculated. This step requires the use of the principle of triangulation and the spatial position information of the ultrasonic sensor to convert the distance value into three-dimensional coordinates. Next, the coordinates of all measurement points are organized into a point cloud data structure, that is, a preliminary three-dimensional point cloud model is formed. The point cloud data structure is usually stored and managed using the PCL library (Point Cloud Library) to facilitate subsequent processing and visualization. Finally, the point cloud model is simply post-processed, such as downsampling and smoothing, to optimize the quality and efficiency of the model. The downsampling operation improves the processing speed of the model by reducing the number of point clouds; while the smoothing operation reduces the burrs and noise on the model surface by locally fitting the point cloud.

[0042] S104, extracting features from the preliminary ultrasonic three-dimensional point cloud model to obtain three-dimensional point cloud graphic feature data, and integrating the three-dimensional point cloud graphic feature data with the three-dimensional point cloud graphic feature data to obtain target image feature data; Specifically, firstly, the feature extraction of the preliminary ultrasonic three-dimensional point cloud model is performed to obtain the three-dimensional point cloud graphic feature data. The purpose of this step is to extract key geometric feature information from the point cloud model to provide a basis for subsequent data fusion and model optimization. Specifically, the point cloud model can be segmented into multiple sub-regions by using a point cloud segmentation algorithm, and then each sub-region is characterized to extract features such as curvature, normal vector, shape, etc. These feature data can effectively characterize the geometric properties of the point cloud model and facilitate subsequent processing. Next, the extracted three-dimensional point cloud graphic feature data and the previously obtained image feature data are integrated to obtain the target image feature data. The reason for data integration is that ultrasonic measurement and visual measurement perceive the inside of the refrigeration equipment from different angles, and the feature data they obtain are complementary. By fusing the two feature data, a more comprehensive and accurate representation of the internal information of the refrigeration equipment can be obtained.

[0043] When implementing data integration, you first need to spatially align the 3D point cloud graphic feature data and the image feature data so that they are in the same coordinate system. This can be achieved through a feature matching algorithm, such as identifying common feature points in the two data, calculating the transformation matrix, and transforming the coordinates of one data to the coordinate system of the other data. After the alignment is completed, the two feature data can be fused according to the corresponding spatial positions. There are many fusion methods, such as weighted average, maximum method, minimum method, etc., and the appropriate fusion strategy needs to be selected according to the specific application scenario.

[0044] Through the above steps, the fused target image feature data is obtained. This fused feature data contains dual information of ultrasonic measurement and visual measurement, which can more accurately and meticulously reflect the spatial structure and item distribution inside the refrigeration equipment. Based on this high-quality feature data, the subsequent three-dimensional model will also be more accurate and realistic, providing a solid data foundation for the calculation of available volume.

[0045] Based on the above embodiment, as an optional embodiment, feature extraction is performed on the preliminary ultrasonic three-dimensional point cloud model to obtain three-dimensional point cloud graphic feature data, and the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data are integrated to obtain target image feature data, including: S301, performing point cloud segmentation on the preliminary ultrasonic three-dimensional point cloud model to obtain multiple object point clouds, and performing point cloud data extraction on the object point cloud to obtain point cloud data; Specifically, in step S301, the preliminary ultrasonic three-dimensional point cloud model is segmented to obtain multiple object point clouds, and the object point cloud is extracted to obtain point cloud data. The reason for performing point cloud segmentation and point cloud data extraction is that the preliminary ultrasonic three-dimensional point cloud model usually contains point cloud data of multiple objects in the scene. There may be occlusion, adhesion, etc. between these object point clouds. It is difficult to obtain accurate and effective feature representation by directly extracting features from the entire point cloud model. Through point cloud segmentation, the point cloud data of different objects can be separated to eliminate the interference and influence between objects. On this basis, point cloud data extraction is performed separately for each object point cloud, so that more accurate and detailed object point cloud features can be obtained, providing reliable data support for subsequent three-dimensional reconstruction, recognition and other tasks. Specifically, the preliminary ultrasonic three-dimensional point cloud model is first preprocessed, including denoising, downsampling, smoothing and other operations to improve the quality and regularity of the point cloud data. Then, one or more point cloud segmentation algorithms, such as region growing method, clustering method, graph cut method, etc., are used to segment the point cloud model into multiple independent object point clouds. These algorithms classify point cloud data belonging to the same object into one category by analyzing the geometric features, topological structure, semantic information, etc. of the point cloud, forming separate object point clouds. On the basis of point cloud segmentation, point cloud data extraction is performed on each object point cloud, and the spatial distribution, geometric properties, topological relationships, etc. of each object point cloud are analyzed to extract a set of data that can characterize the point cloud features of the object. Common point cloud data extraction methods include spatial distribution features, geometric features, topological features, statistical features, etc. The point cloud data of each object point cloud is obtained by combining the extracted spatial distribution features, geometric features, topological features, statistical features, etc. These point cloud data characterize the three-dimensional shape, structure, distribution and other features of the object point cloud from different angles, providing a rich and reliable data foundation for subsequent three-dimensional reconstruction, object recognition and other tasks. Through point cloud segmentation and point cloud data extraction, an independent point cloud data representation of each object in the scene can be obtained. This separated object point cloud data has the following advantages: each object point cloud is a complete and independent three-dimensional model, eliminating interference and occlusion from other objects, which helps to improve the accuracy of three-dimensional reconstruction and recognition; the data volume of each object point cloud is relatively small, which is convenient for storage, transmission and processing, and improves the efficiency and real-time performance of the system; each object point cloud extracts rich feature data, describes the three-dimensional properties of the object from multiple angles, and provides more discriminant information for intelligent analysis and interaction.

[0046] S302, aligning the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data according to the spatial position, establishing a corresponding relationship between the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data, and fusing the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data through a feature fusion algorithm according to the corresponding relationship to obtain fused target image feature data.

[0047] Specifically, the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data are registered according to the spatial position, the corresponding relationship between the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data is established, and the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data are fused according to the corresponding relationship through the feature fusion algorithm to obtain the fused target image feature data. The reason for feature registration and fusion is that the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data describe the three-dimensional features of the object from different angles, and it is difficult to fully and accurately represent the three-dimensional properties of the object using any one feature data alone. By spatially registering and fusing the two feature data, their complementarity and correlation can be fully utilized to obtain a more complete, accurate and detailed three-dimensional feature representation, providing more reliable and effective data support for subsequent three-dimensional reconstruction, object recognition, intelligent interaction and other tasks. Specifically, the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data need to be spatially registered first. Since the two feature data may use different coordinate systems and scales, direct fusion will cause feature distortion and dislocation. Therefore, it is necessary to unify the two feature data into the same coordinate system through rigid body transformation, scale transformation and other operations, and perform scale normalization. Commonly used spatial registration algorithms include ICP algorithm, NDT algorithm, FGR algorithm, etc. These algorithms estimate the optimal transformation parameters by minimizing the distance measurement between the two feature data, such as Euclidean distance, Mahalanobis distance, etc., to achieve spatial alignment of feature data. After completing the spatial registration, it is necessary to establish the correspondence between the 3D point cloud graphic feature data and the 3D point cloud graphic feature data. Specifically, it is to find the feature points or feature areas belonging to the same object or part in the two feature data, and establish a mapping relationship between them. This correspondence can be determined based on criteria such as spatial proximity, geometric similarity, and topological consistency of the feature data. Commonly used methods include nearest neighbor search, feature matching, and graph matching. The purpose of establishing the correspondence is to provide a basis for subsequent feature fusion and ensure the correctness and accuracy of the fusion process. After the correspondence is established, the 3D point cloud graphic feature data and the 3D point cloud graphic feature data can be fused through the feature fusion algorithm to obtain the fused target image feature data. Commonly used feature fusion algorithms include weighted average method, principal component analysis method, Kalman filtering method, deep learning method, etc. These algorithms adaptively adjust the fusion weights and strategies according to factors such as the credibility, importance, and complementarity of the feature data to generate a fused feature representation that combines the advantages of the two feature data.For example, the weighted average method can calculate the fusion weight based on the local curvature, density and other indicators of the feature data, highlighting the detailed features of the object; the principal component analysis method can extract the main components of the two feature data through feature dimensionality reduction and reconstruction, and remove redundancy and noise; the Kalman filter method can track the changes of feature data through dynamic updates and predictions, and improve the temporal consistency of fusion; the deep learning method can automatically learn the optimal fusion strategy through end-to-end feature extraction and fusion, and improve the semantic understanding ability of fusion. The fused target image feature data contains both the spatial structure information of the three-dimensional point cloud and the texture appearance information of the two-dimensional image. It can describe the three-dimensional features of the object from multiple dimensions and multiple levels, with higher resolution, stronger expression ability, and better robustness, providing strong data support for tasks such as three-dimensional reconstruction, object recognition, and intelligent interaction.

[0048] S105, constructing a target visual three-dimensional model based on the target image feature data, and correcting the preliminary ultrasonic three-dimensional point cloud model based on the image feature data to obtain a target ultrasonic three-dimensional point cloud model; Specifically, the present application constructs a target visual three-dimensional model based on the target image feature data. The reason for constructing a visual three-dimensional model is that the visual image contains rich appearance information such as texture and color. This information can be used to construct a high-fidelity three-dimensional model to make it closer to the real scene. The specific method of constructing a visual three-dimensional model is to input the target image feature data into a three-dimensional reconstruction algorithm, such as the Structure from Motion (SFM) algorithm or the Multi-ViewStereo (MVS) algorithm. These algorithms estimate the three-dimensional structure and camera pose of the scene by analyzing the correspondence between multi-view images, and then reconstruct the three-dimensional model. The reconstructed target visual three-dimensional model has a high degree of visual realism, but its scale and accuracy may deviate from the actual scene to a certain extent.

[0049] In order to further improve the accuracy of the three-dimensional model, it is also necessary to correct the preliminary ultrasonic three-dimensional point cloud model based on the image feature data. The reason why the image feature data is used to correct the ultrasonic model is that the resolution and texture information of the visual image are usually better than the ultrasonic point cloud, which can provide more geometric details and constraint information for the point cloud model. The specific steps of the correction are to first align the image feature data with the ultrasonic point cloud model to establish a corresponding relationship between them. This can be achieved through methods such as feature matching and mutual information. After the registration is completed, the high-precision feature points in the image feature data are used to geometrically transform and optimize the ultrasonic point cloud model to make it more consistent with the scale and structure of the real scene. For example, the coordinates of each point in the point cloud model can be adjusted through optimization algorithms such as the least squares method to make it coincide with the corresponding image feature points as much as possible. After correction, the target ultrasonic three-dimensional point cloud model has improved its geometric structure and details while maintaining the original ultrasonic ranging accuracy, which is closer to the real situation.

[0050] Based on the above embodiment, as an optional embodiment, the preliminary ultrasonic three-dimensional point cloud model is corrected based on the image feature data to obtain the target ultrasonic three-dimensional point cloud model, including: S401, spatially corresponding the image feature data with the preliminary ultrasonic three-dimensional point cloud model to obtain an association relationship between the image feature data and the preliminary ultrasonic three-dimensional point cloud model, and correspondingly correcting the preliminary ultrasonic three-dimensional point cloud model and the image feature data according to the association relationship to obtain corrected point cloud image data; Specifically, in step S401, the image feature data is spatially corresponded with the preliminary ultrasonic three-dimensional point cloud model to obtain the association relationship between the image feature data and the preliminary ultrasonic three-dimensional point cloud model, and the preliminary ultrasonic three-dimensional point cloud model and the image feature data are correspondingly corrected according to the association relationship to obtain the corrected point cloud image data. The reason for spatial correspondence and corresponding correction is that the preliminary ultrasonic three-dimensional point cloud model is based on the original three-dimensional data obtained by the ultrasonic sensor. Affected by factors such as sensor accuracy, calibration error, motion distortion, etc., its spatial position and shape may have deviations and distortions, and it is difficult to accurately reflect the real three-dimensional structure of the object. The image feature data integrates point cloud features and image features, contains richer, more accurate and detailed three-dimensional information, and can be used as a reference standard to correct errors and deviations in the preliminary ultrasonic three-dimensional point cloud model. By spatially corresponding and correspondingly correcting the two types of data, the advantages of the image feature data can be used to improve the quality and accuracy of the preliminary ultrasonic three-dimensional point cloud model, and a more accurate, realistic and detailed three-dimensional model can be obtained, providing a more reliable data basis for subsequent three-dimensional analysis, display and application. Specifically, it is necessary to spatially correspond the image feature data with the preliminary ultrasonic three-dimensional point cloud model. Since the two data come from different sensors and algorithms, there may be problems such as inconsistent coordinate systems, scale mismatches, and different perspectives between them, and they cannot be directly compared and fused. Therefore, it is necessary to unify the two data into the same coordinate system through spatial transformation and matching algorithms, such as rigid body transformation, similarity transformation, affine transformation, etc., and establish a corresponding relationship between them. Commonly used spatial correspondence algorithms include ICP algorithm, feature point matching algorithm, image registration algorithm, etc. These algorithms estimate the optimal transformation parameters and correspondence relationship by minimizing the distance metric between the two data or maximizing the similarity metric between them, so as to achieve spatial matching and alignment of data. After establishing the spatial correspondence relationship, the preliminary ultrasonic three-dimensional point cloud model and the image feature data can be correspondingly corrected according to this association relationship. Specifically, it is to use the prior knowledge of spatial position, geometric shape, topological structure, etc. provided by the image feature data to adjust and correct the point cloud data in the preliminary ultrasonic three-dimensional point cloud model to make it more consistent with the real three-dimensional structure of the object. This correspondence correction can be based on a variety of criteria and strategies, such as the least squares method, robust estimation method, local weighted average method, etc. According to the spatial deviation and confidence between the image feature data and the point cloud data, the coordinates, normal vector, curvature and other attributes of the point cloud are adaptively adjusted to obtain a corrected point cloud image data. The corrected point cloud image data integrates the spatial information of the ultrasonic point cloud and the semantic information of the image features, has higher accuracy, stronger integrity, better consistency, and can more realistically reflect the three-dimensional structure and appearance details of the object.For example, the high-resolution texture information provided by the image feature data can be used to complete the details and smooth the surface of the point cloud model, improving the visual quality and realism of the model; the depth edge and contour information provided by the image feature data can be used to correct the shape and optimize the topology of the point cloud model, improving the geometric accuracy and topological correctness of the model; the semantic annotation information provided by the image feature data can be used to segment objects and annotate attributes of the point cloud model, improving the intelligent understanding and interaction capabilities of the model. The corrected point cloud image data provides high-quality data support for applications such as 3D reconstruction, 3D display, and virtual reality, helping to improve the user's immersion and interactivity.

[0051] S402, constructing a target ultrasonic three-dimensional point cloud model according to the point cloud image data.

[0052] Specifically, in step S402, a target ultrasonic three-dimensional point cloud model is constructed according to the point cloud image data. The reason for constructing a three-dimensional model is that although the point cloud image data has undergone spatial correspondence and correspondence correction, and has integrated the spatial information of the ultrasonic point cloud and the semantic information of the image features, it is still a discrete, irregular, and discontinuous data representation form, which is difficult to be directly used for three-dimensional visualization, analysis, and interaction. By further constructing the point cloud image data into a structured, regularized, and semantic three-dimensional model, it can be endowed with richer geometric, topological, and semantic attributes, so that it can better express the three-dimensional shape, structure, and content of the object, and provide a more convenient, efficient, and intelligent data carrier for subsequent three-dimensional processing, display, and application. Specifically, the construction of the three-dimensional model mainly includes three steps: point cloud preprocessing, geometric reconstruction, and semantic modeling. In the point cloud preprocessing stage, it is necessary to perform denoising, smoothing, downsampling, and other operations on the point cloud image data to remove outliers, noise points, and redundant points, and improve the quality and regularity of the point cloud. Common point cloud preprocessing algorithms include statistical filtering, bilateral filtering, voxelization, surface fitting, etc. These algorithms analyze the local neighborhood relationship, normal vector consistency, curvature change and other characteristics of the point cloud, and adaptively adjust the coordinates, normal vectors, density and other properties of the point cloud to make it more consistent with the actual shape and detail characteristics of the object surface. In the geometric reconstruction stage, it is necessary to generate a three-dimensional geometric model of the object based on the preprocessed point cloud data, including basic geometric elements such as vertices, edges, and faces. Common geometric reconstruction algorithms include Poisson reconstruction, Malkbus reconstruction, greedy triangulation, spherical harmony, etc. These algorithms extract a realistic, smooth, and compact triangular mesh model from the point cloud data by approximating the implicit surface of the point cloud, minimizing the reconstruction error, and optimizing the mesh quality. The three-dimensional shape and topological structure of the object are truly reproduced. The reconstructed geometric model has a regular topological structure and continuous surface shape, which is convenient for geometric processing such as mesh simplification, subdivision, and deformation, and is also convenient for realistic display such as material texture mapping and physical rendering. In the semantic modeling stage, it is necessary to add semantic attribute information of objects on the basis of geometric models, including category labels, relationship maps, functional descriptions, etc. Common semantic modeling methods include rule-based semantic mapping, learning-based semantic segmentation, graph-based semantic reasoning, etc. These methods automatically associate semantic labels with different parts of geometric models by integrating multivariate information such as geometry, texture, and context, and using prior knowledge, sample data, and reasoning rules to generate a three-dimensional model with rich semantic information. Semantic modeling enables the three-dimensional model to have not only geometric shapes, but also semantic contents such as object categories, component structures, and spatial relationships, which can support intelligent three-dimensional retrieval, recognition, understanding, and interaction. After point cloud preprocessing, geometric reconstruction, and semantic modeling, the point cloud image data is finally constructed into a target ultrasonic three-dimensional point cloud model.The model inherits the spatial structure of the ultrasonic point cloud, the appearance semantics of the image features, and the geometric topology of the three-dimensional shape. It is a high-quality three-dimensional representation that integrates multi-source information and multi-layer semantics, and can fully, stereoscopically, and accurately reflect the three-dimensional properties and content of the object. Based on this semantically enriched three-dimensional model, intelligent applications such as visual three-dimensional display, interactive three-dimensional operation, and semantic three-dimensional analysis can be performed, which greatly expands the application space of ultrasound data and provides more advanced and effective technical means for ultrasound-guided surgical planning, auxiliary diagnosis, training and teaching, etc.

[0053] S106, fusing the target visual 3D model and the target ultrasonic 3D point cloud model to obtain a target 3D model; Specifically, the target visual 3D model and the target ultrasonic 3D point cloud model are registered to the same coordinate system. Since the two models have been preliminarily registered and optimized in the previous steps, the registration process in this step is relatively simple, mainly to unify the scale and coordinate system. After the registration is completed, the two models need to be fused. There are many methods for fusion, the most common of which are weighted average method, graph cut method, Poisson reconstruction method, etc. Taking the weighted average method as an example, the basic idea is to assign a weight to each spatial position, and the values ​​of the visual model and the ultrasonic model at this position are weighted averaged according to the weight to obtain the fused value. The calculation of the weight can be determined based on factors such as the credibility of the model and the point density. For example, in areas with rich edges and textures, the weight of the visual model should be higher, while in areas with flat, single-tone colors, the weight of the ultrasonic model should be higher. By setting the weights reasonably, the advantages of the two models can be fully utilized in the fusion process to obtain a more accurate and realistic 3D representation.

[0054] The target 3D model obtained by fusion reconstruction combines the advantages of visual models and ultrasonic models, and has the characteristics of high precision and high fidelity. It can reflect the real situation inside the refrigeration equipment with higher spatial resolution and measurement accuracy, including the placement, shape and size of items. Compared with a single visual model or ultrasonic model, the fused target 3D model has significantly improved visual quality and geometric accuracy, and can provide more reliable data support for subsequent volume calculations. At the same time, since the fusion process takes into account multiple factors and adopts different fusion strategies for different regions, the generated model is more reasonable in local details and overall structure, and the color and texture are more natural and realistic. This high-quality 3D model can not only be used for volume calculation, but also can be visualized, providing users with an immersive interactive experience and enhancing users' intuitive understanding of the internal space of the refrigeration equipment.

[0055] Based on the above embodiment, as an optional embodiment, the target visual 3D model and the target ultrasonic 3D point cloud model are fused and reconstructed to obtain the target 3D model, including: S501, placing the target visual 3D model and the target ultrasonic 3D point cloud model in the same coordinate system, aligning the coordinate systems of the target visual 3D model and the target ultrasonic 3D point cloud model through coordinate transformation, and obtaining a first target visual 3D model and a first target ultrasonic 3D point cloud model in the same coordinate system; Specifically, in step S501, the target visual three-dimensional model and the target ultrasonic three-dimensional point cloud model are placed in the same coordinate system, and the coordinate systems of the target visual three-dimensional model and the target ultrasonic three-dimensional point cloud model are aligned by coordinate transformation to obtain the first target visual three-dimensional model and the first target ultrasonic three-dimensional point cloud model in the same coordinate system. The reason for the coordinate system alignment is that the target visual three-dimensional model and the target ultrasonic three-dimensional point cloud model are derived from different imaging devices and reconstruction algorithms, respectively. They each have independent coordinate systems and scale units, and there may be large differences in spatial position, direction, size, etc., and they cannot be directly compared and fused. By unifying and aligning the coordinate systems of the two models, the spatial differences and inconsistencies between them can be eliminated, so that they can represent the three-dimensional structure and position of the object in the same reference frame, and provide a unified spatial basis for subsequent model registration, fusion, analysis and other tasks. Specifically, coordinate system alignment is mainly achieved through coordinate transformation. Coordinate transformation is the process of converting the coordinates of a three-dimensional point in one coordinate system to another coordinate system. Common coordinate transformations include rigid body transformation, similarity transformation, affine transformation, etc. Among them, the rigid body transformation keeps the shape and size of the object unchanged, and only changes its translation and rotation; the similarity transformation adds scale scaling on the basis of the rigid body transformation; the affine transformation adds scale inequality and shearing on the basis of the similarity transformation. For the target visual 3D model and the target ultrasonic 3D point cloud model, it is necessary to select the appropriate coordinate transformation method and estimate the optimal transformation parameters according to their imaging principles, scale units, reconstruction algorithms and other factors. Commonly used coordinate transformation estimation methods include absolute orientation method, relative orientation method, bundle adjustment, etc. These methods solve the optimal translation, rotation and scaling parameters by minimizing the spatial errors between the models, such as distance error, angle error, etc., to achieve the best matching and alignment of the two models in space. After the coordinate system is aligned, the target visual 3D model and the target ultrasonic 3D point cloud model are converted to the same coordinate system, and the first target visual 3D model and the first target ultrasonic 3D point cloud model are obtained respectively. In the unified coordinate system, the spatial position, direction, size and other attributes of the two models are consistent and corresponding, eliminating the original coordinate difference and scale difference. This facilitates subsequent model registration, fusion, analysis and other tasks, making it possible to directly compare and fuse the spatial information of the two models, display and operate the two models in the same three-dimensional space, and use the complementary characteristics of the two models to optimize the effect of three-dimensional reconstruction. The first-target visual three-dimensional model integrates the high-resolution texture and fine contour of the visual image, and the first-target ultrasonic three-dimensional point cloud model integrates the internal structure and tissue information of the ultrasonic data. The combination of the two can generate a more realistic, complete and accurate three-dimensional anatomical model, providing a more reliable spatial reference for medical diagnosis, surgical planning, robotic control, etc.

[0056] S502, rigidly registering the first target visual 3D model and the first target ultrasonic 3D point cloud model to obtain a registered second target visual 3D model and a second target ultrasonic 3D point cloud model; Specifically, in step S502, the first target visual three-dimensional model and the first target ultrasonic three-dimensional point cloud model are rigidly registered to obtain the registered second target visual three-dimensional model and the second target ultrasonic three-dimensional point cloud model. The reason for rigid registration is that although the first target visual three-dimensional model and the first target ultrasonic three-dimensional point cloud model are already in the same coordinate system after the coordinate system alignment, due to the differences in imaging mechanism, data accuracy, reconstruction algorithm and other factors, the two models may still have some deviations and misalignments in spatial details, affecting the subsequent fusion effect and analysis accuracy. Through rigid registration, the spatial position and direction of the two models can be further adjusted while keeping the shape and size of the two models unchanged, so that they can achieve more accurate alignment and matching in local details, providing a more refined and reliable spatial reference for subsequent fusion reconstruction. Specifically, rigid registration is a three-dimensional registration method based on rigid transformation, which assumes that there are only differences in translation and rotation between the two models, and there are no complex transformations such as scale scaling and nonlinear deformation. The goal of rigid registration is to find an optimal rigid transformation matrix so that the two registered models can achieve the best overlap and matching in space. Common rigid body registration algorithms include ICP algorithm, feature point matching algorithm, mutual information registration algorithm, etc. These algorithms use iterative optimization to minimize the distance metric between the two models or maximize the similarity metric between them, thereby estimating the optimal rigid body transformation parameters. Taking the ICP algorithm as an example, it first selects some corresponding point pairs on the two models, such as surface feature points, skeleton key points, etc., and then calculates the distance error between these corresponding point pairs, and iteratively solves the optimal translation vector and rotation matrix according to the error minimization criterion, so that the distance error between the two models at the corresponding points is minimized. The feature point matching algorithm uses the significant features on the model, such as curvature extreme points, inflection points, etc., to establish the corresponding relationship of feature points through feature descriptor matching, and estimate the rigid body transformation based on the corresponding points. The mutual information registration algorithm maximizes the mutual information of the two models to find the optimal rigid body transformation, so that the registered model achieves the maximum statistical correlation in grayscale, gradient and other attributes. After rigid body registration, the first target visual 3D model and the first target ultrasonic 3D point cloud model are further aligned and matched in spatial details, and the registered second target visual 3D model and the second target ultrasonic 3D point cloud model are obtained. The two registered models remain unchanged in overall properties such as shape, size, and direction, but achieve higher overlap and consistency in local position and details, eliminating spatial deviations introduced by imaging differences and reconstruction errors. The registered visual model and ultrasonic model can more accurately describe the external morphology and internal structure of the same object. Their combination can provide more comprehensive, three-dimensional, and realistic anatomical information, laying a more solid data foundation for subsequent visualization fusion, three-dimensional analysis, virtual simulation and other tasks.

[0057] S503, reconstructing the target three-dimensional model by weighted fusion of the second target visual three-dimensional model and the second target ultrasonic three-dimensional point cloud model.

[0058] Specifically, the registered visual 3D model of the second target and the registered ultrasonic 3D point cloud model of the second target are weighted fused and reconstructed to obtain the target 3D model. The reason for weighted fusion reconstruction is that the two sensing modes of vision and ultrasound have complementary advantages and characteristics, and a single mode is difficult to fully and accurately reflect the 3D shape and internal structure of the target. Visual 3D reconstruction generates a fine surface mesh model through the texture, illumination, contour and other information of the target surface, but it cannot depict the internal details and material distribution; ultrasonic 3D reconstruction generates a discrete voxel point cloud model through the sound wave reflection and attenuation characteristics inside the target, but the resolution and accuracy are limited. By weighted fusion of visual and ultrasonic models, the surface shape and internal structure can be taken into account, the fineness, completeness and accuracy of the model can be balanced, and a comprehensive and realistic 3D model of the target can be generated.

[0059] S107: Determine the available volume of the refrigeration equipment based on the target three-dimensional model.

[0060] Specifically, the target 3D model is preprocessed, including denoising, smoothing and other operations, to eliminate some noise points and irregular surfaces in the model. Then, the target 3D model is segmented into several independent sub-regions using 3D model segmentation algorithms, such as region growing method, graph cut method, etc., each sub-region corresponds to an object or a spatial region. Next, each sub-region is semantically annotated to identify its corresponding object category or spatial attribute. This can be achieved through 3D model recognition algorithms, such as 3D convolutional neural networks based on deep learning. After semantic annotation, a 3D model with semantic information can be obtained, in which each sub-region has a clear category identification.

[0061] On this basis, it is relatively simple to calculate the available volume. First, calculate the total volume inside the entire refrigeration equipment, which can be obtained through numerical calculation methods such as integration or Monte Carlo method. Then, traverse all the sub-areas of the items and calculate the sum of their volumes to get the total volume occupied by the current items. Finally, subtract the volume occupied by the items from the total volume to get the real-time available volume of the refrigeration equipment. For the convenience of users, you can also calculate the ratio of the available volume to the total volume and present it as a percentage.

[0062] This method of calculating available volume based on a 3D model overcomes the subjectivity and inaccuracy of traditional manual estimation and can provide objective and quantitative evaluation results. Due to the use of advanced 3D reconstruction and semantic understanding technology, this method can depict the space occupancy inside the refrigeration equipment with centimeter-level accuracy, taking into account factors such as the shape and placement of items, and the calculation results are more accurate and reliable. Moreover, since the calculation process of available volume is fully automatic, users do not need to perform any manual measurements or inputs, which greatly improves the convenience of use.

[0063] Based on the above embodiment, as an optional embodiment, determining the available volume of the refrigeration equipment based on the target three-dimensional model includes: S601, determining the total volume of the refrigeration equipment based on the target three-dimensional model; Specifically, the determination of the total volume mainly utilizes the geometric properties and topological relationships of the target three-dimensional model. The target three-dimensional model represents the three-dimensional shape and structure of the refrigeration equipment through triangular meshes, voxels, parametric surfaces, etc., contains rich spatial information such as scale, orientation, and topology, and can be directly used for volume calculation and analysis. Common three-dimensional volume calculation methods include triangular mesh volume method, section integration method, voxel counting method, etc. These methods use the principles of geometry and calculus to calculate the spatial volume occupied by the model according to different representation forms of the model. Taking the triangular mesh volume method as an example, it first decomposes the triangular mesh of the model into a series of non-intersecting tetrahedrons, each of which consists of a triangular face on the surface of the model and a vertex inside the model. Then, the volume of each tetrahedron is calculated using the tetrahedron volume formula, and finally the volumes of all tetrahedrons are summed to obtain the total volume of the model. The triangular mesh volume method is suitable for closed two-dimensional manifold surface models. The calculation is simple and efficient, and real-time performance can be achieved through graphics hardware acceleration. The section integration rule cuts the model into a series of parallel sections at equal intervals along one direction, calculates the area of ​​each section, and then uses the numerical integration method to accumulate the cross-sectional areas to obtain the total volume of the model. The section integration method is applicable to three-dimensional models of arbitrary shapes, including non-closed surfaces and non-manifold topologies, but the number of slices and the precision of the integration will affect the efficiency and accuracy of the calculation. The voxel counting method embeds the model into a regular three-dimensional grid, calculates the number of voxels occupied inside the model, and then multiplies the volume of each voxel to obtain the total volume of the model. The voxel counting method is applicable to models represented by voxelization, such as medical imaging data such as CT and MRI. The calculation is simple and intuitive, but the voxel resolution will affect the precision of the volume estimation. In addition to geometric methods, methods based on topological analysis can also be used for volume calculation, such as using topological invariants such as Euler characteristic and Betti number to characterize the connectivity, pores, cavities and other characteristics of the model, and then infer the filling volume inside. Topological methods are applicable to models with complex internal structures and atypical shapes, such as porous materials and spongy bones, but require a deeper understanding and mastery of topological concepts and algorithms. After the total volume is determined, the overall space occupancy and capacity of the refrigeration equipment can be quantified and analyzed, which provides important basic data for subsequent tasks such as available volume estimation, energy efficiency evaluation, and optimal design.

[0064] S602, determining the occupied volume of the space occupied by the items in the refrigeration equipment based on the target three-dimensional model; Specifically, in step S602, the occupied volume of the space occupied by the items in the refrigeration equipment is determined based on the target three-dimensional model. The reason for determining the occupied volume is that the occupied volume reflects the actual loading situation and space utilization efficiency inside the refrigeration equipment, and is a key factor in evaluating its available volume and storage capacity. By identifying and extracting the geometric information of the area occupied by the items from the target three-dimensional model, the degree of occupation of the refrigeration space by the items can be quantitatively analyzed, the bottleneck of space utilization and the potential for optimization can be found, and a basis for the design improvement and inventory management of the refrigeration equipment can be provided. Specifically, the determination of the occupied volume requires the comprehensive use of the semantic information and geometric information of the target three-dimensional model. First, the area occupied by the items should be identified from the target three-dimensional model, which requires the use of semantic segmentation technology to associate and match the geometric elements of the model with the item category labels. Commonly used three-dimensional semantic segmentation methods include rule-based methods, learning-based methods, and graph-based methods. The rule-based method uses the prior knowledge of the items, such as size, shape, topology, etc., to design a series of segmentation rules and decision trees, and recursively segment the model from top to bottom until the item area is completely separated from the background area. The learning-based method manually annotates or automatically generates a large number of training samples of the occupied areas of items, and uses machine learning models such as support vector machines, random forests, and convolutional neural networks to learn the discriminant patterns of items from the geometry, texture, context and other features of the model, and predict and classify the item areas of unknown models. The graph-based method represents the model as an attribute relationship graph. The nodes in the graph correspond to the basic geometric elements of the model, such as points, faces, and bodies. The edges in the graph represent the adjacency, subordination, and dependency relationships between elements. Through operations such as cutting, clustering, and simplification of the graph, the item nodes are separated from the background nodes to obtain a subgraph representation of the occupied area of ​​the item. After obtaining the occupied area of ​​the item, the aforementioned geometric methods such as the triangular mesh volume method, the cross-sectional integral method, and the voxel counting method can be used to directly calculate its volume and obtain the quantitative result of the occupied volume. It is worth noting that when calculating the occupied volume, factors such as the compactness of the items and the void ratio should be considered. This requires further extraction of geometric constraints such as the relative position, direction, and contact relationship of the items in the model, and the use of spatial arrangement optimization, collision detection and other technologies to correct and compensate the occupied volume to reflect the actual filling efficiency and space utilization level. For example, for regularly stacked box items, the gap volume between the boxes can be estimated based on the size and placement of the boxes, and deducted from the occupied volume; for irregularly stacked bulk items, approximate methods such as convex hull volume and alpha shape volume can be used to estimate the looseness of the item pile, and then infer its occupied volume after compaction.

[0065] S603: Determine the available volume of the refrigeration equipment based on the total volume and the occupied volume.

[0066] Specifically, the available volume of the refrigeration equipment is determined based on the total volume and occupied volume. The reason for determining the available volume is that the available volume directly reflects the actual storage capacity and remaining space of the refrigeration equipment, and is a key indicator for evaluating its storage potential and optimizing space utilization. Through comparative analysis of the total volume and occupied volume, the space utilization efficiency and storage margin of the refrigeration equipment can be quantitatively characterized, the main factors affecting the available volume can be found, and a quantitative basis can be provided for equipment improvement and cargo management. Specifically, the determination of the available volume requires comprehensive consideration of the structural characteristics and usage requirements of the equipment itself on the basis of the total volume and occupied volume. Generally speaking, the available volume can be simply calculated by subtracting the occupied volume from the total volume, that is, the available volume is equal to the difference between the total volume and the occupied volume. This calculation method is intuitive and simple, and is suitable for situations where the occupied volume is much smaller than the total volume and the items are placed more neatly. However, in practical applications, the following factors need to be considered in their impact on the available volume: First, the internal structure and necessary components of the equipment, such as partitions, pipes, fans, lighting, etc., will occupy a certain amount of internal space. Although this part of the space is included in the total volume, it cannot be used for item storage and needs to be deducted from the available volume; second, the heat dissipation and ventilation requirements of the equipment, such as cold air convection in the refrigerator and defrosting drainage in the freezer, require a certain amount of space to ensure the normal operation of the equipment, and this part of the space cannot be fully used for item storage; third, the compactness and rationality of the placement of items. Too crowded or loose placement will reduce the efficiency of space utilization, and the limitation of ergonomics also makes it difficult for some space to be fully utilized, resulting in the available volume being lower than the theoretical value. Therefore, when calculating the available volume, it is necessary to introduce correction parameters such as structural coefficient, ventilation coefficient, and compact coefficient to correct the difference between the total volume and the occupied volume. These correction coefficients can be obtained through empirical estimation or statistical analysis, or they can be customized according to the design parameters and operating conditions of the specific equipment. For example, for a household refrigerator with a total volume of 500L and an occupied volume of 400L, assuming that its structural coefficient is 0.05, ventilation coefficient is 0.1, and compact coefficient is 0.85, then its available volume = 500*(1-0.05-0.1)-400*0.85=80(L). The result shows that under the current occupied volume, the refrigerator can still store 80L of items, and the space utilization rate is 84%, which still has room for optimization. If you want to further improve the available volume of the refrigerator, you can start from the following aspects: first, optimize the size and layout of the internal partitions to reduce unnecessary space; second, improve the air duct design and defrost system to minimize the reserved space while meeting the ventilation and heat dissipation requirements; third, strengthen inventory management and display skills to improve the compactness and orderliness of item placement; fourth, develop customized storage containers and auxiliary tools to use scattered space to store special items. Through the above measures, the available volume and space utilization efficiency of the refrigerator can be significantly improved without changing the total volume.

[0067] See also Figure 2 , Figure 2 An architecture diagram of a refrigeration equipment available volume measurement system provided in an embodiment of the present application, the refrigeration equipment available volume measurement system may include: A data acquisition module 1 is used to acquire ultrasonic measurement data measured by an ultrasonic measurement device and a visual image measured by a visual measurement device in the refrigeration equipment; The image feature extraction module 2 is used to perform image segmentation on the visual image to obtain a number of segmented images, and perform feature extraction on the segmented images to obtain image feature data; An ultrasonic three-dimensional point cloud model building module 3 is used to build a preliminary ultrasonic three-dimensional point cloud model inside the refrigeration equipment based on ultrasonic measurement data; The visual model building module 4 is used to extract features from the preliminary ultrasonic three-dimensional point cloud model to obtain three-dimensional point cloud graphic feature data, and integrate the three-dimensional point cloud graphic feature data with the three-dimensional point cloud graphic feature data to obtain target image feature data; A correction module 5 is used to construct a target visual three-dimensional model based on the target image feature data, and to correct the preliminary ultrasonic three-dimensional point cloud model based on the image feature data to obtain a target ultrasonic three-dimensional point cloud model; A fusion reconstruction module 6 is used to fuse and reconstruct the target visual 3D model and the target ultrasonic 3D point cloud model to obtain a target 3D model; The available volume calculation module 7 of the refrigeration equipment is used to determine the available volume of the refrigeration equipment based on the target three-dimensional model.

[0068] Please refer to Figure 3 The application also discloses an electronic device. Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0069] The communication bus 302 is used to realize the connection and communication between these components.

[0070] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0071] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0072] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field~Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0073] The memory 305 may include a random access memory (RAM) or a read-only memory (Read~Only Memory). The memory 305 includes a non-transitory computer-readable medium. The memory 305 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally be at least one storage system located away from the aforementioned processor 301. Refer to Figure 3 The memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a method for measuring the available capacity of a refrigeration device.

[0074] exist Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program storing the road assessment method in the memory 305. When executed by one or more processors 301, the electronic device 300 executes one or more methods in the above-mentioned embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application. In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0075] In the several implementations provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of the system or unit can be electrical or other forms.

[0076] The present application also provides a computer storage medium that can store multiple instructions, which are suitable for being loaded and executed by a processor as described above. Figure 1 The road assessment method of the embodiment shown in the figure can be found in the specific implementation process. Figure 1 The specific description of the illustrated embodiment will not be repeated here.

[0077] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0078] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes N instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0079] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0080] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for measuring the available volume of refrigeration equipment, characterized in that: Applied to a refrigeration device provided with an ultrasonic measuring device and a visual measuring device, the method comprises: Acquiring ultrasonic measurement data measured by the ultrasonic measurement device and visual images measured by the visual measurement device in the refrigeration equipment; Performing image segmentation on the visual image to obtain a plurality of segmented images, and performing feature extraction on the segmented images to obtain image feature data; constructing a preliminary ultrasonic three-dimensional point cloud model of the interior of the refrigeration equipment based on the ultrasonic measurement data; Extracting features from the preliminary ultrasonic three-dimensional point cloud model to obtain three-dimensional point cloud graphic feature data, and integrating the three-dimensional point cloud graphic feature data with the three-dimensional point cloud graphic feature data to obtain target image feature data; Constructing a target visual three-dimensional model based on the target image feature data, and correcting the preliminary ultrasonic three-dimensional point cloud model based on the image feature data to obtain a target ultrasonic three-dimensional point cloud model; The target visual three-dimensional model and the target ultrasonic three-dimensional point cloud model are fused and reconstructed to obtain a target three-dimensional model; The available volume of the refrigeration equipment is determined based on the target three-dimensional model.

2. The method according to claim 1, characterized in that The performing image segmentation on the visual image to obtain a plurality of segmented images, and performing feature extraction on the segmented images to obtain image feature data, includes: Performing image segmentation on the visual image based on an image segmentation algorithm to obtain the plurality of segmented images; Preprocessing the segmented image region to obtain a preprocessed image region; Extracting texture features from the preprocessed image area to obtain texture image feature data; Performing edge recognition on the processed image region to obtain an edge image of the image region, and calculating a geometric moment of the image region based on the edge image; Shape feature calculation is performed based on the geometric moment to obtain shape feature data, and the texture image feature data and the shape feature data are used as the image feature data.

3. The method according to claim 2, characterized in that The geometric moments include area moments, center distance moments, radial distance moments, composite moments and invariant moments; the shape feature calculation based on the geometric moments to obtain shape feature data includes: Calculate shape characteristics according to the area moment to determine area characteristic data; Calculate the shape feature according to the central moment to determine the feature data from the center to the boundary point; Calculate the shape feature according to the radial distance moment to determine the feature data from the shape center to the boundary point; Calculate the shape characteristics according to the composite moment to determine the overall morphological characteristic data of the shape; Calculate the shape characteristics according to the invariant moment to determine the characteristic data of the shape under geometric transformation; The area feature data, the feature data from the center to the boundary points of the shape, the overall morphological feature data of the shape and the feature data under the geometric transformation are used as the shape feature data.

4. The method according to claim 1, characterized in that The extracting features of the preliminary ultrasonic three-dimensional point cloud model to obtain three-dimensional point cloud graphic feature data, and integrating the three-dimensional point cloud graphic feature data with the three-dimensional point cloud graphic feature data to obtain target image feature data, includes: Performing point cloud segmentation on the preliminary ultrasonic three-dimensional point cloud model to obtain a plurality of object point clouds, and performing point cloud data extraction on the object point clouds to obtain point cloud data; The three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data are aligned according to the spatial position, a corresponding relationship between the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data is established, and the three-dimensional point cloud graphic feature data and the three-dimensional point cloud graphic feature data are fused by a feature fusion algorithm according to the corresponding relationship to obtain fused target image feature data.

5. The method according to claim 1, characterized in that The step of correcting the preliminary ultrasonic three-dimensional point cloud model based on the image feature data to obtain a target ultrasonic three-dimensional point cloud model includes: Performing spatial correspondence between the image feature data and the preliminary ultrasonic three-dimensional point cloud model to obtain an association relationship between the image feature data and the preliminary ultrasonic three-dimensional point cloud model, and performing corresponding correction between the preliminary ultrasonic three-dimensional point cloud model and the image feature data according to the association relationship to obtain corrected point cloud image data; The target ultrasonic three-dimensional point cloud model is constructed according to the point cloud image data.

6. The method according to claim 1, characterized in that The step of fusing and reconstructing the target visual three-dimensional model and the target ultrasonic three-dimensional point cloud model to obtain the target three-dimensional model includes: Placing the target visual three-dimensional model and the target ultrasonic three-dimensional point cloud model in the same coordinate system, aligning the coordinate systems of the target visual three-dimensional model and the target ultrasonic three-dimensional point cloud model through coordinate transformation, and obtaining a first target visual three-dimensional model and a first target ultrasonic three-dimensional point cloud model in the same coordinate system; Performing rigid body registration on the first target visual three-dimensional model and the first target ultrasonic three-dimensional point cloud model to obtain a registered second target visual three-dimensional model and a second target ultrasonic three-dimensional point cloud model; The second target visual three-dimensional model and the second target ultrasonic three-dimensional point cloud model are weightedly fused and reconstructed to obtain the target three-dimensional model.

7. The method according to claim 1, characterized in that The determining the available volume of the refrigeration equipment based on the target three-dimensional model includes: Determining a total volume of the refrigeration equipment based on the target three-dimensional model; Determining the occupied volume of the space occupied by items in the refrigeration equipment based on the target three-dimensional model; Based on the total volume and the occupied volume, an available volume of the refrigeration equipment is determined.

8. A refrigeration equipment available volume measurement system, characterized in that: Applicable to a refrigeration device provided with an ultrasonic measuring device and a visual measuring device, the system comprises: A data acquisition module, used to acquire ultrasonic measurement data measured by the ultrasonic measurement device and visual images measured by the visual measurement device in the refrigeration equipment; An image feature extraction module, used to perform image segmentation on the visual image to obtain a plurality of segmented images, and perform feature extraction on the segmented images to obtain image feature data; An ultrasonic three-dimensional point cloud model building module, used to build a preliminary ultrasonic three-dimensional point cloud model inside the refrigeration equipment based on the ultrasonic measurement data; A visual model building module is used to extract features from the preliminary ultrasonic three-dimensional point cloud model to obtain three-dimensional point cloud graphic feature data, and integrate the three-dimensional point cloud graphic feature data with the three-dimensional point cloud graphic feature data to obtain target image feature data; A correction module, used for constructing a target visual three-dimensional model based on the target image feature data, and correcting the preliminary ultrasonic three-dimensional point cloud model based on the image feature data to obtain a target ultrasonic three-dimensional point cloud model; A fusion reconstruction module, used for fusing the target visual 3D model and the target ultrasonic 3D point cloud model to obtain a target 3D model; The available volume calculation module of the refrigeration equipment is used to determine the available volume of the refrigeration equipment based on the target three-dimensional model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes a processor, a memory and a transceiver, the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 7.

Images