Cabinet equipment internal component space estimation method, apparatus, device, medium

Abstract

This application relates to a method, apparatus, electronic device, storage medium, and computer program product for spatial estimation of internal components of a cabinet-type device. The method includes: acquiring an image of the cabinet-type device to obtain a target image containing the cabinet-type device and multiple QR codes disposed on its surface; performing image recognition on each QR code to obtain its identifier pose in a preset coordinate system; recognizing the encoding information of the QR codes, and based on the encoding information, obtaining a first model of the cabinet-type device, a second model of the internal components of the cabinet-type device, and installation information of the internal components; determining multiple reference poses of the first model in the preset coordinate system based on the identifier pose information; calculating a preferred pose of the first model in the preset coordinate system based on the multiple reference poses; and transforming the preferred pose according to the installation information to obtain the target pose of the second model. This method can improve the accuracy of spatial estimation of internal components of cabinet-type devices.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, storage medium, and computer program product for spatial estimation of internal components of a cabinet-type device. Background Technology

[0002] With the development of virtual reality technology, more and more industries are applying it to actual production scenarios. A key aspect of virtual reality technology is spatial estimation of targets. Currently, spatial estimation methods mainly fall into two categories: one is real-time modeling of the target and its location, which can update in real time according to changes in features, but requires significant computing resources; the other is pre-modeling, where the target is modeled in advance and the existing model can be directly called upon during use, but this method struggles to update in real time according to changes in features.

[0003] Cabinet-type equipment is typically manufactured in a factory, and its internal components usually have consistent installation standards. Therefore, when applying virtual reality technology to scenarios such as the assembly, teaching, use, and maintenance of the internal components of cabinet-type equipment, a pre-modeling approach can often be adopted. This involves modeling the cabinet-type equipment itself and its internal components separately, and then spatially estimating the model positions of the internal components.

[0004] When performing spatial estimation of internal components of cabinet-type equipment, image recognition is typically used. First, the pose of the cabinet-type equipment as a whole or the QR code embedded in it is identified in the image to determine the location of the cabinet-type equipment model. Then, based on the installation standards of the internal components, the spatial positions of the internal components are estimated based on the location of the cabinet-type equipment model. However, this spatial estimation method is prone to errors between the estimated model position of the cabinet-type equipment and its actual position due to factors such as image quality and acquisition angle, which in turn leads to inaccuracies in the estimation of the model positions of the internal components. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for estimating the internal space of cabinet-type equipment in response to the above-mentioned technical problems.

[0006] In a first aspect, this application provides a method for estimating the internal space of cabinet-type equipment components, including:

[0007] Image acquisition is performed on the cabinet-type equipment to obtain a target image containing the cabinet-type equipment and multiple QR codes set on the surface of the cabinet-type equipment;

[0008] Image recognition is performed on each of the QR codes in the target image to obtain the identification pose of each QR code in a preset coordinate system;

[0009] The encoding information of the QR code is identified, and based on the encoding information, the first model of the cabinet equipment, the second model of the internal components of the cabinet equipment, and the installation information of the internal components of the cabinet equipment are obtained.

[0010] Based on the identification pose of each QR code, multiple reference poses corresponding to the first model are determined in the preset coordinate system;

[0011] Based on the plurality of reference poses, the preferred pose of the first model in the preset coordinate system is calculated; the total error between the preferred pose and the plurality of reference poses is minimized.

[0012] Based on the installation information, the preferred pose is transformed to obtain the target pose of the second model in the preset coordinate system.

[0013] In one embodiment, the plurality of QR codes includes a plurality of QR code groups; each QR code group includes a QR code disposed on two intersecting sides of the cabinet device; the step of determining a plurality of reference poses corresponding to the first model in the preset coordinate system based on the identifier poses of each QR code includes: determining each reference pose corresponding to the first model based on the identifier poses of each QR code in each QR code group; wherein, each QR code group corresponds one-to-one with each reference pose.

[0014] In one embodiment, determining the reference poses corresponding to the first model based on the identifier poses of each QR code in each QR code group includes: selecting a set of corner points from two surfaces of the first model corresponding to the two intersecting sides of the cabinet device as a set of feature points of the first model; determining a set of reference coordinates corresponding to the set of feature points based on the identifier poses of each QR code in the plurality of QR code groups; and determining the reference poses corresponding to the first model based on each set of reference coordinates.

[0015] In one embodiment, calculating the preferred pose of the first model in the preset coordinate system based on the plurality of reference poses includes: using the mutual positional relationship of each feature point in the feature point group as a constraint, calculating the preferred coordinate group corresponding to the feature point group based on the plurality of reference coordinate groups corresponding to the plurality of reference pose information; minimizing the total distance between the preferred coordinate group and the plurality of reference coordinate groups; and determining the preferred pose based on the preferred coordinate group.

[0016] In one embodiment, the preset coordinate system is a three-dimensional Cartesian coordinate system with the plane containing the bottom surface of the cabinet device as the xy plane; determining the reference poses corresponding to the first model based on the identifier poses of the QR codes in each QR code group includes: selecting two surfaces in the first model corresponding to the two intersecting sides of the cabinet device as feature surface groups of the first model; determining multiple reference plane groups corresponding to the feature surface groups based on the identifier poses of the QR codes in each QR code group; each reference plane group includes a reference plane corresponding to each surface in the feature surface group; determining the reference pose of the first model corresponding to each reference plane group based on the intersection lines between the reference planes in each reference plane group and the intersection lines between each reference plane and the xy plane.

[0017] In one embodiment, before performing image recognition on each of the QR codes in the target image to obtain the identification pose of each QR code in a preset coordinate system, the method further includes: if the acquisition angle of the target image is not parallel to the xy plane of the preset coordinate system, then the target image is corrected to obtain the corrected target image.

[0018] Secondly, this application also provides a space estimation device for internal components of cabinet-type equipment, comprising:

[0019] The image acquisition module is used to acquire images of the cabinet-type equipment to obtain a target image containing the cabinet-type equipment and multiple QR codes set on the surface of the cabinet-type equipment;

[0020] The image recognition module is used to perform image recognition on each of the QR codes in the target image to obtain the identification pose of each QR code in a preset coordinate system;

[0021] The acquisition module is used to identify the encoding information of the QR code, and based on the encoding information, acquire the first model of the cabinet equipment, the second model of the internal components of the cabinet equipment, and the installation information of the internal components of the cabinet equipment;

[0022] The reference pose determination module is used to determine multiple reference poses corresponding to the first model in the preset coordinate system based on the identification pose information of each QR code.

[0023] The preferred pose determination module is used to calculate the preferred pose of the first model in the preset coordinate system based on the plurality of reference poses; the total error between the preferred pose and the plurality of reference poses is minimized.

[0024] The target pose determination module is used to perform conversion processing on the preferred pose based on the installation information to obtain the target pose of the second model in the preset coordinate system.

[0025] Thirdly, this application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0026] Image acquisition is performed on the cabinet-type equipment to obtain a target image containing the cabinet-type equipment and multiple QR codes set on the surface of the cabinet-type equipment;

[0027] Image recognition is performed on each of the QR codes in the target image to obtain the identification pose of each QR code in a preset coordinate system;

[0028] The encoding information of the QR code is identified, and based on the encoding information, the first model of the cabinet equipment, the second model of the internal components of the cabinet equipment, and the installation information of the internal components of the cabinet equipment are obtained.

[0029] Based on the identification pose of each QR code, multiple reference poses corresponding to the first model are determined in the preset coordinate system;

[0030] Based on the plurality of reference poses, the preferred pose of the first model in the preset coordinate system is calculated; the total error between the preferred pose and the plurality of reference poses is minimized.

[0031] Based on the installation information, the preferred pose is transformed to obtain the target pose of the second model in the preset coordinate system.

[0032] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0033] Image acquisition is performed on the cabinet-type equipment to obtain a target image containing the cabinet-type equipment and multiple QR codes set on the surface of the cabinet-type equipment;

[0034] Image recognition is performed on each of the QR codes in the target image to obtain the identification pose of each QR code in a preset coordinate system;

[0035] The encoding information of the QR code is identified, and based on the encoding information, the first model of the cabinet equipment, the second model of the internal components of the cabinet equipment, and the installation information of the internal components of the cabinet equipment are obtained.

[0036] Based on the identification pose of each QR code, multiple reference poses corresponding to the first model are determined in the preset coordinate system;

[0037] Based on the plurality of reference poses, the preferred pose of the first model in the preset coordinate system is calculated; the total error between the preferred pose and the plurality of reference poses is minimized.

[0038] Based on the installation information, the preferred pose is transformed to obtain the target pose of the second model in the preset coordinate system.

[0039] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0040] Image acquisition is performed on the cabinet-type equipment to obtain a target image containing the cabinet-type equipment and multiple QR codes set on the surface of the cabinet-type equipment;

[0041] Image recognition is performed on each of the QR codes in the target image to obtain the identification pose of each QR code in a preset coordinate system;

[0042] The encoding information of the QR code is identified, and based on the encoding information, the first model of the cabinet equipment, the second model of the internal components of the cabinet equipment, and the installation information of the internal components of the cabinet equipment are obtained.

[0043] Based on the identification pose of each QR code, multiple reference poses corresponding to the first model are determined in the preset coordinate system;

[0044] Based on the plurality of reference poses, the preferred pose of the first model in the preset coordinate system is calculated; the total error between the preferred pose and the plurality of reference poses is minimized.

[0045] Based on the installation information, the preferred pose is transformed to obtain the target pose of the second model in the preset coordinate system.

[0046] The aforementioned method, apparatus, electronic device, storage medium, and computer program product for spatial estimation of internal components of cabinet-type equipment first acquires images of the cabinet-type equipment to obtain a target image containing the cabinet-type equipment and multiple QR codes set on the surface of the cabinet-type equipment. Then, image recognition is performed on each QR code in the target image to obtain the identification pose of each QR code in a preset coordinate system. By recognizing the encoding information of the QR codes, a first model of the cabinet-type equipment, a second model of the internal components of the cabinet-type equipment, and the installation information of the internal components of the cabinet-type equipment are obtained based on the encoding information. Then, based on the identification pose information of each QR code, multiple reference poses corresponding to the first model are determined in the preset coordinate system. Then, based on the multiple reference poses, the optimal pose with the minimum total error between the first model and the multiple reference poses in the preset coordinate system is calculated. Finally, the optimal pose is transformed based on the installation information to obtain the target pose of the second model in the preset coordinate system. This scheme identifies multiple QR codes on the surface of the cabinet-type equipment to determine multiple reference poses for the corresponding first model. Then, it calculates the optimal pose with the minimum total error by combining these reference poses. This effectively reduces errors caused by low image quality, poor acquisition angles, and abnormal QR code settings, resulting in an optimal pose that accurately reflects the actual position of the cabinet-type equipment. Using this optimal pose, the target pose of the second model can accurately estimate the position of internal components, effectively improving the accuracy of spatial estimation of these components. Furthermore, this scheme leverages the coded information carrying capabilities of QR codes and their ease of identification and positioning. By using QR codes as positioning markers for pose recognition and as carriers of information about the first model, second model, and installation, the implementation cost of the scheme can be effectively reduced. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a diagram illustrating the application environment of a method for estimating the internal component space of a cabinet-type device in one embodiment.

[0049] Figure 2 This is a flowchart illustrating a method for estimating the internal space of cabinet-type equipment in one embodiment;

[0050] Figure 3 This is a flowchart illustrating the steps for determining the reference pose of a first model in one embodiment.

[0051] Figure 4 This is a schematic diagram of the feature point group of the first model in one embodiment;

[0052] Figure 5 This is a flowchart illustrating the steps for determining the reference pose of the first model in another embodiment;

[0053] Figure 6 This is a schematic diagram of the reference pose of a first model determined based on the target images before and after correction in one embodiment.

[0054] Figure 7 This is a structural block diagram of a cabinet-type equipment internal component space estimation device in one embodiment;

[0055] Figure 8 This is a diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] The internal component space estimation method for cabinet-type equipment provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 can acquire images of the cabinet-type device and its surrounding environment and perform image recognition processing. Server 104 can store data such as the first model, the second model, and installation information of internal components of the cabinet-type device through the data storage system. Terminal 102 can be, but is not limited to, various virtual reality (VR) devices, augmented reality (AR) devices, mixed reality (MR) devices, extended reality (XR) devices, emulated reality (ER) devices, brain-computer interfaces, and other interactive devices. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0058] In one exemplary embodiment, such as Figure 2 As shown, a method for estimating the space of internal components of a cabinet-type device is provided, which can be applied to... Figure 1 Taking the terminal in the example, the following steps are included.

[0059] Step S201: Image acquisition is performed on the cabinet-type device to obtain a target image containing the cabinet-type device and multiple QR codes set on the surface of the cabinet-type device.

[0060] Among them, cabinet-type equipment can be equipment such as substation cabinets, which can be equipped with internal components such as air switches and terminal blocks.

[0061] The surface of the cabinet-type equipment can be equipped with multiple QR codes. These QR codes can be placed in different locations on the surface of the cabinet-type equipment through methods such as pasting or printing.

[0062] In this step, the terminal can capture images of the cabinet-type device to obtain a target image that includes the appearance of the cabinet-type device and multiple QR codes set on its surface.

[0063] Step S202: Perform image recognition on each QR code in the target image to obtain the identification pose of each QR code in the preset coordinate system.

[0064] The preset coordinate system can be a three-dimensional rectangular coordinate system set according to the environment in which the cabinet equipment is located. For example, it can be a three-dimensional rectangular coordinate system with a point on the ground in the environment in which the cabinet equipment is located as the origin and the ground as the xy plane.

[0065] In this step, by performing image recognition on each QR code in the target image, the corresponding identifier pose can be obtained. The identifier pose for each QR code can include the coordinates of its four corner points in a preset coordinate system.

[0066] Step S203: Identify the encoding information of the QR code, and based on the encoding information, obtain the first model of the cabinet equipment, the second model of the internal components of the cabinet equipment, and the installation information of the internal components of the cabinet equipment.

[0067] The first model can be a 3D virtual model obtained by pre-modeling the cabinet-type equipment, and the second model can be a 3D virtual model obtained by pre-modeling the internal components of the cabinet-type equipment. The installation information of the internal components of the cabinet-type equipment can include information such as the installation position and installation direction of the internal components in the cabinet-type equipment.

[0068] The QR code on the surface of the cabinet equipment can contain information such as the equipment model and the model of the internal components contained in the cabinet equipment. Using the equipment model and the model of the internal components, the first model of the cabinet equipment, the second model of the internal components, and the installation information of the internal components in the cabinet equipment can be retrieved from the data storage system.

[0069] Step S204: Determine multiple reference poses corresponding to the first model in a preset coordinate system based on the identification poses of each QR code.

[0070] The reference pose corresponding to the first model may include the coordinate information of multiple feature points in the first model in the preset coordinate system, or the coordinate information of the center point of the first model in the preset coordinate system and the pose information of the model in the preset coordinate system.

[0071] For example, in this step, the coordinates of multiple feature points, such as corner points, on the surface of the cabinet device can be calculated from the identifier pose of each QR code based on the positional relationship between each QR code in the target image and the surface of the cabinet device. Then, based on the fixed positional relationship between the surfaces in the first model, the pose of the entire model in the preset coordinate system can be determined.

[0072] It is understandable that in the multiple reference poses obtained in this step, each reference pose can be determined individually by the identifier pose of each QR code, or it can be determined based on the combination of the identifier poses of two or more QR codes.

[0073] Step S205: Calculate the preferred pose of the first model in the preset coordinate system based on multiple reference poses.

[0074] Specifically, based on multiple reference poses, an optimal pose of the first model can be determined in a preset coordinate system, minimizing the total error between the optimal pose and the multiple reference poses. The total error between the optimal pose and the multiple reference poses can be the sum of the errors between the optimal pose and each reference pose; the higher the similarity between two poses, the smaller the error between them.

[0075] For example, in this step, an optimization algorithm can be used to determine the preferred pose of the first model. First, one or more hypothetical poses of the first model can be determined in a preset coordinate system. For each hypothetical pose, its similarity to each reference pose is calculated, and the error between the hypothetical pose and each reference pose is determined based on the similarity. Then, the total error between the hypothetical pose and all reference poses can be obtained. Next, it can be determined whether the total error between each hypothetical pose and multiple reference poses is less than a preset threshold. If the total error corresponding to one or more hypothetical poses is less than the preset threshold, the hypothetical pose with the smallest total error can be taken as the preferred pose of the first model. Otherwise, the optimization algorithm can be used to iteratively update each hypothetical pose, repeatedly calculating the total error of each hypothetical pose after each update until the preferred pose is obtained.

[0076] Step S206: Based on the installation information, the preferred pose is transformed to obtain the target pose of the second model in the preset coordinate system.

[0077] Based on the installation information, the installation position and orientation of internal components within the cabinet-type equipment can be determined. Therefore, in this step, the pose relationship between the second model and the first model can be determined based on the installation information. Furthermore, based on the preferred pose of the first model, this step can transform the preferred pose of the first model using matrix transformations or similar methods to obtain the target pose of the second model.

[0078] In the above-mentioned method for spatial estimation of internal components of cabinet equipment, the cabinet equipment is first image acquired to obtain a target image containing the cabinet equipment and multiple QR codes set on the surface of the cabinet equipment. Then, image recognition is performed on each QR code in the target image to obtain the identification pose of each QR code in a preset coordinate system. By recognizing the encoding information of the QR codes, the first model of the cabinet equipment, the second model of the internal components of the cabinet equipment, and the installation information of the internal components of the cabinet equipment are obtained based on the encoding information. Then, based on the identification pose information of each QR code, multiple reference poses corresponding to the first model are determined in the preset coordinate system. Then, based on the multiple reference poses, the optimal pose with the smallest total error between the first model and the multiple reference poses in the preset coordinate system is calculated. Finally, the optimal pose is transformed based on the installation information to obtain the target pose of the second model in the preset coordinate system. This scheme identifies multiple QR codes on the surface of the cabinet-type equipment to determine multiple reference poses for the corresponding first model. Then, it calculates the optimal pose with the minimum total error by combining these reference poses. This effectively reduces errors caused by low image quality, poor acquisition angles, and abnormal QR code settings, resulting in an optimal pose that accurately reflects the actual position of the cabinet-type equipment. Using this optimal pose, the target pose of the second model can accurately estimate the position of internal components, effectively improving the accuracy of spatial estimation of these components. Furthermore, this scheme leverages the coded information carrying capabilities of QR codes and their ease of identification and positioning. By using QR codes as positioning markers for pose recognition and as carriers of information about the first model, second model, and installation, the implementation cost of the scheme can be effectively reduced.

[0079] In an exemplary embodiment, the multiple QR codes may include multiple QR code groups, and each QR code group may include QR codes disposed on two intersecting sides of the cabinet-type device. The step of determining multiple reference poses corresponding to the first model in a preset coordinate system based on the identifier poses of each QR code may include: determining each reference pose corresponding to the first model based on the identifier poses of each QR code in each QR code group; wherein, each QR code group corresponds one-to-one with each reference pose.

[0080] The multiple QR codes set on the cabinet surface can include multiple QR codes respectively set on two intersecting sides of the cabinet equipment. Based on this, these QR codes can be grouped so that each QR code group includes QR codes set on the two intersecting sides. Then, for each QR code group, the pose of the corresponding surface in the first model in a preset coordinate system can be determined based on the identifier poses of the QR codes on different sides of the cabinet equipment. Therefore, by combining the identifier poses of each QR code in the same group, a reference pose corresponding to the first model of the cabinet equipment can be determined.

[0081] For example, the cabinet-type device may include intersecting first and second sides, and the QR codes on its surface may include N QR codes on the first side and N QR codes on the second side. Based on this, the 2N QR codes can be grouped into N QR code groups, and each QR code group may contain two QR codes respectively set on the first and second sides. Then, based on the identifier poses of each QR code in the N QR code groups, N reference poses corresponding to the first model can be obtained.

[0082] In this embodiment, multiple QR codes are set on two intersecting sides of the cabinet-type device, forming multiple QR code groups. Different QR code groups are then used to determine different reference poses of the first model. This scheme can determine the pose of the corresponding surface in the first model based on the identifier pose of the QR codes set on different sides of the cabinet-type device in the QR code group. Then, by combining the poses of the corresponding two intersecting surfaces in the first model, the reference pose of the first model can be determined. This can better eliminate the influence of recognition errors and obtain a more accurate reference pose.

[0083] In one exemplary embodiment, such as Figure 3 As shown, the steps described above for determining the reference poses corresponding to the first model based on the identifier poses of each QR code in each QR code group may include:

[0084] Step S301: Select the corner point group of the two surfaces corresponding to the two intersecting sides of the cabinet equipment in the first model as the feature point group of the first model.

[0085] In this step, based on the two intersecting sides of the cabinet-type equipment involved in the multiple QR code groups, the two corresponding surfaces in the first model can be determined, and the corner point group composed of multiple corner points of each of these two surfaces can be used as the feature point group of the first model.

[0086] Step S302: Determine multiple reference coordinate groups corresponding to the feature point group based on the identifier pose of each QR code in the multiple QR code groups.

[0087] Each reference coordinate group contains the reference coordinates of each feature point in the feature point group. Specifically, for each QR code group, the coordinates of the corner points of the surface on which the QR code is located can be calculated in a preset coordinate system based on the positional relationship between each QR code and the surface of the cabinet device it is situated on, using the identifier pose of each QR code. Then, by combining the corner point coordinates of different surfaces calculated from the identifier poses of different QR codes in the QR code group, the reference coordinate information corresponding to each feature point in the feature point group of the first model can be obtained.

[0088] Step S303: Determine the reference poses corresponding to the first model based on each reference coordinate group.

[0089] Based on the multiple reference coordinate sets obtained in step S302, this step can combine the fixed positional relationships between the surfaces in the first model to determine a reference pose corresponding to the first model for each reference coordinate set. For example, as shown... Figure 4 As shown, the feature point set of the first model may include corner point A of its first surface, corner point C of its second surface, and corner point B shared by the first and second surfaces. The reference coordinate set may include the reference coordinates of points A, B, and C respectively. Since the positional relationships between the surfaces in the first model are relatively fixed, a unique reference pose of the first model can be determined in a preset coordinate system based on the reference coordinates of feature points A, B, and C. Subsequently, multiple reference poses corresponding to the first model can be obtained from multiple reference coordinate sets.

[0090] In this embodiment, based on the characteristic that the positions of different points in the first model are relatively fixed, the corner point group of the two surfaces corresponding to the two intersecting sides of the cabinet device in the first model is selected as the feature point group. In this way, the reference coordinates of each feature point in the feature point group can be calculated conveniently and quickly according to the identification pose of the QR code set on the two sides in each QR code group. Then, different reference poses of the first model can be determined according to different reference coordinate groups.

[0091] In an exemplary embodiment, the step of calculating the preferred pose of the first model in a preset coordinate system based on multiple reference poses may include: using the mutual positional relationship between each feature point in the feature point group as a constraint, calculating the preferred coordinate group corresponding to the feature point group based on multiple reference coordinate groups corresponding to multiple reference pose information; and determining the preferred pose based on the preferred coordinate group.

[0092] Since the pose of the first model can be uniquely determined based on the coordinate positions of the feature point sets, the similarity between two poses can be determined by calculating the distance between the coordinate sets of the feature point sets corresponding to two different poses of the first model. The greater the distance between the coordinate sets, the lower the similarity between the two poses; conversely, the shorter the distance, the higher the similarity. Therefore, by calculating the optimal coordinate set with the shortest total distance to multiple reference coordinate sets, the optimal pose with the highest overall similarity and the smallest total error with each reference pose can be determined.

[0093] Based on this, in this embodiment, one or more hypothetical poses of the first model can be determined first in a preset coordinate system, and a hypothetical coordinate group corresponding to the feature point group can be determined according to each hypothetical pose. Then, for each hypothetical pose, the total distance D between its corresponding hypothetical coordinate group and multiple reference coordinate groups is calculated. Taking the total distance D being less than a preset threshold as the optimization objective, and using the mutual positional relationship of each feature point in the feature point group as a constraint, an optimization algorithm is used to iteratively update each hypothetical coordinate group until one or more hypothetical coordinate groups that can make the total distance D less than the preset threshold are obtained. The hypothetical coordinate group with the smallest corresponding total distance D is taken as the preferred coordinate group of the first model, and the preferred pose of the first model in the preset coordinate system is determined by the preferred coordinate group.

[0094] Among them, still with Figure 4 For example, the feature point set of the first model may include three feature points A, B, and C, corresponding to a hypothetical pose of the first model. This yields a hypothetical coordinate set containing the hypothetical coordinates of each of these three feature points. The total distance D between this hypothetical coordinate set and multiple reference coordinate sets can be expressed as follows:

[0095]

[0096] in, Let A be the imaginary coordinates of feature point A. Let B be the imaginary coordinates of feature point B. Let C be the imaginary coordinates of the feature point. Let A be the reference coordinate of feature point A in the i-th reference coordinate group. Let B be the reference coordinate of feature point B in the i-th reference coordinate group. Let N be the reference coordinates of feature point C in the i-th reference coordinate group, and N be the number of reference coordinate groups.

[0097] In some exemplary embodiments, the preset coordinate system may be a three-dimensional rectangular coordinate system with the plane containing the bottom surface of the cabinet device as the xy plane.

[0098] Therefore, when the first and second sides of the cabinet-type equipment are perpendicular to each other and respectively perpendicular to the bottom surface, it is still considered as... Figure 4For example, it can be determined that feature points A, B, and C in the first model are all located in the xy plane, and the first surface where feature points A and B are located and the second surface where feature points B and C are located are perpendicular to each other. Furthermore, both the first and second surfaces are perpendicular to the xy plane of the preset coordinate system. Therefore, it can be determined that there are mutual positional constraints between the feature points in the feature point group as shown in the following formula:

[0099]

[0100]

[0101] Therefore, based on this mutual positional constraint, the hypothetical coordinate set of the feature point group can be iteratively updated during the process of determining the preferred coordinate set of the feature point group, which can reduce the amount of total distance calculation and improve the efficiency of obtaining the preferred coordinate set.

[0102] In this embodiment, by simplifying the similarity calculation between different poses in the process of optimizing the pose calculation to the distance calculation between different coordinate groups of feature point groups, the optimization efficiency can be effectively improved and the optimal pose of the first model can be quickly determined.

[0103] In an exemplary embodiment, the aforementioned preset coordinate system is a three-dimensional Cartesian coordinate system with the plane containing the bottom surface of the cabinet device as the xy plane. The steps described above for determining the reference poses corresponding to the first model based on the identifier poses of each QR code in each QR code group are as follows: Figure 5 As shown, it may include:

[0104] Step S501: Select two surfaces in the first model that correspond to the two intersecting sides of the cabinet-type equipment as the feature surface group of the first model.

[0105] Step S502: Based on the identifier pose of the QR codes in each QR code group, determine multiple reference plane groups corresponding to the feature surface group. Each reference plane group includes the reference plane corresponding to each surface in the feature surface group.

[0106] Step S503: Determine the reference pose of the first model corresponding to each reference plane group based on the intersection lines between the reference planes in each reference plane group and the intersection lines between each reference plane and the xy plane.

[0107] In this embodiment, a three-dimensional Cartesian coordinate system can be pre-established, with the plane containing the bottom surface of the cabinet device as the xy-plane, and this system can be used as a preset coordinate system. For example, when the cabinet device is installed on the ground, a point on the ground outside the cabinet device can be used as the origin of the coordinate system, and the ground can be used as the xy-plane of the preset coordinate system. Based on this preset coordinate system, in this embodiment, the reference pose of the first model in the preset coordinate system can be determined by identifying the poses of the two intersecting sides of the cabinet device.

[0108] In step S501, based on the two intersecting sides of the cabinet-type device involved in the multiple QR code groups, two surfaces corresponding to them in the first model can be determined, and these two surfaces can be used as a feature surface group of the first model. For example, based on the first side and the second side of the cabinet-type device involved in the multiple QR code groups, the first surface corresponding to the first side and the second surface corresponding to the second side in the first model can be used as two feature surfaces of the first model to form a feature surface group.

[0109] Then, in step S502, for each QR code group, the plane equation corresponding to the surface where each QR code is located in the preset coordinate system can be determined according to the identifier pose of each QR code. The plane represented by the plane equation is used as the reference plane of the corresponding surface in the first model in the preset coordinate system. Thus, by combining the reference planes corresponding to the first surface and the second surface in the first model, a reference plane group corresponding to the first model can be obtained. Therefore, multiple reference plane groups of the first model can be obtained according to the identifier pose of each QR code in multiple QR code groups.

[0110] For example, the plane equations of the reference planes corresponding to the first and second surfaces can be expressed as follows:

[0111]

[0112]

[0113] in, , , , The coefficients of the plane equation for the reference plane corresponding to the first surface. , , , Let be the plane equation coefficients of the reference plane corresponding to the second surface, and let the normal vectors of the two reference planes be respectively. , .

[0114] When the first and second sides of the cabinet-type equipment are perpendicular to each other and each perpendicular to the bottom surface, it can also be determined that the reference planes corresponding to the first and second surfaces in the first model are perpendicular to each other, and both are perpendicular to the xy plane of the preset coordinate system. Based on this, the parameters of the above two plane equations can be further determined to satisfy the following formula:

[0115]

[0116]

[0117] Therefore, based on the constraints of the above formula, the plane equations of each reference plane group corresponding to the first model can be determined more conveniently and accurately, further improving the overall processing efficiency and reducing errors.

[0118] Furthermore, in step S503, for each group of reference planes, the intersection line between two reference planes can be determined, as well as the intersection line between each reference plane and the xy plane of the preset coordinate system. Thus, combined with the fixed positional relationship between each surface in the first model, the reference pose of the entire model in the preset coordinate system can be determined.

[0119] In this embodiment, the plane where the bottom surface of the cabinet equipment is located is used as the xy plane of the preset coordinate system, and the reference plane group corresponding to the feature surface group in the first model is determined according to the identification pose of the QR code in each QR code group. Thus, the reference pose of the first model can be efficiently and accurately determined in the preset coordinate system according to the intersection lines between the reference planes in each reference plane group and between the reference plane and the xy plane.

[0120] In an exemplary embodiment, before performing image recognition on each QR code in the target image to obtain the identification pose of each QR code in the preset coordinate system, the method may further include: if the acquisition angle of the target image is not parallel to the xy plane of the preset coordinate system, then the target image is corrected to obtain a corrected target image.

[0121] The target image acquisition angle refers to the angle at which the terminal acquires the image. When the terminal's image acquisition angle is not parallel to the xy plane of the preset coordinate system, it can easily lead to abnormal shapes of the QR code in the target image, making it difficult to guarantee the accuracy of the QR code's identification pose in the target image, and consequently affecting the determination of the reference pose of the subsequent first model.

[0122] Based on this, in this embodiment, the gyroscope equipped on the terminal can be used to detect the acquisition angle of the target image and determine whether the angle is parallel to the xy plane of the preset coordinate system (i.e., the plane where the bottom surface of the cabinet device is located). If it is determined that they are not parallel, the target image can be corrected based on the angle between the acquisition angle and the xy plane to obtain a corrected target image. Subsequently, the identification pose of each QR code can be more accurately identified based on the corrected target image.

[0123] Taking a cabinet-type device with perpendicular sides and bottom as an example, after identifying the pose of the QR code on the side of the cabinet-type device in the target image, the pose of the feature surface corresponding to that side in the first model in the preset coordinate system can be determined. For example, Figure 6As shown, when the acquisition angle of the target image is not parallel to the xy plane, the pose of the QR code obtained from the uncorrected target image can be determined as follows: Figure 6 The first pose of the feature surface shown in the dashed box, and the marker pose of the QR code obtained from the corrected target image, can then be determined as follows: Figure 6 The second pose of the feature surface is shown in the solid line frame. It can be seen that the first pose of the feature surface obtained from the uncorrected target image is not perpendicular to the xy plane of the preset coordinate system, which contradicts the perpendicular relationship between the side and bottom surfaces of the cabinet equipment in reality. However, the second pose of the feature surface obtained from the corrected target image is perpendicular to the xy plane, which conforms to the positional relationship between the side and bottom surfaces of the cabinet equipment in reality.

[0124] In this embodiment, when the acquisition angle of the target image is not parallel to the xy plane of the preset coordinate system, the target image is corrected. This can avoid the influence of the image acquisition angle on the identification pose of the QR code, which can improve the accuracy of the reference pose of the subsequently determined first model, and thus improve the accuracy of the preferred pose of the subsequently determined first model and the target pose of the second model.

[0125] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0126] Based on the same inventive concept, this application also provides a cabinet-type equipment internal component space estimation device for implementing the above-mentioned cabinet-type equipment internal component space estimation method. The solution provided by this device is similar to the solution described in the above-described method. Therefore, the specific limitations of one or more cabinet-type equipment internal component space estimation device embodiments provided below can be found in the limitations of the cabinet-type equipment internal component space estimation method described above, and will not be repeated here.

[0127] In one exemplary embodiment, such as Figure 7 As shown, a cabinet-type equipment internal component space estimation device 700 is provided, comprising:

[0128] Image acquisition module 701 is used to acquire images of cabinet-type equipment to obtain a target image containing the cabinet-type equipment and multiple QR codes set on the surface of the cabinet-type equipment;

[0129] Image recognition module 702 is used to perform image recognition on each of the QR codes in the target image to obtain the identification pose of each QR code in a preset coordinate system;

[0130] The acquisition module 703 is used to identify the encoding information of the QR code, and acquire the first model of the cabinet equipment, the second model of the internal components of the cabinet equipment, and the installation information of the internal components of the cabinet equipment based on the encoding information.

[0131] The reference pose determination module 704 is used to determine multiple reference poses corresponding to the first model in the preset coordinate system based on the identification pose information of each QR code.

[0132] The preferred pose determination module 705 is used to calculate the preferred pose of the first model in the preset coordinate system based on the plurality of reference poses; the total error between the preferred pose and the plurality of reference poses is minimized.

[0133] The target pose determination module 706 is used to perform conversion processing on the preferred pose according to the installation information to obtain the target pose of the second model in the preset coordinate system.

[0134] In an exemplary embodiment, the plurality of QR codes includes a plurality of QR code groups; each QR code group includes a QR code disposed on two intersecting sides of the cabinet device; the reference pose determination module 704 is further configured to: determine each reference pose corresponding to the first model according to the identifier pose of each QR code in each QR code group; wherein, each QR code group corresponds one-to-one with each reference pose.

[0135] In an exemplary embodiment, the reference pose determination module 704 is further configured to: select a set of corner points of two surfaces corresponding to the two intersecting sides of the cabinet device in the first model as a set of feature points of the first model; determine a plurality of reference coordinate groups corresponding to the set of feature points according to the identifier pose of each of the plurality of QR codes in the plurality of QR code groups; and determine each of the reference poses corresponding to the first model according to each of the reference coordinate groups.

[0136] In an exemplary embodiment, the preferred pose determination module 705 is further configured to: use the mutual positional relationship of each feature point in the feature point group as a constraint, calculate a preferred coordinate group corresponding to the feature point group according to the multiple reference coordinate groups corresponding to the multiple reference pose information; the preferred coordinate group has the shortest total distance to the multiple reference coordinate groups; and determine the preferred pose according to the preferred coordinate group.

[0137] In an exemplary embodiment, the preset coordinate system is a three-dimensional Cartesian coordinate system with the plane containing the bottom surface of the cabinet device as the xy plane; the reference pose determination module 704 is further configured to: select two surfaces in the first model corresponding to the two intersecting sides of the cabinet device as feature surface groups of the first model; determine multiple reference plane groups corresponding to the feature surface groups according to the identifier pose of the QR codes in each QR code group; each reference plane group includes a reference plane corresponding to each surface in the feature surface group; determine the reference pose of the first model corresponding to each reference plane group according to the intersection line between the two surfaces in each reference plane group and the intersection line between each surface and the xy plane.

[0138] In an exemplary embodiment, the device further includes a correction module, configured to perform correction processing on the target image if the acquisition angle of the target image is not parallel to the xy plane of the preset coordinate system, thereby obtaining the corrected target image.

[0139] Each module in the aforementioned cabinet-type equipment internal component space estimation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the electronic device in hardware form or independent of it, or stored in the memory of the electronic device in software form, so that the processor can call and execute the operations corresponding to each module.

[0140] In one exemplary embodiment, an electronic device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the electronic device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for estimating the internal component space of a cabinet-type device. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The input device of the electronic device can be a touch layer covering the display screen, or a button, trackball, or touchpad set on the casing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0141] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0143] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0144] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0146] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for estimating the internal space of cabinet-type equipment, characterized in that, The method includes: Image acquisition is performed on the cabinet-type equipment to obtain a target image containing the cabinet-type equipment and multiple QR codes set on the surface of the cabinet-type equipment; Image recognition is performed on each of the QR codes in the target image to obtain the identification pose of each QR code in a preset coordinate system; The encoding information of the QR code is identified, and based on the encoding information, the first model of the cabinet equipment, the second model of the internal components of the cabinet equipment, and the installation information of the internal components of the cabinet equipment are obtained. Based on the identification pose of each QR code, multiple reference poses corresponding to the first model are determined in the preset coordinate system; Based on the plurality of reference poses, the preferred pose of the first model in the preset coordinate system is calculated; the total error between the preferred pose and the plurality of reference poses is minimized. Based on the installation information, the preferred pose is transformed to obtain the target pose of the second model in the preset coordinate system.

2. The method according to claim 1, characterized in that, The plurality of QR codes includes a plurality of QR code groups; each QR code group includes a QR code disposed on two intersecting sides of the cabinet-type equipment; The step of determining multiple reference poses corresponding to the first model in the preset coordinate system based on the identifier poses of each of the QR codes includes: Based on the identifier pose of each QR code in each QR code group, the reference pose corresponding to the first model is determined respectively; wherein, each QR code group corresponds one-to-one with each reference pose.

3. The method according to claim 2, characterized in that, The step of determining the reference poses corresponding to the first model based on the identifier poses of each QR code in each QR code group includes: The corner point group of the two surfaces corresponding to the two intersecting sides of the cabinet equipment in the first model is selected as the feature point group of the first model; Based on the identifier pose of each QR code in the plurality of QR code groups, determine a plurality of reference coordinate groups corresponding to the feature point group; Based on each of the reference coordinate groups, determine each of the reference poses corresponding to the first model.

4. The method according to claim 3, characterized in that, The step of calculating the preferred pose of the first model in the preset coordinate system based on the plurality of reference poses includes: Using the relative positions of the feature points in the feature point group as constraints, and based on the multiple reference coordinate groups corresponding to the multiple reference pose information, a preferred coordinate group corresponding to the feature point group is calculated; the preferred coordinate group has the shortest total distance to the multiple reference coordinate groups. The preferred pose is determined based on the preferred coordinate set.

5. The method according to claim 2, characterized in that, The preset coordinate system is a three-dimensional rectangular coordinate system with the plane containing the bottom surface of the cabinet equipment as the xy plane; The step of determining the reference poses corresponding to the first model based on the identifier poses of each QR code in each QR code group includes: Two surfaces in the first model that correspond to the two intersecting sides of the cabinet-type equipment are selected as the feature surface group of the first model; Based on the identifier pose of the QR code in each QR code group, a plurality of reference plane groups corresponding to the feature surface group are determined; each reference plane group includes a reference plane corresponding to each surface in the feature surface group. The reference pose of the first model corresponding to each reference plane group is determined based on the intersection lines between the reference planes in each reference plane group and the intersection lines between each reference plane and the xy plane.

6. The method according to claim 5, characterized in that, Before performing image recognition on each of the QR codes in the target image to obtain the identifier pose of each QR code in a preset coordinate system, the method further includes: If the acquisition angle of the target image is not parallel to the xy plane of the preset coordinate system, the target image is corrected to obtain the corrected target image.

7. A device for estimating the internal space of cabinet-type equipment, characterized in that, The device includes: The image acquisition module is used to acquire images of the cabinet-type equipment to obtain a target image containing the cabinet-type equipment and multiple QR codes set on the surface of the cabinet-type equipment; The image recognition module is used to perform image recognition on each of the QR codes in the target image to obtain the identification pose of each QR code in a preset coordinate system; The acquisition module is used to identify the encoding information of the QR code, and based on the encoding information, acquire the first model of the cabinet equipment, the second model of the internal components of the cabinet equipment, and the installation information of the internal components of the cabinet equipment; The reference pose determination module is used to determine multiple reference poses corresponding to the first model in the preset coordinate system based on the identification pose information of each QR code. The preferred pose determination module is used to calculate the preferred pose of the first model in the preset coordinate system based on the plurality of reference poses; the total error between the preferred pose and the plurality of reference poses is minimized. The target pose determination module is used to perform conversion processing on the preferred pose based on the installation information to obtain the target pose of the second model in the preset coordinate system.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images