Monocular furniture size detection method, device, system and medium

Abstract

The invention discloses a monocular furniture size detection method, device and system and a medium. The method comprises the following steps: acquiring a target image of to-be-detected furniture provided with a calibration triangular pyramid; extracting edge information of the target image by using a Laplacian operator to form a grayscale image containing an edge gradient; stacking the target image and the grayscale image together, and importing the stacked target image and grayscale image into a preset model, so as to output a result image with the size information of the furniture to be detected through the model; the model comprises a feature extraction network module, a feature pyramid network module, a single-stage headless network module, a non-maximum suppression module and an output module. A non-contact measurement mode is adopted, limitation of a traditional contact type measurement tool is avoided, and an automatic and efficient detection process can be achieved. The overall size of the furniture can be measured at a time, and the size accuracy of the furniture can be evaluated more comprehensively. The method is applied to the technical field of machine vision.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and particularly to a monocular furniture size detection method, device, system and medium. Background Art

[0002] In the furniture manufacturing field, traditional size detection methods mainly rely on manual spot checks using vernier calipers and tape measures. These methods are greatly affected by subjective factors, have low precision and slow speed. Therefore, the furniture size detection technology based on machine vision has emerged. It has the advantages of non-contact, high precision, high efficiency, etc., and is widely used in the industrial production field. However, in the existing furniture size detection solutions based on machine vision, it is often only possible to detect the size of a single component of the furniture, and it is impossible to achieve a one-time and comprehensive measurement of the size of the entire assembled furniture, which limits its application in overall quality control. Therefore, how to achieve a comprehensive measurement of the entire furniture is a technical problem that urgently needs to be studied in the industry. Summary of the Invention

[0003] The present invention provides a monocular furniture size detection method, device, system and medium to solve one or more technical problems existing in the prior art, and at least provide a beneficial choice or create conditions.

[0004] In a first aspect, the present invention provides a monocular furniture size detection method, including: obtaining a target image of a furniture to be measured provided with a calibration triangular pyramid; extracting edge information of the target image by using a Laplace operator to form a grayscale image containing edge gradients; stacking the target image and the grayscale image together and importing them into a preset proposed model to output a result image with the size information of the furniture to be measured through the proposed model;

[0005] Wherein, the proposed model includes: a feature extraction network module, a feature pyramid network module, a single-stage headless network module, a non-maximum suppression module and an output module;

[0006] The feature extraction network module is used to extract feature maps of different levels from the stacked target image and grayscale image;

[0007] The feature pyramid network module is used to transmit information of the different-level feature maps through top-down and bottom-up paths to generate a multi-scale feature pyramid;

[0008] The single-stage headless network module is used to process the multi-scale feature pyramid to obtain a prediction result, where the prediction result includes three prediction branches, namely: a classification branch, a regression branch, and a key point detection branch; the classification branch is used to: predict whether each anchor box contains a calibrated triangular pyramid; the regression branch is used to: predict the position of the furniture box to be measured; the key point detection branch is used to: predict the positions of the 4 corner points of the calibrated triangular pyramid;

[0009] The non-maximum suppression module is used to perform non-maximum suppression on the prediction result, remove overlapping anchor boxes, and obtain the best detection result;

[0010] The output module is used to utilize the best detection result, the calibrated key point coordinates, and the geometric features of the calibrated triangular pyramid to output a result image with the size information of the furniture to be measured.

[0011] Further, in the output module, utilizing the best detection result, the calibrated key point coordinates, and the geometric features of the calibrated triangular pyramid to output a result image with the size information of the furniture to be measured specifically includes: converting the calibrated key point coordinates and the geometric features of the calibrated triangular pyramid to obtain the coordinates of each vertex of the calibrated triangular pyramid; obtaining the length of the anchor box from the best detection result, and calculating the height of the furniture to be measured through the length of the anchor box and the coordinates of each vertex of the calibrated triangular pyramid; calculating the distance to the top of the furniture to be measured using the width of the anchor box.

[0012] Further, the feature extraction network module uses StarNet-S2 as the backbone network to extract the features of the image, where StarNet-S2 adopts a four-stage hierarchical structure, where the first-stage structure includes: 2 StarNet blocks, the second-stage structure includes: 6 StarNet blocks, the third-stage structure includes: 2 StarNet blocks, and the fourth-stage structure includes: 2 StarNet blocks;

[0013] Wherein, each StarNet block includes: a convolutional layer and a star operation block, and the convolutional layer is used for downsampling.

[0014] Further, the star operation block includes: a depth convolutional layer, a linear layer, an activation function, and a star operation layer, where the linear layer and the activation function form a fully connected layer, the depth convolutional layer is connected to the star operation layer through the fully connected layer, the depth convolutional layer is used to perform a convolutional operation independently on each channel, the linear layer is used to perform a linear transformation on the features, the star operation layer fuses the features through element-wise multiplication, and the mathematical expression of the star operation layer is:

[0015]

[0016] Among them, Output is the output result of the star operation layer, X is the input feature; W1 and W2 are the weight matrices of the linear transformation; ⊙ represents element-wise multiplication; act is the activation function.

[0017] Furthermore, the activation function act of the star operation layer is ReLU6(x), and the formula of the ReLU6(x) is: ReLU6(x) = min(max(0, x), 6);

[0018] Among them, when the input (x) is less than 0, the output is 0; when the input (x) is between 0 and 6, the output is equal to the input (x); when the input (x) is greater than 6, the output is 6.

[0019] In a second aspect, a monocular furniture size detection device is provided, including: a processor and a memory, where the memory is used to store a computer-readable program; when the computer-readable program is executed by the processor, the processor is caused to implement the monocular furniture size detection method described in any one of the above technical solutions.

[0020] In a third aspect, a monocular furniture size detection system is provided, including: an acquisition module, an extraction module, and an import module;

[0021] The acquisition module is used to: acquire a target image of a furniture to be measured provided with a calibration triangular pyramid;

[0022] The extraction module is used to: use the Laplace operator to extract the edge information of the target image to form a grayscale image containing edge gradients;

[0023] The import module is used to: stack the target image and the grayscale image together and import them into a preset proposed model, so as to output a result image with the size information of the furniture to be measured through the proposed model;

[0024] Among them, the proposed model includes: a feature extraction network module, a feature pyramid network module, a single-stage headless network module, a non-maximum suppression module, and an output module;

[0025] The feature extraction network module is used to extract feature maps of different levels from the stacked target image and grayscale image;

[0026] The feature pyramid network module is used to transmit information of the different-level feature maps through top-down and bottom-up paths to generate a multi-scale feature pyramid;

[0027] The single-stage headless network module is used to process the multi-scale feature pyramid to obtain a prediction result, where the prediction result includes three prediction branches, namely: a classification branch, a regression branch, and a key point detection branch; the classification branch is used to: predict whether each anchor box contains a calibrated triangular pyramid; the regression branch is used to: predict the position of the furniture box to be measured; the key point detection branch is used to: predict the positions of the 4 corner points of the calibrated triangular pyramid;

[0028] The non-maximum suppression module is used to perform non-maximum suppression on the prediction result, remove overlapping anchor boxes, and obtain the best detection result;

[0029] The output module is used to utilize the best detection result, the calibrated key point coordinates, and the geometric features of the calibrated triangular pyramid to output a result image with the size information of the furniture to be measured.

[0030] Further, in the output module, utilizing the best detection result, the calibrated key point coordinates, and the geometric features of the calibrated triangular pyramid to output a result image with the size information of the furniture to be measured specifically includes: converting the calibrated key point coordinates and the geometric features of the calibrated triangular pyramid to obtain the coordinates of each vertex of the calibrated triangular pyramid; obtaining the length of the anchor box from the best detection result, and calculating the height of the furniture to be measured through the length of the anchor box and the coordinates of each vertex of the calibrated triangular pyramid; calculating the distance to the top of the furniture to be measured using the width of the anchor box.

[0031] Further, the feature extraction network module uses StarNet-S2 as the backbone network to extract the features of the image, where StarNet-S2 adopts a four-stage hierarchical structure. Among them, the first-stage structure includes: 2 StarNet blocks, the second-stage structure includes: 6 StarNet blocks, the third-stage structure includes: 2 StarNet blocks, and the fourth-stage structure includes: 2 StarNet blocks; where each StarNet block includes: a convolutional layer and a star operation block, and the convolutional layer is used for downsampling.

[0032] Fourthly, a computer-readable storage medium is provided, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the monocular furniture size detection method described in any one of the above technical solutions.

[0033] The present invention has at least the following beneficial effects: The present invention adopts a non-contact measurement method, avoiding the limitations of traditional contact measurement tools, and helping to achieve an automated and efficient detection process. On the other hand, the present invention can measure the overall dimensions of furniture at one time, rather than just individual components, which makes the present invention more advantageous in overall quality control and can more comprehensively evaluate the dimensional accuracy of furniture. The present invention is used in the field of machine vision technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings are used to provide a further understanding of the technical solution of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation to the technical solution of the present invention.

[0035] Figure 1 is a flowchart of the steps of a monocular furniture dimension detection method;

[0036] Figure 2 is a schematic diagram of the device structure of a monocular furniture dimension detection device;

[0037] Figure 3 is a schematic diagram of the system structure of a monocular furniture dimension detection system;

[0038] Figure 4 is a schematic diagram of a target image;

[0039] Figure 5 is a grayscale image obtained by extracting the target image with a Laplacian operator;

[0040] Figure 6 is the hierarchical structure of four stages of a feature extraction network module;

[0041] Figure 7 is a schematic diagram of the structure of a star operation block;

[0042] Figure 8 is a result image with the dimension information of the furniture to be measured;

[0043] Figure 9 is a simple schematic diagram of the dimension calculation of the furniture to be measured. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0045] It should be noted that although the functional modules are divided in the system schematic diagram and the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be executed in a different module division from that in the system or a different order from that in the flowchart. Terms such as "first" and "second" in the description, claims, and the above-mentioned drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence.

[0046] Reference Figure 1 , Figure 1 is the step flowchart of the monocular furniture size detection method. This monocular furniture size detection method can be run through an intelligent device. The purpose of this monocular furniture size detection method is to make the measurement of furniture more comprehensive and efficient.

[0047] To achieve this goal, this application discloses a monocular furniture size detection method, including the following steps.

[0048] Step 1: Obtain a target image of the furniture to be measured with a calibrated triangular pyramid.

[0049] Among them, in this embodiment, a rectangular stool is taken as an example of the furniture to be measured. Among them, reference Figure 4 , Figure 4 is a schematic diagram of the target image.

[0050] When the furniture to be measured is determined, for the convenience of description, the furniture to be measured is denoted as the furniture to be measured. The calibrated triangular pyramid is made by customization or 3D printing, and the calibrated triangular pyramid is placed at the set position of the furniture to be measured. The intelligent device controls the image acquisition device to obtain a target image of the furniture to be measured with a calibrated triangular pyramid. Among them, the image acquisition device can be a camera. The camera takes pictures of the furniture to be measured to obtain the target image. Among them, in some further specific embodiments, the right-angled side length of the calibrated triangular pyramid is 10 cm. It should be noted that the length of the right-angled side of the calibrated triangular pyramid is not limited to 10 cm and can be adjusted according to the actual situation.

[0051] Step 2: Use the Laplace operator to extract the edge information of the target image to form a grayscale image containing edge gradients. Specifically, see Figure 5 , Figure 5 is the grayscale image after the target image is extracted by the Laplace operator.

[0052] After the intelligent device determines that the target image has been obtained, it is necessary to process the target image. Among them, the intelligent device can extract the edge information of the target image by using the Laplace operator to form a grayscale image containing edge gradients. Using the Laplace operator to process the target image, so that the obtained grayscale image contains edge gradient information. Among them, the mathematical expression of the Laplace operator is: After applying the Laplace operator, an edge feature map of the image can be obtained, enhancing the edge information of the image.

[0053] Step 3: Stack the target image and the grayscale image together and import them into a preset proposed model to output a result image with the size information of the furniture to be measured through the proposed model. As Figure 8 shown, Figure 8 is the result image with the size information of the furniture to be measured.

[0054] When the intelligent device obtains the target image and the grayscale image, it can stack the target image and the grayscale image together. Since the target image is an RGB image and the grayscale image is a grayscale image. Therefore, when the target image and the grayscale image are stacked together, a stacked image will be formed. Among them, the stacked image can be represented by [B H W 4], where B represents the batch size of the target image and the grayscale image. H represents the height of the target image. Since the size of the target image is the same as that of the grayscale image, H can also represent the height of the grayscale image. W represents the width of the target image. Since the size of the target image is the same as that of the grayscale image, W can also represent the width of the grayscale image. 4 represents three RGB channels plus one grayscale channel.

[0055] Import the stacked image into the preset proposed model in a four-channel manner. Process the stacked image through the proposed model to output a result image with the size information of the furniture to be measured.

[0056] Among them, the proposed model includes: a feature extraction network module, a feature pyramid network module, a single-stage headless network module, a non-maximum suppression module, and an output module.

[0057] The feature extraction network module is used to extract feature maps at different levels from the stacked target image and grayscale image.

[0058] Refer to Figure 6 , Figure 6It is a four-stage hierarchical structure of the feature extraction network module. In some further specific embodiments, the feature extraction network module uses StarNet-S2 as the backbone network to extract features of an image. Among them, StarNet-S2 adopts a four-stage hierarchical structure. The structure of the first stage includes: 2 StarNet blocks; the structure of the second stage includes: 6 StarNet blocks; the structure of the third stage includes: 2 StarNet blocks; the structure of the fourth stage includes: 2 StarNet blocks. Among them, each StarNet block includes: a convolutional layer and a star operation block, and the convolutional layer is used for downsampling.

[0059] Downsampling is performed using the convolutional layer, and a modified demo block is used for feature extraction. Layer Normalization is replaced with Batch Normalization and placed after the depth convolution (which can be fused during the inference process). Inspired by MobileNeXt, depth convolution is added at the end of each block. The channel expansion factor is always set to 4, and the network width doubles at each stage. The GELU activation function in the demo block is replaced with ReLU6, following the design of MobileNetv2. Different sizes of StarNet are constructed only by changing the number of blocks and the number of input embedding channels. StarNet is a lightweight neural network architecture based on star operations (element-wise multiplication), designed to achieve high performance and low latency through efficient feature fusion and computation.

[0060] Reference Figure 7 , Figure 7 It is a schematic structural diagram of the star operation block. In some further specific embodiments, the star operation block includes: a depth convolutional layer, a linear layer, an activation function, and a star operation layer. Among them, the linear layer and the activation function form a fully connected layer. The depth convolutional layer is connected to the star operation layer through the fully connected layer. The depth convolutional layer is used to perform convolutional operations independently on each channel. The linear layer is used to perform linear transformation on the features. The star operation layer fuses the features through element-wise multiplication. The mathematical expression of the star operation layer is:

[0061]

[0062] Among them, Output is the output result of the star operation layer, X is the input feature; W1 and W2 are the weight matrices of the linear transformation; ⊙ represents element-wise multiplication; act is the activation function.

[0063] Among them, the activation function act of the star operation layer is ReLU6(x), and the formula of ReLU6(x) is: ReLU6(x) = min(max(0, x), 6). Among them, when the input (x) is less than 0, the output is 0; when the input (x) is between 0 and 6, the output is equal to the input (x); when the input (x) is greater than 6, the output is 6. While maintaining the non-linearity and sparse activation properties of the ReLU function, the ReLU6 function can reduce the problem of gradient disappearance by limiting the maximum value to 6, improving the training stability of the network. It is particularly useful in lightweight networks and mobile devices because it can reduce the computational load and improve efficiency.

[0064] The characteristics of the star operation layer include: high-dimensional feature mapping, efficiency, and non-linear fusion.

[0065] High-dimensional feature mapping: The star operation can map the input features to a high-dimensional implicit feature space, similar to the role of the polynomial kernel function. This mapping method enables the model to implicitly obtain high-dimensional features from low-dimensional inputs, thereby improving the feature expression ability.

[0066] Efficiency: Different from the traditional method of increasing the network width to obtain high-dimensional features, the star operation realizes high-dimensional feature mapping without increasing the computational complexity. This enables StarNet to obtain richer and more expressive feature representations while maintaining computational efficiency.

[0067] Non-linear feature fusion: Through element-wise multiplication, the star operation can effectively fuse the features of different subspaces, enhancing the non-linear representation ability of the features. This is of great significance for processing complex image features and improving the classification performance of the model.

[0068] The Feature Pyramid Network (FPN) module is used to transmit information through top-down and bottom-up paths for the feature maps at different levels, generating a multi-scale feature pyramid. Among them, the scale changes of the multi-scale feature pyramid are: 1 / 4 of the original size, 1 / 8 of the original size, and 1 / 16 of the original size, where the original size is the size of the target image. It should be noted that the scale changes of the multi-scale feature pyramid are not limited to 1 / 4 of the original size, 1 / 8 of the original size, and 1 / 16 of the original size, and can be adjusted according to the actual situation.

[0069] The Single Stage Headless (SSH) module is used to process the multi-scale feature pyramid to obtain a prediction result. The prediction result includes three prediction branches, namely: a classification branch, a regression branch, and a key point detection branch. The classification branch is used to: predict whether each anchor box contains a calibrated triangular pyramid. The regression branch is used to: predict the position of the furniture box to be measured. The key point detection branch is used to: predict the positions of the 4 corner points of the calibrated triangular pyramid. The Single Stage Headless (SSH) module contains two parallel convolutional layers and a fusion layer for extracting features of different scales in the multi-scale feature pyramid. In this specific embodiment, the feature map output by the single stage headless network module has the same scale as the input feature map.

[0070] The Non-Maximum Suppression (NMS) module is used to perform non-maximum suppression on the prediction result to remove overlapping anchor boxes and obtain the best detection result. Among them, the non-maximum suppression module removes overlapping boxes through the Intersection over Union (IoU). The calculation formula of the Intersection over Union (IoU) is: Area ofOverlap represents the intersection area of two boxes, and Area ofUnion represents the union area of two boxes. If the IoU exceeds the set threshold, the box with the highest score is retained, and other boxes are removed.

[0071] The output module is used to utilize the best detection result, the calibrated key point coordinates, and the geometric features of the calibrated triangular pyramid to output a result image with the size information of the furniture to be measured.

[0072] Among them, in the output module, utilizing the best detection result, the calibrated key point coordinates, and the geometric features of the calibrated triangular pyramid to output a result image with the size information of the furniture to be measured specifically includes: converting the calibrated key point coordinates and the geometric features of the calibrated triangular pyramid to obtain the coordinates of each vertex of the calibrated triangular pyramid. Obtain the length of the anchor box from the best detection result, and calculate the height of the furniture to be measured through the length of the anchor box and the coordinates of each vertex of the calibrated triangular pyramid. Calculate the distance to the top of the furniture to be measured using the width of the anchor box.

[0073] Reference Figure 9 , Figure 9 is a simple schematic diagram of the size calculation of the furniture to be measured.

[0074] Among them, the coordinates of points i, j, k, and m can be obtained through the output module. Through the output module, the coordinates of points a, b, c, and d in the anchor box can also be obtained, as well as the lengths of the respective border dimensions in the anchor box, that is, the length from point a to point b, the length from point b to point c, the length from point c to point d, and the length from point d to point a.

[0075] The point-slope method is used to obtain the coordinates of points e and g. Based on the coordinates of point i and point e, the distance from point i to point e is calculated, thereby obtaining the width dimension of the furniture to be measured (dimension in the image). Based on the coordinates of point i and point g, the distance from point i to point g is calculated, thereby obtaining the length dimension of the furniture to be measured (dimension in the image).

[0076] Using the coordinates of points i, e, and g, the upper surface dimension information of the furniture to be measured can be obtained (such as the dimension information of the e-f-g-i surface).

[0077] The distance from point i to point h is calculated using the coordinates of point i and point h, thereby obtaining the height dimension of the furniture to be measured (dimension in the image).

[0078] Since the actual dimensions of the triangular pyramid are known, therefore, the distance from point i to point e can be converted through the distance from point i to point j, that is, the width dimension of the furniture to be measured (dimension in reality) is obtained. The distance from point i to point g is converted through the distance from point i to point k, that is, the length dimension of the furniture to be measured (dimension in reality) is obtained. The distance from point i to point h is converted through the distance from point i to point m, that is, the height dimension of the furniture to be measured (dimension in reality) is obtained.

[0079] Among them, an example of the point-slope method is as follows:

[0080] 1. Calculation of the slope of a straight line:

[0081]

[0082] Among them, p1 = (x1, y1) and p3 = (x3, y3) are the coordinates of two input key points. The slope of the connecting line is calculated through the difference in the coordinates of the two points, which is used for subsequent geometric derivation of the intersection point of the perpendicular line.

[0083] 2. Calculation of the intersection point and distance of the perpendicular line:

[0084] Step-by-step analysis of the formula derivation:

[0085] (1) Calculation of the slope of the perpendicular line

[0086]

[0087] The slope of the perpendicular line is the negative reciprocal of the slope of the original line.

[0088] (2) Intersection coordinate calculation:

[0089] Assume that the intersection point of the straight line extended to the boundary of the bounding box is a1 = (ax, ay), then:

[0090]

[0091] Use the point - slope form equation combined with the abscissa of the boundary coordinate $a_1$ to solve the intersection point of the extended line and the boundary.

[0092] (3) Normalized distance calculation:

[0093]

[0094] I = (xintercept, yintercept) is the intersection coordinate. By normalizing the Euclidean distance, the pixel distance is converted into the actual physical distance (such as centimeters).

[0095] The present invention adopts a non - contact measurement method, avoiding the limitations of traditional contact measurement tools, and contributing to the realization of an automated and efficient detection process. On the other hand, the present invention can measure the overall dimensions of furniture at one time, rather than just individual components, which makes the present invention more advantageous in overall quality control and can more comprehensively evaluate the dimensional accuracy of furniture.

[0096] In the second aspect, referring to Figure 2 , Figure 2 is the schematic structural diagram of a monocular furniture dimension detection device.

[0097] Provide a monocular furniture dimension detection device, including: a processor and a memory; wherein, the memory is used to store a computer - readable program. When the computer - readable program is executed by the processor, the processor implements the monocular furniture dimension detection method as described in any one of the above technical solutions.

[0098] Those of ordinary skill in the art can understand that all or some of the steps and systems disclosed in the above methods can be implemented as software, firmware, hardware, and their appropriate combinations. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. It is well known to those of ordinary skill in the art that communication media typically contains computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0099] In a third aspect, referring to Figure 3 , Figure 3 is a schematic structural diagram of a monocular furniture size detection system.

[0100] A monocular furniture size detection system is provided, including: an acquisition module, an extraction module, and an import module.

[0101] The acquisition module is used to: acquire a target image of a furniture to be measured provided with a calibration triangular pyramid.

[0102] Herein, in this embodiment, a cuboid stool is taken as an example of the furniture to be measured. Among them, referring to Figure 4 , Figure 4 is a schematic diagram of the target image.

[0103] When the furniture to be measured is determined, for the sake of convenience in description, the furniture to be measured is denoted as the furniture under test. A calibration triangular pyramid is fabricated through customization or 3D printing and placed at a set position on the furniture under test. The acquisition module controls an image acquisition device to acquire a target image of the furniture under test provided with the calibration triangular pyramid. Herein, the image acquisition device may be a camera. The camera captures the furniture under test to obtain the target image. Among them, in some further specific embodiments, the right-angled side length of the calibration triangular pyramid is 10 cm. It should be noted that the length of the right-angled side of the calibration triangular pyramid is not limited to 10 cm and can be adjusted according to the actual situation.

[0104] The extraction module is used to: extract the edge information of the target image by using the Laplacian operator to form a grayscale image containing edge gradients. Specifically, refer to Figure 5 , Figure 5 which is the grayscale image after the target image is extracted by the Laplacian operator.

[0105] After the extraction module determines the target image, the target image needs to be processed. Among them, the extraction module can extract the edge information of the target image by using the Laplacian operator to form a grayscale image containing edge gradients. The Laplacian operator is used to process the target image so that the obtained grayscale image contains edge gradient information. The mathematical expression of the Laplacian operator is: After applying the Laplacian operator, an edge feature map of the image can be obtained, enhancing the edge information of the image.

[0106] The import module is used to: stack the target image and the grayscale image together and import them into a preset proposed model to output a result image with the size information of the furniture under test through the proposed model. As Figure 8 shown, Figure 8 which is the result image with the size information of the furniture under test.

[0107] After the import module obtains the target image and the grayscale image, it can stack the target image and the grayscale image together. Since the target image is an RGB image and the grayscale image is a grayscale map. Therefore, when the target image and the grayscale image are stacked together, a stacked map will be formed. Among them, the stacked map can be represented by [B H W 4], where B represents the batch size of the target image and the grayscale image. H represents the height of the target image. Since the size of the target image is the same as that of the grayscale image, H can also represent the height of the grayscale image. W represents the width of the target image. Since the size of the target image is the same as that of the grayscale image, W can also represent the width of the grayscale image. 4 represents three RGB channels plus one grayscale channel.

[0108] Import the stacked image into the preset proposed model in a 4-channel manner. Process the stacked image through the proposed model to output a result image with the dimensional information of the furniture to be measured.

[0109] Among them, the proposed model includes: a feature extraction network module, a feature pyramid network module, a single-stage headless network module, a non-maximum suppression module, and an output module.

[0110] The feature extraction network module is used to extract feature maps at different levels from the stacked target image and grayscale image.

[0111] Reference Figure 6 , Figure 6 is the four-stage hierarchical structure of the feature extraction network module. In some further specific embodiments, the feature extraction network module uses StarNet-S2 as the backbone network to extract image features. Among them, StarNet-S2 adopts a four-stage hierarchical structure. Among them, the first-stage structure includes: 2 StarNet blocks, the second-stage structure includes: 6 StarNet blocks, the third-stage structure includes: 2 StarNet blocks, and the fourth-stage structure includes: 2 StarNet blocks. Among them, each StarNet block includes: a convolutional layer and a star operation block, and the convolutional layer is used for downsampling.

[0112] Use the convolutional layer for downsampling and use the modified demo block for feature extraction. Replace layer normalization with batch normalization and place it after the depth convolution (it can be fused during the inference process). Inspired by MobileNeXt, depth convolution is added at the end of each block. The channel expansion factor is always set to 4, and the network width of each stage is doubled. The GELU activation function in the demo block is replaced with ReLU6, following the design of MobileNetv2. Different sizes of StarNet are constructed only by changing the number of blocks and the number of input embedding channels. StarNet is a lightweight neural network architecture based on star operations (element-wise multiplication), and its design aims to achieve high performance and low latency through efficient feature fusion and calculation.

[0113] Reference Figure 7 , Figure 7It is a structural schematic diagram of the star operation block. In some further specific embodiments, the star operation block includes: a depth convolution layer, a linear layer, an activation function, and a star operation layer. Among them, the linear layer and the activation function form a fully connected layer, the depth convolution layer is connected to the star operation layer through the fully connected layer, the depth convolution layer is used to perform convolution operations independently on each channel, the linear layer is used to perform linear transformation on features, and the star operation layer fuses features through element-wise multiplication. The mathematical expression of the star operation layer is:

[0114]

[0115] where Output is the output result of the star operation layer, X is the input feature; W1 and W2 are the weight matrices of the linear transformation; ⊙ represents element-wise multiplication; act is the activation function.

[0116] Among them, the activation function act of the star operation layer is ReLU6(x), and the formula of ReLU6(x) is: ReLU6(x) = min(max(0, x), 6). Among them, when the input (x) is less than 0, the output is 0; when the input (x) is between 0 and 6, the output is equal to the input (x); when the input (x) is greater than 6, the output is 6. While maintaining the non-linearity and sparse activation properties of the ReLU function, the ReLU6 function can reduce the problem of gradient disappearance and improve the training stability of the network by limiting the maximum value to 6. It is particularly useful in lightweight networks and mobile devices because it can reduce the amount of computation and improve efficiency.

[0117] The characteristics of the star operation layer include: high-dimensional feature mapping, efficiency, and non-linear fusion.

[0118] High-dimensional feature mapping: The star operation can map the input feature to a high-dimensional implicit feature space, similar to the role of the polynomial kernel function. This mapping method enables the model to implicitly obtain high-dimensional features in low-dimensional inputs, thereby improving the expressive ability of features.

[0119] Efficiency: Different from the traditional method of increasing the network width to obtain high-dimensional features, the star operation realizes the mapping of high-dimensional features without increasing the computational complexity. This enables StarNet to obtain richer and more expressive feature representations while maintaining computational efficiency.

[0120] Non-linear feature fusion: Through element-wise multiplication, the star operation can effectively fuse features in different subspaces and enhance the non-linear representation ability of features. This is of great significance for processing complex image features and improving the classification performance of the model.

[0121] The Feature Pyramid Network (FPN) module is used to transmit information through top-down and bottom-up paths for feature maps at different levels to generate a multi-scale feature pyramid. Among them, the scale changes of the multi-scale feature pyramid are respectively: 1 / 4 of the original size, 1 / 8 of the original size, and 1 / 16 of the original size, where the original size is the size of the target image. It should be noted that the scale changes of the multi-scale feature pyramid are not limited to 1 / 4 of the original size, 1 / 8 of the original size, and 1 / 16 of the original size, and can be adjusted according to actual situations.

[0122] The Single Stage Headless (SSH) module is used to process the multi-scale feature pyramid to obtain a prediction result, where the prediction result includes three prediction branches, namely: a classification branch, a regression branch, and a key point detection branch; the classification branch is used to: predict whether each anchor box contains a calibrated triangular pyramid; the regression branch is used to: predict the position of the furniture box to be measured; the key point detection branch is used to: predict the positions of the 4 corner points of the calibrated triangular pyramid. Among them, the Single Stage Headless (SSH) module includes two parallel convolutional layers and a fusion layer for extracting features at different scales in the multi-scale feature pyramid. In this specific embodiment, the scale of the feature map output by the single stage headless network module is the same as that of the input feature map.

[0123] The Non-Maximum Suppression (NMS) module is used to perform non-maximum suppression on the prediction result to remove overlapping anchor boxes and obtain the best detection result. Among them, the non-maximum suppression module removes overlapping boxes through the Intersection over Union (IoU). The calculation formula of the Intersection over Union (IoU) is: Area ofOverlap represents the intersection area of two boxes, and Area ofUnion represents the union area of two boxes. If the IoU exceeds the set threshold, the box with the highest score is retained, and other boxes are removed.

[0124] The output module is used to utilize the best detection result, the calibrated key point coordinates, and the geometric features of the calibrated triangular pyramid to output a result image with the size information of the furniture to be measured.

[0125] Among them, in the output module, using the optimal detection result, the calibrated key point coordinates, and the geometric features of the calibrated triangular pyramid to output a result image with the dimension information of the furniture to be measured specifically includes: converting the calibrated key point coordinates and the geometric features of the calibrated triangular pyramid to obtain the coordinates of each vertex of the calibrated triangular pyramid. Obtaining the length of the anchor box from the optimal detection result, and calculating the height of the furniture to be measured through the length of the anchor box and the coordinates of each vertex of the calibrated triangular pyramid. Calculating the distance to the top of the furniture to be measured using the width of the anchor box.

[0126] In a fourth aspect, a computer-readable storage medium is provided, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the monocular furniture dimension detection method described in any one of the above specific embodiments.

[0127] The embodiments of the present application also disclose a computer program product, including a computer program or computer instructions. The computer program or computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the computer device executes the monocular furniture dimension detection method described in any of the previous embodiments.

[0128] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products or devices.

[0129] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item) of the following" or its similar expressions refer to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0130] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be indirect couplings or communication connections through some interfaces, devices, or units, and can be in electrical, mechanical, or other forms.

[0131] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] In addition, the functional units in each embodiment of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0133] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0134] Although the description of this application has been quite detailed and several of the described embodiments have been particularly described, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be regarded as effectively covering the intended scope of this application by considering the prior art to provide a broad interpretation of these claims. In addition, the foregoing description of this application with embodiments foreseeable by the inventors is for the purpose of providing a useful description, and non-substantive changes to this application that are not currently foreseeable may still represent equivalent changes to this application.

Claims

1. A monocular furniture size detection method, characterized in that, Including: Obtain a target image of the furniture to be measured provided with a calibration triangular pyramid; Use the Laplace operator to extract the edge information of the target image to form a grayscale image containing edge gradients; Stack the target image and the grayscale image together and import them into a preset proposed model to output a result image with the size information of the furniture to be measured through the proposed model; Wherein, the proposed model includes: a feature extraction network module, a feature pyramid network module, a single-stage headless network module, a non-maximum suppression module, and an output module; The feature extraction network module is used to extract feature maps of different levels from the stacked target image and grayscale image; The feature pyramid network module is used to transmit information through top-down and bottom-up paths for the feature maps of different levels to generate a multi-scale feature pyramid; The single-stage headless network module is used to process the multi-scale feature pyramid to obtain a prediction result, wherein the prediction result includes three prediction branches, and the three prediction branches are respectively: a classification branch, a regression branch, and a key point detection branch; the classification branch is used to: predict whether each anchor box contains a calibration triangular pyramid; the regression branch is used to: predict the position of the box containing the furniture to be measured; the key point detection branch is used to: predict the positions of the 4 corner points of the calibration triangular pyramid; The non-maximum suppression module is used to perform non-maximum suppression on the prediction result to remove overlapping anchor boxes and obtain the best detection result; The output module is used to utilize the best detection result, the calibrated key point coordinates, and the geometric features of the calibration triangular pyramid to output a result image with the size information of the furniture to be measured.

2. The monocular furniture size detection method according to claim 1, wherein In the output module, using the best detection result, the calibrated key point coordinates, and the geometric features of the calibration triangular pyramid to output a result image with the size information of the furniture to be measured specifically includes: converting the calibrated key point coordinates and the geometric features of the calibration triangular pyramid to obtain the coordinates of each vertex of the calibration triangular pyramid; obtaining the length of the anchor box from the best detection result, and calculating the height of the furniture to be measured through the length of the anchor box and the coordinates of each vertex of the calibration triangular pyramid; calculating the distance to the top end of the furniture to be measured using the width of the anchor box.

3. The monocular furniture dimension detection method according to claim 1, wherein The feature extraction network module uses StarNet-S2 as the backbone network to extract the features of the image, wherein StarNet-S2 adopts a four-stage hierarchical structure, and wherein the first-stage structure includes: 2 StarNet blocks, the second-stage structure includes: 6 StarNet blocks, the third-stage structure includes: 2 StarNet blocks, and the fourth-stage structure includes: 2 StarNet blocks; Wherein, each StarNet block includes: a convolutional layer and a star operation block, and the convolutional layer is used for downsampling.

4. The monocular furniture size detection method according to claim 3, characterized in that The star operation block includes: a depth convolution layer, a linear layer, an activation function, and a star operation layer. Among them, the linear layer and the activation function form a fully connected layer. The depth convolution layer is connected to the star operation layer through the fully connected layer. The depth convolution layer is used to perform convolution operations independently on each channel. The linear layer is used to perform linear transformation on features. The star operation layer fuses features through element-wise multiplication. The mathematical expression of the star operation layer is: where Output is the output result of the star operation layer, X is the input feature; W1 and W2 are the weight matrices of the linear transformation; ⊙ represents element-wise multiplication; act is the activation function.

5. The monocular furniture size detection method according to claim 4, wherein The activation function act of the star operation layer is ReLU6(x), and the formula of ReLU6(x) is: ReLU6(x) = min(max(0, x), 6); where, when the input (x) is less than 0, the output is 0; when the input (x) is between 0 and 6, the output is equal to the input (x); when the input (x) is greater than 6, the output is 6.

6. A monocular furniture size detection device, characterized in that, Including: A processor; A memory for storing computer-readable programs; When the computer-readable program is executed by the processor, the processor implements the monocular furniture size detection method according to any one of claims 1 to 5.

7. A monocular furniture size detection system, characterized in that, Including: An acquisition module, an extraction module, and an import module; The acquisition module is used to: acquire a target image of the furniture to be measured provided with a calibration triangular pyramid; The extraction module is used to: extract the edge information of the target image by using a Laplace operator to form a grayscale image containing edge gradients; The import module is used to: stack the target image and the grayscale image together and import them into a preset proposed model, so as to output a result image with the size information of the furniture to be measured through the proposed model; where, the proposed model includes: a feature extraction network module, a feature pyramid network module, a single-stage headless network module, a non-maximum suppression module, and an output module; The feature extraction network module is used to extract feature maps of different levels from the stacked target image and grayscale image; The feature pyramid network module is used to transfer information of the different-level feature maps through top-down and bottom-up paths to generate a multi-scale feature pyramid; The single-stage headless network module is used to process the multi-scale feature pyramid to obtain a prediction result. Among them, the prediction result includes three prediction branches, and the three prediction branches are respectively: a classification branch, a regression branch, and a key point detection branch; the classification branch is used to: predict whether each anchor box contains a calibration triangular pyramid; the regression branch is used to: predict the position of the box containing the furniture to be measured; the key point detection branch is used to: predict the positions of the 4 corner points of the calibration triangular pyramid; The non-maximum suppression module is used to perform non-maximum suppression on the prediction result to remove overlapping anchor boxes and obtain the best detection result; The output module is used to utilize the best detection result, the calibrated key point coordinates, and the geometric features of the calibration triangular pyramid to output a result image with the size information of the furniture to be measured.

8. A monocular furniture size detection system according to claim 7, characterized in that, In the output module, using the optimal detection result, the calibrated key point coordinates, and the geometric features of the calibrated triangular pyramid to output a result image with the dimension information of the furniture to be measured specifically includes: converting the calibrated key point coordinates and the geometric features of the calibrated triangular pyramid to obtain the coordinates of each vertex of the calibrated triangular pyramid; obtaining the length of the anchor box from the optimal detection result, and calculating the height of the furniture to be measured through the length of the anchor box and the coordinates of each vertex of the calibrated triangular pyramid; calculating the distance to the top of the furniture to be measured using the width of the anchor box.

9. A monocular furniture size detection system according to claim 7, characterized in that, The feature extraction network module uses StarNet-S2 as the backbone network to extract the features of the image. Among them, StarNet-S2 adopts a four-stage hierarchical structure. Among them, the first-stage structure includes: 2 StarNet blocks, the second-stage structure includes: 6 StarNet blocks, the third-stage structure includes: 2 StarNet blocks, and the fourth-stage structure includes: 2 StarNet blocks; Among them, each StarNet block includes: a convolutional layer and a star operation block, and the convolutional layer is used for downsampling.

10. A computer-readable storage medium, characterized in that, It stores a program executable by a processor, and when the program executable by the processor is executed by the processor, it is used to implement the monocular furniture dimension detection method according to any one of claims 1 to 5.

Images