An intelligent mirror, a height measurement method, device, processor and medium

Abstract

The application discloses an intelligent mirror, a height measurement method and device, a processor and a medium. After the processor receives a first image collected and sent by a first camera and a second image collected and sent by a second camera, a first mapping ratio between the images is determined according to a first vertical distance between a first key point and a second key point of a target object in the first image and a second vertical distance between a third key point and a fourth key point corresponding to the first key point and the second key point in the second image. Then, a second mapping ratio between an image distance and a physical distance is determined according to the first mapping ratio and a vertical physical distance between the first camera and the second camera. Finally, the physical height of the target object is measured according to the second mapping ratio. The application does not need to correct the field of view angle of the first camera and the second camera, avoids the complicated process of camera field of view angle calibration in the related art, and directly leads to the problem of inaccurate physical height measurement once the camera field of view angle calibration is inaccurate.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and in particular to a smart mirror, a height measurement method, a device, a processor, and a medium. Background Technology

[0002] To obtain human height information, smart mirrors often use LiDAR, depth cameras, and structured light stereo cameras. The problem with existing technologies is that LiDAR, depth cameras, and structured light stereo cameras are expensive, especially for mass-produced smart devices, which significantly increases production costs.

[0003] To reduce costs, related technologies employ RGB cameras for height measurement, such as the height measurement method and display device for a target object disclosed in application publication number CN114782351A. Specifically, the disclosed display device includes: a display panel, a processor, and a first camera and a second camera sequentially arranged along the height direction of the display panel. The pixel density of the first camera's field of view differs from that of the second camera. The processor is used to: determine the equivalent distance in the second image based on the pixel density of the first camera's field of view, the pixel density of the second camera's field of view, and a first distance between the target feature point of the target object and a first reference line in the first image; and determine the ratio of the physical distance to the pixel distance in the second image based on the sum of the equivalent distance and a second distance between the target feature point and the second reference line in the second image, as well as the target distance between the first and second cameras. Based on the pixel height and ratio of the target object in the second image, the physical height of the target object is obtained.

[0004] The height measurement solution provided by CN114782351A necessitates determining the mapping ratio based on the pixel density of the first camera's field of view and the pixel density of the second camera's field of view before subsequent physical height measurement can be performed. This requires professional personnel to calibrate the field of view of the RGB cameras. The problem with this technology is that the calibration process for RGB camera field of view is cumbersome and time-consuming. Having professionals calibrate the field of view of RGB cameras after the display device leaves the factory results in a significant waste of human resources, and the accuracy of the calibrated field of view cannot be guaranteed. Inaccurate field of view calibration directly leads to inaccurate physical height measurement. Summary of the Invention

[0005] This application provides a smart mirror, a height measurement method, a device, a processor, and a medium to solve the problem in related technologies where the process of calibrating the field of view of an RGB camera is cumbersome and the accuracy of the calibrated field of view cannot be guaranteed. Inaccurate field of view calibration directly leads to inaccurate physical height measurement.

[0006] In a first aspect, this application provides a smart mirror, the smart mirror comprising: a display panel, a processor, and a first camera and a second camera arranged sequentially along the height direction of the display panel;

[0007] The first camera is used to capture a first image and send it to the processor;

[0008] The second camera is used to capture a second image and send it to the processor, wherein the second image contains a complete target object;

[0009] The processor is used to execute:

[0010] Obtain a first key point and a second key point in the first image that have different vertical heights of the target object, and determine a third key point in the second image that corresponds to the first key point and a fourth key point that corresponds to the second key point;

[0011] Based on the first longitudinal distance between the first key point and the second key point, and the second longitudinal distance between the third key point and the fourth key point, a first mapping ratio between the images is determined; based on the first mapping ratio between the images and the longitudinal physical distance between the first camera and the second camera, a second mapping ratio between the image distance and the physical distance is determined.

[0012] The physical height of the target object is determined based on the third vertical distance of the target object in the second image and the second mapping ratio.

[0013] Furthermore, the processor is configured to perform:

[0014] Obtain the fourth longitudinal distance between the fifth key point and the first center reference point of the target object in the first image, and map the fourth longitudinal distance onto the second image according to the first mapping ratio to obtain the fifth longitudinal distance;

[0015] Determine the sixth key point in the second image that corresponds to the fifth key point, obtain the sixth longitudinal distance between the sixth key point and the second center reference point in the second image, and determine the second mapping ratio between image distance and physical distance based on the fifth longitudinal distance, the sixth longitudinal distance, and the longitudinal physical distance between the first camera and the second camera.

[0016] Furthermore, the processor is configured to perform:

[0017] If the sixth key point is located above the second center reference point, determine the sum of the fifth longitudinal distance and the sixth longitudinal distance, and determine the second mapping ratio of image distance to physical distance based on the sum and the longitudinal physical distance between the first camera and the second camera;

[0018] If the sixth key point is located below the second center reference point, the difference between the fifth longitudinal distance and the sixth longitudinal distance is determined. Based on the difference and the longitudinal physical distance between the first camera and the second camera, a second mapping ratio between the image distance and the physical distance is determined.

[0019] Furthermore, the processor is configured to perform:

[0020] Using a human skeleton point detection algorithm, a third key point corresponding to the first key point, a fourth key point corresponding to the second key point, and a sixth key point corresponding to the fifth key point are determined in the second image.

[0021] Furthermore, the processor is configured to perform:

[0022] If the first image contains multiple objects, the first human detection box for each object is determined by a human detection algorithm; the object corresponding to the first human detection box with the largest area is taken as the target object.

[0023] Furthermore, the processor is configured to perform:

[0024] If the second image contains multiple objects, the second human detection box for each object is determined by a human detection algorithm.

[0025] For each second human detection box, determine the Euclidean distance between the first center pixel of the second human detection box and the second center pixel of the second image; and take the object corresponding to the second human detection box with the smallest Euclidean distance as the target object.

[0026] Furthermore, the processor is configured to perform:

[0027] The physical height of the target object is corrected based on the pre-determined fitting relationship between the measured physical height value and the error value.

[0028] Secondly, this application provides a height measurement method, the method comprising:

[0029] Receive a first image captured and transmitted by a first camera; receive a second image captured and transmitted by a second camera, wherein the second image contains a complete target object; the first camera and the second camera are arranged sequentially along the height direction of the display panel;

[0030] Obtain a first key point and a second key point in the first image that have different vertical heights of the target object, and determine a third key point in the second image that corresponds to the first key point and a fourth key point that corresponds to the second key point;

[0031] Based on the first longitudinal distance between the first key point and the second key point, and the second longitudinal distance between the third key point and the fourth key point, a first mapping ratio between the images is determined; based on the first mapping ratio between the images and the longitudinal physical distance between the first camera and the second camera, a second mapping ratio between the image distance and the physical distance is determined.

[0032] The physical height of the target object is determined based on the third vertical distance of the target object in the second image and the second mapping ratio.

[0033] Thirdly, this application provides a height measuring device, the device comprising:

[0034] The receiving module is used to receive a first image captured and transmitted by a first camera; and to receive a second image captured and transmitted by a second camera, wherein the second image contains a complete target object; the first camera and the second camera are arranged sequentially along the height direction of the display panel;

[0035] The first determining module is used to acquire a first key point and a second key point with different vertical heights of the target object in the first image, and to determine a third key point in the second image corresponding to the first key point and a fourth key point corresponding to the second key point;

[0036] The second determining module is used to determine a first mapping ratio between images based on the first longitudinal distance between the first key point and the second key point and the second longitudinal distance between the third key point and the fourth key point; and to determine a second mapping ratio between image distance and physical distance based on the first mapping ratio between images and the longitudinal physical distance between the first camera and the second camera.

[0037] The third determining module is used to determine the physical height of the target object based on the third longitudinal distance of the target object in the second image and the second mapping ratio.

[0038] Fourthly, this application provides a processor, including a processing unit, a communication interface, a storage unit, and a communication bus, wherein the processing unit, the communication interface, and the storage unit communicate with each other through the communication bus;

[0039] Storage units are used to store computer programs;

[0040] The processing unit is used to implement the above method steps when executing the program stored in the storage unit.

[0041] Fifthly, this application provides a computer-readable storage medium storing a computer program executable by a processor, which, when run on the processor, causes the processor to perform the above-described method steps.

[0042] This application provides a smart mirror, a height measurement method, a device, a processor, and a medium. The smart mirror includes: a display panel, a processor, and a first camera and a second camera sequentially arranged along the height direction of the display panel; the first camera is used to capture a first image and send it to the processor; the second camera is used to capture a second image and send it to the processor, wherein the second image contains a complete target object; the processor is used to perform: acquiring a first key point and a second key point in the first image with different vertical heights of the target object; determining a third key point in the second image corresponding to the first key point and a fourth key point corresponding to the second key point; determining a first mapping ratio between images based on a first vertical distance between the first key point and the second key point and a second vertical distance between the third key point and the fourth key point; determining a second mapping ratio between image distance and physical distance based on the first mapping ratio between images and the vertical physical distance between the first camera and the second camera; and determining the physical height of the target object based on a third vertical distance of the target object in the second image and the second mapping ratio.

[0043] The above technical solution has the following advantages or beneficial effects:

[0044] In this application, after the processor receives the first image captured and transmitted by the first camera and the second image captured and transmitted by the second camera, it determines a first mapping ratio between the images based on the first longitudinal distance between the first and second key points of the target object in the first image, and the second longitudinal distance between the corresponding third and fourth key points in the second image. Then, based on the first mapping ratio and the longitudinal physical distance between the first and second cameras, it determines a second mapping ratio between the image distance and the physical distance. Finally, based on the second mapping ratio, it measures the physical height of the target object. This eliminates the need for field-of-view calibration of the first and second cameras, avoiding the cumbersome field-of-view calibration process in related technologies, where the accuracy of the calibrated field of view cannot be guaranteed. Inaccurate field-of-view calibration directly leads to inaccurate physical height measurement. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the smart mirror structure provided in this application;

[0047] Figure 2 This application provides a schematic diagram of the smart mirror displaying the target object.

[0048] Figure 3 A schematic diagram illustrating the content displayed on the smart mirror provided in this application;

[0049] Figure 4 A schematic diagram for determining the first key point and the second key point in the first image provided in this application;

[0050] Figure 5 The second image provided in this application is a schematic diagram;

[0051] Figure 6 A schematic diagram illustrating the process of determining a target object in a first image as provided in this application;

[0052] Figure 7 A schematic diagram illustrating the process of determining a target object in a second image as provided in this application;

[0053] Figure 8 A schematic diagram illustrating the linear fitting relationship between the error and the measured values ​​provided in this application;

[0054] Figure 9 The height measurement flowchart provided for this application;

[0055] Figure 10 A schematic diagram of the height measurement process provided for this application;

[0056] Figure 11 A schematic diagram of the height measuring device provided in this application;

[0057] Figure 12 A schematic diagram of the processor structure provided in this application. Detailed Implementation

[0058] To improve the accuracy of height measurement, embodiments of this application provide a smart mirror, a height measurement method, a device, a processor, and a medium.

[0059] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0060] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0061] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0062] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0063] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0065] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

[0066] Figure 1The schematic diagram of the smart mirror structure provided in this application includes: a display panel 11, a processor 12, and a first camera 13 and a second camera 14 arranged sequentially along the height direction of the display panel;

[0067] The first camera 13 is used to capture a first image and send it to the processor 12;

[0068] The second camera 14 is used to capture a second image and send it to the processor 12, wherein the second image contains a complete target object;

[0069] The processor 12 is used to perform:

[0070] Obtain a first key point and a second key point with different vertical heights of the target object in the first image, and determine a third key point in the second image corresponding to the first key point and a fourth key point corresponding to the second key point;

[0071] Based on the first longitudinal distance between the first key point and the second key point, and the second longitudinal distance between the third key point and the fourth key point, a first mapping ratio between the images is determined; based on the first mapping ratio between the images and the longitudinal physical distance between the first camera and the second camera, a second mapping ratio between the image distance and the physical distance is determined.

[0072] The physical height of the target object is determined based on the third vertical distance of the target object in the second image and the second mapping ratio.

[0073] In this application, the processor 12 is connected to the first camera 13 and the second camera 14, which are arranged sequentially along the height of the display panel. The upper camera is referred to as the first camera, and the lower camera as the second camera. The first camera 13 and the second camera 14 can be located on one side of the smart mirror 11, or on both sides of the smart mirror 11, or in other positions, as long as the vertical physical distance between them is greater than 0. Preferably, such as... Figure 1 As shown, the first camera 13 and the second camera 14 are positioned on one side of the smart mirror 11. Figure 1 The diagram shows the first camera 13 and the second camera 14 positioned on the left side of the smart mirror 11. Similarly, the first camera 13 and the second camera 14 can also be positioned on the right side of the smart mirror 11. It should be noted that, to reduce costs, both the first camera 13 and the second camera 14 in this application can be visible light imaging sensor RGB cameras.

[0074] The processor 12 can be a microprocessor (MCU) used to measure height based on images captured by the first camera 13 and the second camera 14. The processor 12 can be located on the back of the smart mirror 11.

[0075] In this application, the smart mirror 11 is connected to the display panel 11, and the processor 12 can control the display content of the display panel 11. When the processor 12 controls the display panel 11 to display a mirror image, such as... Figure 2 As shown, only a mirror image of the target object is displayed. When the processor 12 controls the display panel 11 to display specified content, such as... Figure 3 As shown, a corresponding display area is allocated on the display panel 11 for specified content. The specified content includes weather, news, first and second images captured by the camera, and the determined physical height of the target object. Additionally, the processor 12 can also control the display panel 11 to display video footage within the specified area.

[0076] In this application, the image captured by the first camera 13 is referred to as the first image, and the image captured by the second camera 14 is referred to as the second image. The first image generally contains a local area of ​​the target object, such as the upper body of the target object, while the second image contains the complete target object. The target object refers to the object currently being measured in terms of physical height, which can be a person or an object. The first camera 13 and the second camera 14 can simultaneously capture the first image and the second image. The first camera 13 sends the captured first image to the processor 12, and the second camera 14 sends the captured second image to the processor 12.

[0077] After receiving the first image and the second image, the processor 12 first acquires the first key points and the second key points in the first image that differ in vertical height of the target object. Taking the target object as a person as an example, the methods for acquiring the first key points and the second key points include: Figure 4 As shown, a human face segmentation algorithm is used to obtain the contour information of the target object in the first image. Based on the contour information, the pixel at the top of the head is determined as the first keypoint, and a pixel at the bottom edge of the first image is selected as the second keypoint. Alternatively, a human face segmentation algorithm can be used to obtain the contour information of the target object in the first image, and then two pixels with different vertical heights are selected within the contour information as the first and second keypoints. Another approach is to use a human keypoint detection algorithm to detect the keypoints of the target object, and then select two keypoints with different vertical heights from the detected keypoints as the first and second keypoints.

[0078] After the processor 12 acquires the first and second key points with different vertical heights of the target object in the first image, it can determine the third key point corresponding to the first key point in the second image using a human skeleton point detection algorithm, and then determine the fourth key point corresponding to the second key point in the second image. If the first key point is a pixel at the top of the head, when determining the third key point in the second image, a human face segmentation algorithm can also be used to acquire the contour information of the target object in the second image, and then the pixel at the top of the head can be determined as the third key point based on the contour information of the target object in the second image. Finally, the human skeleton point detection algorithm can be used to determine the fourth key point corresponding to the second key point in the second image. Figure 4 M1 is the first key point, and M2 is the second key point. Figure 5 This is a schematic diagram of the second image provided in this application. Figure 5 N1 is the third key point and N2 is the fourth key point.

[0079] Determine the first longitudinal distance between the first keypoint and the second keypoint in the first image, such as... Figure 4 In the image, b1 represents the first vertical distance, and the second vertical distance represents the distance between the third and fourth keypoints in the second image, as shown below. Figure 5 In this context, b2 represents the second vertical distance. Based on the first and second vertical distances, a first mapping ratio between the images is determined. This first mapping ratio can be either the ratio of the first and second vertical distances or the ratio of the second and first vertical distances.

[0080] After determining the first mapping ratio, a second mapping ratio between the image distance and the physical distance is determined based on the first mapping ratio and the longitudinal physical distance between the first camera and the second camera.

[0081] Specifically, the processor 12 is used to perform:

[0082] Obtain the fourth longitudinal distance between the fifth key point and the first center reference point of the target object in the first image, and map the fourth longitudinal distance onto the second image according to the first mapping ratio to obtain the fifth longitudinal distance;

[0083] Determine the sixth key point in the second image that corresponds to the fifth key point, obtain the sixth longitudinal distance between the sixth key point and the second center reference point in the second image, and determine the second mapping ratio between image distance and physical distance based on the fifth longitudinal distance, the sixth longitudinal distance, and the longitudinal physical distance between the first camera and the second camera.

[0084] In this application, any key point of the target object in the first image that is not on the same horizontal line as the first central reference point in the first image can be used as the fifth key point. A human skeleton point detection algorithm is then used to determine the sixth key point in the second image corresponding to the fifth key point. To reduce algorithm power consumption, the second key point M2 can be directly used as the fifth key point. Figure 4 As shown, the fourth longitudinal distance p1 between the fifth keypoint M2 and the first center reference point M3 is determined. The fourth longitudinal distance p1 is then mapped onto the second image according to the first mapping ratio to obtain the fifth longitudinal distance p3. For example, if the first mapping ratio is a1 = b1 / b2, the fifth longitudinal distance p3 = p1 / a1.

[0085] Determine the sixth keypoint in the second image that corresponds to the fifth keypoint. If the second keypoint M2 is taken as the fifth keypoint, then the determined sixth keypoint is: Figure 5 In the second image, determine the sixth key point N2 and the sixth longitudinal distance p2 between the second central reference point N3.

[0086] Based on the fifth longitudinal distance p3, the sixth longitudinal distance p2, and the longitudinal physical distance h between the first camera and the second camera, a second mapping ratio between the image distance and the physical distance is determined.

[0087] Processor 12 is used to execute:

[0088] If the sixth key point is located above the second central reference point, determine the sum of the fifth longitudinal distance and the sixth longitudinal distance. Based on the sum and the longitudinal physical distance between the first camera and the second camera, determine the second mapping ratio between the image distance and the physical distance.

[0089] If the sixth key point is located below the second central reference point, the difference between the fifth longitudinal distance and the sixth longitudinal distance is determined. Based on the difference and the longitudinal physical distance between the first camera and the second camera, the second mapping ratio between the image distance and the physical distance is determined.

[0090] If the sixth key point is located above the second central reference point, determine the sum p4 of the fifth longitudinal distance p3 and the sixth longitudinal distance p2, and determine the longitudinal physical distance h between the first camera and the second camera. After the first camera and the second camera are installed, their longitudinal physical distance is known. Based on the ratio of the sum p4 to the longitudinal physical distance h between the first camera and the second camera, determine the second mapping ratio a2 between the image distance and the physical distance, i.e., a2 = p4 / h.

[0091] If the sixth key point is located below the second central reference point, determine the difference p4 between the fifth longitudinal distance p3 and the sixth longitudinal distance p2. Based on the ratio of the difference p4 to the longitudinal physical distance h between the first camera and the second camera, determine the second mapping ratio a2 between the image distance and the physical distance.

[0092] The physical height of the target object is determined based on the third longitudinal distance and the second mapping ratio of the target object in the second image.

[0093] Specifically, a human detection algorithm can be used to determine the detection box region of the target object in the second image, and the vertical distance of the detection box region can be used as the third vertical distance p5. Preferably, a human face segmentation algorithm can be used to obtain the contour information of the target object in the second image, and the vertical distance between the highest and lowest pixels in the contour information of the target object can be used as the third vertical distance p5. The ratio of the third vertical distance p5 to the second mapping ratio a2 is determined as the physical height of the target object.

[0094] In this application, to avoid interference from other objects, the processor 12 is configured to perform:

[0095] If the first image contains multiple objects, the first human detection box for each object is determined by a human detection algorithm; the object corresponding to the first human detection box with the largest area is taken as the target object.

[0096] If the first image contains multiple objects, the first human detection box for each object can be determined by the human detection algorithm. At the same time, the area of ​​each first human detection box can be determined, and then the object corresponding to the first human detection box with the largest area is taken as the target object.

[0097] In this application, the vertical distance between the ordinate of the highest point of the target object's head and the ordinate of the lowest point of the image captured by the first camera is calculated.

[0098] A human detection algorithm is used to detect human bodies in all portraits in the first image, and the output list is [[fx1,fy1,fx2,fy2],[],[],....], where fx1,fy1,fx2,fy2 are the coordinates of the upper left and lower right corners of the detected human body detection boxes.

[0099] When the first image contains multiple human bodies, the object within the largest human body detection box in the first image is selected as the target object. Specifically: when only one human body detection box is detected in the first image, human body segmentation is performed directly on that body, the coordinates of the highest point of the head are obtained, and the vertical distance between the vertical coordinates of the highest point of the head and the vertical coordinates of the lowest point in the image is calculated; when multiple human body detection boxes are detected in the first image, the human body detection box with the largest area ((fx2-fx1)*(fy2-y1)) is selected as the target object's human body detection box, and the vertical distance between the vertical coordinates of the highest point of the head and the vertical coordinates of the lowest point in the image is calculated.

[0100] Figure 6 The process of determining a target object in a first image provided in this application includes the following steps:

[0101] S101: Determine the human detection bounding boxes for all portraits in the first image using a human detection algorithm.

[0102] S102: Determine whether the number of human detection boxes is at least two. If not, proceed to S103; if yes, proceed to S104.

[0103] S103: Use the object in the human body detection box as the target object.

[0104] S104: The object in the human detection box with the largest area in the first image is the target object.

[0105] The processor 12 is used to perform:

[0106] If the second image contains multiple objects, the second human detection box for each object is determined by a human detection algorithm.

[0107] For each second human detection box, determine the Euclidean distance between the first center pixel of the second human detection box and the second center pixel of the second image; and take the object corresponding to the second human detection box with the smallest Euclidean distance as the target object.

[0108] If the second image contains multiple objects, a human detection algorithm is used to determine the second human detection bounding box for each object; simultaneously, the first center pixel of each second human detection bounding box is determined. Furthermore, the Euclidean distance between each first center pixel and the second center pixel of the second image is determined. The object corresponding to the second human detection bounding box with the smallest Euclidean distance is selected as the target object.

[0109] A human detection algorithm is used to detect human bodies in all portraits in the second image, and the output list is [[x1,y1,x2,y2],[],[],....], where x1,y1,x2,y2 are the coordinates of the upper left and lower right corners of the detected human body detection boxes.

[0110] When only one person is detected in the second image, the object within the human detection bounding box is directly taken as the target object. When multiple people are detected in the second image, the object with the closest Euclidean distance between the center coordinate of the human detection bounding box and the center coordinate of the second image is taken as the target object. Let the center coordinate of the second image be center_x, and the center coordinate of the human detection bounding box be Pcenter = (x1 + x2) / 2, then the Euclidean distance is ((center_x - Pcenter) * 2). 0.5 The object in the human detection box with the closest Euclidean distance is selected as the target object.

[0111] Figure 7 The process of determining a target object in a second image provided in this application includes the following steps:

[0112] S201: Determine the human detection bounding boxes for all portraits in the second image using a human detection algorithm.

[0113] S202: Determine whether the number of human detection boxes is at least two. If not, proceed to S203; if yes, proceed to S204.

[0114] S203: Use the object in the human detection box as the target object.

[0115] S204: Select the object whose Euclidean distance from the center point of the human detection box in the second image is closest to the center point of the second image as the target object.

[0116] In this application, to further improve the accuracy of the determined physical height, the processor is configured to perform:

[0117] The physical height of the target object is corrected based on the pre-determined fitting relationship between the measured physical height value and the error value.

[0118] Collect user body data, fit a linear relationship between the error between the measured value and the true value and the measured value. In other words, predetermine the fitting relationship between the physical height measurement value and the error value, and adaptively correct the measured physical height according to the fitting relationship.

[0119] Optional, such as Figure 8As shown, the actual heights of real subjects in three height ranges—155cm-170cm, 171-180cm, and 181-195cm—were collected respectively. The user's measured value was obtained through a height measurement algorithm. The error between the measured value and the actual value was calculated. The LinearRegression() function in the sklearn third-party library of Python was used to achieve linear fitting between the error and the measured value, and the slope k and intercept b of the fitted relationship were obtained. The fitted relationship between the error y and the measured value Y was formed as y = kY + b. Therefore, the corrected physical height is: Y' = Y + kY + b.

[0120] The height measurement method provided in this application will be described in detail below with reference to the accompanying drawings.

[0121] Figure 9 The height measurement flowchart provided for this application includes the following steps:

[0122] S301: Click on the smart body measurement in the front-end interactive interface.

[0123] The front-end interactive interface is the interface displayed on the display panel. The intelligent body measurement function is a reconstruction of the human body model, which requires measuring the parameter values ​​of various dimensions of the body. Height information is also used in the intelligent body measurement function.

[0124] S302: Enables simultaneous shooting with dual RGB cameras.

[0125] S303: Images captured by dual RGB cameras are transmitted to the processor.

[0126] S304: The processor is image-based and uses a Java program in the background management center to call a height measurement algorithm to measure height.

[0127] S305: Transmits the measured height value to the front-end interface for height information display via the back-end management center interface.

[0128] The smart mirror of this application may include a 43-inch 1080P resolution display panel, dual RGB cameras (e.g., 1080P resolution cameras), and a height difference between the dual RGB cameras (e.g., 67.8cm).

[0129] Figure 10 The height measurement process provided in this application includes the following steps:

[0130] S401: Receive a first image captured and transmitted by a first camera; receive a second image captured and transmitted by a second camera, wherein the second image contains a complete target object; the first camera and the second camera are arranged sequentially along the height direction of the display panel.

[0131] S402: Obtain a first key point and a second key point in the first image that have different vertical heights of the target object, and determine a third key point in the second image that corresponds to the first key point and a fourth key point that corresponds to the second key point.

[0132] S403: Determine a first mapping ratio between images based on the first longitudinal distance between the first key point and the second key point, and the second longitudinal distance between the third key point and the fourth key point; determine a second mapping ratio between image distance and physical distance based on the first mapping ratio between images and the longitudinal physical distance between the first camera and the second camera.

[0133] S404: Determine the physical height of the target object based on the third longitudinal distance of the target object in the second image and the second mapping ratio.

[0134] Determining the second mapping ratio between image distance and physical distance based on the first mapping ratio between the images and the longitudinal physical distance between the first camera and the second camera includes:

[0135] Obtain the fourth longitudinal distance between the fifth key point and the first center reference point of the target object in the first image, and map the fourth longitudinal distance onto the second image according to the first mapping ratio to obtain the fifth longitudinal distance;

[0136] Determine the sixth key point in the second image that corresponds to the fifth key point, obtain the sixth longitudinal distance between the sixth key point and the second center reference point in the second image, and determine the second mapping ratio between image distance and physical distance based on the fifth longitudinal distance, the sixth longitudinal distance, and the longitudinal physical distance between the first camera and the second camera.

[0137] Determining the second mapping ratio between image distance and physical distance based on the fifth longitudinal distance, the sixth longitudinal distance, and the longitudinal physical distance between the first camera and the second camera includes:

[0138] If the sixth key point is located above the second center reference point, determine the sum of the fifth longitudinal distance and the sixth longitudinal distance, and determine the second mapping ratio of image distance to physical distance based on the sum and the longitudinal physical distance between the first camera and the second camera;

[0139] If the sixth key point is located below the second center reference point, the difference between the fifth longitudinal distance and the sixth longitudinal distance is determined. Based on the difference and the longitudinal physical distance between the first camera and the second camera, a second mapping ratio between the image distance and the physical distance is determined.

[0140] Using a human skeleton point detection algorithm, a third key point corresponding to the first key point, a fourth key point corresponding to the second key point, and a sixth key point corresponding to the fifth key point are determined in the second image.

[0141] If the first image contains multiple objects, the first human detection box for each object is determined by a human detection algorithm; the object corresponding to the first human detection box with the largest area is taken as the target object.

[0142] If the second image contains multiple objects, the second human detection box for each object is determined by a human detection algorithm.

[0143] For each second human detection box, determine the Euclidean distance between the first center pixel of the second human detection box and the second center pixel of the second image; and take the object corresponding to the second human detection box with the smallest Euclidean distance as the target object.

[0144] The physical height of the target object is corrected based on the pre-determined fitting relationship between the measured physical height value and the error value.

[0145] Figure 11 The schematic diagram of the height measuring device provided in this application shows that the device includes:

[0146] The receiving module 131 is used to receive a first image captured and transmitted by a first camera; and to receive a second image captured and transmitted by a second camera, wherein the second image contains a complete target object; the first camera and the second camera are arranged sequentially along the height direction of the display panel;

[0147] The first determining module 132 is used to acquire a first key point and a second key point with different vertical heights of the target object in the first image, and to determine a third key point in the second image corresponding to the first key point and a fourth key point corresponding to the second key point.

[0148] The second determining module 133 is used to determine a first mapping ratio between images based on the first longitudinal distance between the first key point and the second key point and the second longitudinal distance between the third key point and the fourth key point; and to determine a second mapping ratio between image distance and physical distance based on the first mapping ratio between images and the longitudinal physical distance between the first camera and the second camera.

[0149] The third determining module 134 is used to determine the physical height of the target object based on the third longitudinal distance of the target object in the second image and the second mapping ratio.

[0150] The second determining module 133 is specifically used to obtain the fourth longitudinal distance between the fifth key point and the first center reference point of the target object in the first image, and to map the fourth longitudinal distance to the second image according to the first mapping ratio to obtain the fifth longitudinal distance;

[0151] Determine the sixth key point in the second image that corresponds to the fifth key point, obtain the sixth longitudinal distance between the sixth key point and the second center reference point in the second image, and determine the second mapping ratio between image distance and physical distance based on the fifth longitudinal distance, the sixth longitudinal distance, and the longitudinal physical distance between the first camera and the second camera.

[0152] The second determining module 133 is specifically used to determine the sum of the fifth longitudinal distance and the sixth longitudinal distance if the sixth key point is located above the second center reference point, and to determine the second mapping ratio between the image distance and the physical distance based on the sum and the longitudinal physical distance between the first camera and the second camera.

[0153] If the sixth key point is located below the second center reference point, the difference between the fifth longitudinal distance and the sixth longitudinal distance is determined. Based on the difference and the longitudinal physical distance between the first camera and the second camera, a second mapping ratio between the image distance and the physical distance is determined.

[0154] The first determining module 132 is specifically used to determine, through a human skeleton point detection algorithm, a third key point corresponding to the first key point and a fourth key point corresponding to the second key point in the second image.

[0155] The second determining module 133 is specifically used to determine the sixth key point in the second image corresponding to the fifth key point by using a human skeleton point detection algorithm.

[0156] The first determining module 132 is further configured to, if the first image contains multiple objects, determine the first human detection box of each object through a human detection algorithm; and take the object corresponding to the first human detection box with the largest area as the target object.

[0157] The first determining module 132 is further configured to determine the second human detection box of each object by means of a human detection algorithm if the second image contains multiple objects.

[0158] For each second human detection box, determine the Euclidean distance between the first center pixel of the second human detection box and the second center pixel of the second image; and take the object corresponding to the second human detection box with the smallest Euclidean distance as the target object.

[0159] The third determining module 134 is further configured to correct the physical height of the target object based on a pre-determined fitting relationship between the physical height measurement value and the error value.

[0160] The application also provides a processor, such as Figure 12 As shown, it includes: a processing unit 141, a communication interface 142, a storage unit 143, and a communication bus 144, wherein the processing unit 141, the communication interface 142, and the storage unit 143 communicate with each other through the communication bus 144.

[0161] The storage unit 143 stores a computer program, which, when executed by the processing unit 141, causes the processing unit 141 to perform any of the above method steps.

[0162] The communication bus mentioned above by the processor can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0163] Communication interface 142 is used for communication between the above-mentioned electronic device and other devices.

[0164] The storage unit may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the storage unit may also be at least one storage device located remotely from the aforementioned processing unit.

[0165] The aforementioned processing unit can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0166] This application also provides a computer-readable storage medium storing a computer program executable by a processor, which, when run on the processor, causes the processor to perform any of the above method steps.

[0167] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0168] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A smart mirror, characterized in that, The smart mirror includes: a display panel, a processor, and a first camera and a second camera arranged sequentially along the height direction of the display panel; The first camera is used to capture a first image and send it to the processor; The second camera is used to capture a second image and send it to the processor, wherein the second image contains a complete target object; The processor is used to execute: Obtain a first key point and a second key point in the first image that have different vertical heights of the target object, and determine a third key point in the second image that corresponds to the first key point and a fourth key point that corresponds to the second key point; Based on the first longitudinal distance between the first key point and the second key point, and the second longitudinal distance between the third key point and the fourth key point, a first mapping ratio between the images is determined; based on the first mapping ratio between the images and the longitudinal physical distance between the first camera and the second camera, a second mapping ratio between the image distance and the physical distance is determined. The physical height of the target object is determined based on the third vertical distance of the target object in the second image and the second mapping ratio; The processor is used to execute: Obtain the fourth longitudinal distance between the fifth key point and the first center reference point of the target object in the first image, and map the fourth longitudinal distance onto the second image according to the first mapping ratio to obtain the fifth longitudinal distance; Determine the sixth key point in the second image that corresponds to the fifth key point, obtain the sixth longitudinal distance between the sixth key point and the second center reference point in the second image, and determine the second mapping ratio between image distance and physical distance based on the fifth longitudinal distance, the sixth longitudinal distance, and the longitudinal physical distance between the first camera and the second camera.

2. The smart mirror as described in claim 1, characterized in that, The processor is used to execute: If the sixth key point is located above the second center reference point, determine the sum of the fifth longitudinal distance and the sixth longitudinal distance, and determine the second mapping ratio of image distance to physical distance based on the sum and the longitudinal physical distance between the first camera and the second camera; If the sixth key point is located below the second center reference point, the difference between the fifth longitudinal distance and the sixth longitudinal distance is determined. Based on the difference and the longitudinal physical distance between the first camera and the second camera, a second mapping ratio between the image distance and the physical distance is determined.

3. The smart mirror as described in claim 1, characterized in that, The processor is used to execute: Using a human skeleton point detection algorithm, a third key point corresponding to the first key point, a fourth key point corresponding to the second key point, and a sixth key point corresponding to the fifth key point are determined in the second image.

4. The smart mirror as described in claim 1, characterized in that, The processor is used to execute: If the first image contains multiple objects, the first human detection box for each object is determined by a human detection algorithm; the object corresponding to the first human detection box with the largest area is taken as the target object.

5. The smart mirror as described in claim 1, characterized in that, The processor is used to execute: If the second image contains multiple objects, the second human detection box for each object is determined by a human detection algorithm. For each second human detection box, determine the Euclidean distance between the first center pixel of the second human detection box and the second center pixel of the second image; and take the object corresponding to the second human detection box with the smallest Euclidean distance as the target object.

6. The smart mirror as described in claim 1, characterized in that, The processor is used to execute: The physical height of the target object is corrected based on the pre-determined fitting relationship between the measured physical height value and the error value.

7. A method for measuring height, characterized in that, The method includes: Receive a first image captured and transmitted by a first camera; receive a second image captured and transmitted by a second camera, wherein the second image contains a complete target object; the first camera and the second camera are arranged sequentially along the height direction of the display panel; Obtain a first key point and a second key point in the first image that have different vertical heights of the target object, and determine a third key point in the second image that corresponds to the first key point and a fourth key point that corresponds to the second key point; Based on the first longitudinal distance between the first key point and the second key point, and the second longitudinal distance between the third key point and the fourth key point, a first mapping ratio between the images is determined; based on the first mapping ratio between the images and the longitudinal physical distance between the first camera and the second camera, a second mapping ratio between the image distance and the physical distance is determined. The physical height of the target object is determined based on the third vertical distance of the target object in the second image and the second mapping ratio; Specifically, the fourth longitudinal distance between the fifth key point and the first center reference point of the target object in the first image is obtained, and the fourth longitudinal distance is mapped onto the second image according to the first mapping ratio to obtain the fifth longitudinal distance; Determine the sixth key point in the second image that corresponds to the fifth key point, obtain the sixth longitudinal distance between the sixth key point and the second center reference point in the second image, and determine the second mapping ratio between image distance and physical distance based on the fifth longitudinal distance, the sixth longitudinal distance, and the longitudinal physical distance between the first camera and the second camera.

8. A height measuring device, characterized in that, The device includes: The receiving module is used to receive a first image captured and transmitted by a first camera; and to receive a second image captured and transmitted by a second camera, wherein the second image contains a complete target object; the first camera and the second camera are arranged sequentially along the height direction of the display panel; The first determining module is used to acquire a first key point and a second key point with different vertical heights of the target object in the first image, and to determine a third key point in the second image corresponding to the first key point and a fourth key point corresponding to the second key point; The second determining module is used to determine a first mapping ratio between images based on the first longitudinal distance between the first key point and the second key point and the second longitudinal distance between the third key point and the fourth key point; and to determine a second mapping ratio between image distance and physical distance based on the first mapping ratio between images and the longitudinal physical distance between the first camera and the second camera. The third determining module is used to determine the physical height of the target object based on the third vertical distance of the target object in the second image and the second mapping ratio; Specifically, the fourth longitudinal distance between the fifth key point and the first center reference point of the target object in the first image is obtained, and the fourth longitudinal distance is mapped onto the second image according to the first mapping ratio to obtain the fifth longitudinal distance; Determine the sixth key point in the second image that corresponds to the fifth key point, obtain the sixth longitudinal distance between the sixth key point and the second center reference point in the second image, and determine the second mapping ratio between image distance and physical distance based on the fifth longitudinal distance, the sixth longitudinal distance, and the longitudinal physical distance between the first camera and the second camera.

9. A processor, characterized in that, It includes a processing unit, a communication interface, a storage unit, and a communication bus, wherein the processing unit, the communication interface, and the storage unit communicate with each other through the communication bus; Storage units are used to store computer programs; The processing unit, when executing the program stored in the storage unit, implements the steps of the method described in claim 7.

Images