A target-based distance measurement method, device, and terminal device

Abstract

This application is applicable to the field of image processing technology and provides a target-based distance measurement method, apparatus, and terminal device. The method includes: acquiring target image data, processing the target detection image based on an optimized lightweight target detection network model to obtain an output target detection frame, determining a first distance and a second distance based on the target detection frame, and determining the target distance based on the first distance and the second distance; wherein the target distance is the relative distance from the human head to the camera. This application dynamically determines the distance from the human head to the camera by combining the first distance and the second distance calculated by two algorithms, which can improve the accuracy and robustness of the distance estimation results using a low-cost device.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a target-based distance measurement method, device and terminal equipment. Background Art

[0002] During the movement of the robot, the distance between the robot and other objects needs to be determined in real time.

[0003] Since monocular ranging is cheaper than binocular ranging or lidar ranging, it is often used in various robots (such as small robots or sweeping and logistics robots).

[0004] Related monocular ranging methods include: QR code-based monocular ranging methods, monocular depth estimation-based ranging methods, and triangulation ranging methods based on pinhole imaging principles. Among them, QR code-based monocular ranging methods rely too heavily on QR code generation, resulting in poor measurement stability. Ranging methods based on monocular depth estimation are expensive and easily affected by factors such as lighting and the external environment, resulting in low measurement accuracy in complex environments. Triangulation ranging methods based on pinhole imaging principles offer simple calculations but low measurement accuracy.

[0005] Therefore, how to improve the stability and accuracy of the robot's monocular ranging results at low cost is a problem to be solved. Summary of the Invention

[0006] The embodiments of the present application provide a target-based distance measurement method, apparatus, and terminal device, which can solve the problems of low measurement accuracy and poor stability of related monocular ranging methods.

[0007] In a first aspect, an embodiment of the present application provides a target-based distance measurement method, comprising:

[0008] Acquire target image data; wherein the target image data is an image or video containing a human head;

[0009] Processing the target detection image based on the optimized lightweight target detection network model to obtain an output target detection frame; wherein the target detection frame is used to represent the position of the human head portrait in the target image data;

[0010] determining a first distance and a second distance according to the target detection frame;

[0011] The target distance is determined according to the first distance and the second distance; wherein the target distance is the relative distance from the human head to the camera.

[0012] In one embodiment, determining the first distance and the second distance according to the target detection frame includes:

[0013] Determining the intrinsic parameter matrix and distortion parameters of the camera;

[0014] Determine the center position of the human head according to the target detection frame, and establish a world coordinate system with the center position of the human head as the origin;

[0015] Based on the world coordinate system, calculating the intrinsic parameter matrix and the target detection frame according to a triangulation algorithm to obtain a first distance between the human head and the camera;

[0016] Based on the world coordinate system, the intrinsic parameter matrix, the distortion parameter and the target detection frame are calculated according to the PNP algorithm to obtain a second distance of the human head relative to the camera.

[0017] In one embodiment, the calculating the intrinsic parameter matrix and the target detection frame according to a triangulation algorithm based on the world coordinate system to obtain a first distance of the human head relative to the camera includes:

[0018] Determine the width and height of the human head respectively according to the target detection frame;

[0019] Based on the world coordinate system, the intrinsic parameter matrix and the height are calculated according to the triangulation principle to obtain a first relative height distance of the human head relative to the camera;

[0020] Based on the world coordinate system and according to the principle of triangulation, the intrinsic parameter matrix and the width are calculated to obtain a first width relative distance of the human head relative to the camera.

[0021] In one embodiment, the calculating the intrinsic parameter matrix, the distortion parameter, and the target detection frame according to the PNP algorithm based on the world coordinate system to obtain the second distance of the human head relative to the camera includes:

[0022] Correcting the target image data based on the distortion parameter;

[0023] Calculate the intrinsic parameter matrix, the distortion parameter and the target detection frame according to the PNP algorithm to obtain a pose estimation matrix;

[0024] A second distance between the human head and the camera is determined according to the pose estimation matrix.

[0025] In one embodiment, determining the target distance according to the first distance and the second distance includes:

[0026] detecting whether the target detection frame is located within a preset area in the target image data;

[0027] When it is detected that the target detection frame is located within a preset area of the target image data, determining a ratio of the height to the width;

[0028] When it is detected that the ratio is greater than a first preset threshold, determining that the first distance is a first width relative distance;

[0029] Setting a first weight of the first distance to a first preset weight value, and a second weight of the second distance to a second preset weight value; wherein the first preset weight value is greater than the second preset weight value;

[0030] Alternatively, when it is detected that the ratio is less than a first preset threshold, the first weight is correspondingly set to a second preset weight value, and the second weight is correspondingly set to the first preset weight value;

[0031] A target distance is calculated based on the first distance, the first preset weight value, the second distance, and the second preset weight value.

[0032] In one embodiment, determining the target distance according to the first distance and the second distance further includes:

[0033] When it is detected that the target detection frame is not located in the preset area of the target image data, comparing the width and height;

[0034] When it is detected that the width is greater than the height, determining the first distance as a first width relative distance;

[0035] When it is detected that the height is greater than the width, determining the first distance to be a first height relative distance;

[0036] Setting the first weight to a third preset weight value, and the second weight to a fourth preset weight value;

[0037] The target distance is calculated based on the first distance, the third preset weight value, the second distance and the fourth preset weight value; the third preset weight value is greater than the first preset weight value, and the fourth preset weight value is less than the second preset weight value.

[0038] In one embodiment, the method further comprises:

[0039] The lightweight target detection network model is optimized through the pruning algorithm to obtain a lightweight target detection network model.

[0040] In a second aspect, an embodiment of the present application provides a target-based distance measurement device, comprising:

[0041] A data acquisition module is used to acquire target image data; wherein the target image data is an image or video containing a human head;

[0042] A model processing module is used to process the target detection image based on the optimized lightweight target detection network model to obtain an output target detection frame; wherein the target detection frame is used to represent the position of the human head portrait in the target image data;

[0043] A first distance determination module, configured to determine a first distance and a second distance according to the target detection frame;

[0044] The second distance determination module is used to determine the target distance according to the first distance and the second distance; wherein the target distance is the relative distance from the human head to the camera.

[0045] In one embodiment, the first distance determination module includes:

[0046] A parameter determination submodule, used to determine the intrinsic parameter matrix and distortion parameters of the camera;

[0047] A coordinate system establishment submodule is used to determine the center position of the human head according to the target detection frame and establish a world coordinate system with the center position of the human head as the origin;

[0048] A first distance determination submodule is configured to calculate the intrinsic parameter matrix and the target detection frame based on the world coordinate system and a triangulation distance measurement algorithm to obtain a first distance between the human head and the camera;

[0049] The second distance determination submodule is used to calculate the intrinsic parameter matrix, the distortion parameter and the target detection frame based on the world coordinate system according to the PNP algorithm to obtain a second distance between the human head and the camera.

[0050] In one embodiment, the first distance determination submodule includes:

[0051] a data determination unit, configured to determine the width and height of the human head according to the target detection frame;

[0052] a height relative distance determining unit, configured to calculate the intrinsic parameter matrix and the height based on the world coordinate system and according to a triangulation principle to obtain a first height relative distance of the human head relative to the camera;

[0053] The width relative distance determination unit is used to calculate the internal parameter matrix and the width based on the world coordinate system and the triangulation principle to obtain a first width relative distance of the human head relative to the camera.

[0054] In one embodiment, the second distance determination submodule includes:

[0055] a correction unit, configured to correct the target image data based on the distortion parameter;

[0056] A pose estimation unit is used to calculate the intrinsic parameter matrix, the distortion parameter and the target detection frame according to the PNP algorithm to obtain a pose estimation matrix;

[0057] A second distance determining unit is used to determine a second distance between the human head and the camera according to the posture estimation matrix.

[0058] In one embodiment, the second distance determination module includes:

[0059] A position detection submodule, configured to detect whether the target detection frame is located within a preset area in the target image data;

[0060] an aspect ratio determination submodule, configured to determine a ratio of the height to the width when detecting that the target detection frame is located within a preset area of the target image data;

[0061] a ratio determination submodule, configured to determine that the first distance is a first width relative distance when detecting that the ratio is greater than a first preset threshold;

[0062] a first weight determination submodule, configured to set a first weight of the first distance to a first preset weight value, and a second weight of the second distance to a second preset weight value; wherein the first preset weight value is greater than the second preset weight value;

[0063] a second weight determination submodule, configured to, when detecting that the ratio is less than a first preset threshold, correspondingly set the first weight to a second preset weight value and the second weight to the first preset weight value;

[0064] The first target distance determination submodule is configured to calculate the target distance according to the first distance, the first preset weight value, the second distance, and the second preset weight value.

[0065] In one embodiment, the second distance determination module further includes:

[0066] a comparison submodule, configured to compare the width and height when it is detected that the target detection frame is not located within a preset area of the target image data;

[0067] a first selection submodule, configured to determine that the first distance is a first width relative distance when detecting that the width is greater than the height;

[0068] a second selection submodule, configured to determine that the first distance is a first height-relative distance when detecting that the height is greater than the width;

[0069] a third weight determination submodule, configured to set the first weight to a third preset weight value, and the second weight to a fourth preset weight value;

[0070] The second target distance determination submodule is used to calculate the target distance based on the first distance, the third preset weight value, the second distance and the fourth preset weight value; the third preset weight value is greater than the first preset weight value, and the fourth preset weight value is less than the second preset weight value.

[0071] In one embodiment, the apparatus further comprises:

[0072] The model optimization module is used to optimize the lightweight target detection network model through the pruning algorithm to obtain a lightweight target detection network model.

[0073] In a third aspect, an embodiment of the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the target-based distance measurement method as described in any one of the first aspects above is implemented.

[0074] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the target-based distance measurement method as described in any one of the first aspects above is implemented.

[0075] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the target-based distance measurement method described in any one of the above-mentioned first aspects.

[0076] Compared with the prior art, the embodiments of the present application have the following advantages: acquiring target image data, processing the target detection image based on an optimized lightweight target detection network model, outputting a target detection frame, determining a first distance and a second distance based on the target detection frame, and determining the target distance based on the first and second distances. By combining the first and second distances calculated by the two algorithms, the distance from the human head to the camera is dynamically determined, improving the accuracy and robustness of the distance estimation results using a low-cost device.

[0077] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0079] Figure 1 1 is a flow chart of a target-based distance measurement method provided in an embodiment of the present application;

[0080] Figure 2 1 is a flow chart of step S103 of the target-based distance measurement method provided in an embodiment of the present application;

[0081] Figure 3 1 is a flow chart of step S104 of the target-based distance measurement method provided in an embodiment of the present application;

[0082] Figure 4 1 is another flow chart of step S104 of the target-based distance measurement method provided in an embodiment of the present application;

[0083] Figure 5 is a schematic structural diagram of a target-based distance measurement device provided in an embodiment of the present application;

[0084] Figure 6 It is a structural diagram of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0086] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0087] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0088] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0089] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0090] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0091] The target-based distance measurement method provided in the embodiments of the present application can be applied to robots (such as small robots, sweeping robots, logistics robots or intelligent robots), mobile phones, tablet computers, vehicle-mounted equipment, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs) and other terminal devices. The embodiments of the present application do not impose any restrictions on the specific type of terminal device.

[0092] Figure 1 A schematic flow chart of a target-based distance measurement method provided in the present application is shown. As an example but not a limitation, the method can be applied to the above-mentioned robot.

[0093] S101. Acquire target image data; wherein the target image data is an image or video containing a human head.

[0094] Specifically, a monocular camera (hereinafter referred to as camera) is pre-installed on the top of the robot. The monocular camera is used to capture images or video data in front of the robot in real time, with images or video data containing a human head as target image data.

[0095] As an example, and not a limitation, when the target image data is a video, each single frame is processed separately as a processing object, and the distance measurement results (i.e., target distances) are averaged to obtain the corresponding relative distance between the human head and the camera. For example, if the target image data is a 20-frame video, each of these 20 frames is processed separately, and the average of the 20 target distances calculated based on these 20 frames is used as the relative distance between the human head and the camera in the image.

[0096] S102. Process the target detection image based on the optimized lightweight target detection network model to obtain an output target detection frame; wherein the target detection frame is used to represent the position of the human head portrait in the target image data.

[0097] Specifically, target detection processing is performed on the target detection image through the pre-optimized lightweight target detection network model to determine the position of each human head portrait in the target detection image in the target image data, and the target detection frame corresponding to each human head portrait is output by the optimized lightweight target detection network model.

[0098] S103: Determine a first distance and a second distance according to the target detection frame.

[0099] Specifically, the target detection frame is calculated according to the triangulation algorithm to obtain a first distance between the human head and the camera, and the target detection frame is calculated according to the PNP (Perspective-n-Point) algorithm to obtain a second distance between the human head and the camera.

[0100] S104. Determine a target distance according to the first distance and the second distance; wherein the target distance is the relative distance from the human head to the camera.

[0101] Specifically, a first weight corresponding to the first distance and a second weight corresponding to the second distance are set, and a target distance is calculated based on the first distance, the first weight, the second distance, and the second weight, wherein the target distance is the relative distance from the human head to the camera.

[0102] like Figure 2 As shown, in one embodiment, the step S103 of determining the first distance and the second distance according to the target detection frame includes:

[0103] S1031. Determine the intrinsic parameter matrix and distortion parameters of the camera;

[0104] S1032: Determine the center position of the human head according to the target detection frame, and establish a world coordinate system with the center position of the human head as the origin;

[0105] S1033. Based on the world coordinate system, calculate the intrinsic parameter matrix and the target detection frame according to a triangulation algorithm to obtain a first distance between the human head and the camera;

[0106] S1034. Based on the world coordinate system, calculate the intrinsic parameter matrix, the distortion parameter, and the target detection frame according to the PNP algorithm to obtain a second distance between the human head and the camera.

[0107] Specifically, the camera is calibrated to determine the camera's intrinsic parameter matrix and distortion parameters. The width and height of the human head in the target detection image are determined based on the target detection frame, and the center position of the human head is determined based on the width and height (specifically, the position of the center of the human head in the target detection image, which can be represented by the average value of the horizontal coordinate and the average value of the vertical coordinate of the human head in the target detection image). Based on the world coordinate system with the center position of the human head as the origin, the average data of the human head is used as the size of the human head (specifically, the length from the left ear to the right ear is used as the width of the human head 20cm, and the length from the top of the head to the chin is used as the height of the human head 25cm). Based on the world coordinate system with the center position of the human head as the origin, the three-dimensional coordinates of the human head are determined according to the target detection frame, and the pinhole imaging principle (similar triangles) is used to calculate the intrinsic parameter matrix, the size of the human head, and the width (or height) of the human head in the target detection image to obtain the first distance of the human head relative to the camera.

[0108] Specifically, the target detection image is corrected based on the distortion parameters and the intrinsic parameter matrix. Based on the world coordinate system with the center position of the human head as the origin, the three-dimensional coordinates of the human head are determined according to the target detection frame. The average data of the human head is used as the size of the human head (specifically, the length from the left ear to the right ear is used as the width of the human head 20 cm, and the length from the top of the head to the chin is used as the height of the human head 25 cm). According to the PNP algorithm, the intrinsic parameter matrix, the size of the human head, and the three-dimensional coordinates of the human head are calculated to obtain the second distance of the human head relative to the camera.

[0109] In one embodiment, the calculating the intrinsic parameter matrix and the target detection frame according to a triangulation algorithm based on the world coordinate system to obtain a first distance of the human head relative to the camera includes:

[0110] Determine the width and height of the human head respectively according to the target detection frame;

[0111] Based on the world coordinate system, the intrinsic parameter matrix and the height are calculated according to the triangulation principle to obtain a first relative height distance of the human head relative to the camera;

[0112] Based on the world coordinate system and according to the principle of triangulation, the intrinsic parameter matrix and the width are calculated to obtain a first width relative distance of the human head relative to the camera.

[0113] Specifically, the width and height of the human head in the target detection image are determined respectively according to the target detection frame, and based on the world coordinate system with the center position of the human head as the origin, the internal parameter matrix, the height of the human head (that is, the length from the top of the head to the chin of the human head as the average height of the human head of 25 cm) and the height of the human head in the target detection image are calculated according to the principle of triangulation, and the first height relative distance of the human head relative to the camera is obtained. Based on the world coordinate system with the center position of the human head as the origin, the internal parameter matrix, the width of the human head (that is, the length from the top of the head to the chin of the human head as the average width of the human head of 20 cm) and the width of the human head in the target detection image are calculated according to the principle of triangulation, and the first width relative distance of the human head relative to the camera is obtained.

[0114] In one embodiment, the calculating the intrinsic parameter matrix, the distortion parameter, and the target detection frame according to the PNP algorithm based on the world coordinate system to obtain the second distance of the human head relative to the camera includes:

[0115] Correcting the target image data based on the distortion parameter;

[0116] Calculate the intrinsic parameter matrix, the distortion parameter and the target detection frame according to the PNP algorithm to obtain a pose estimation matrix;

[0117] A second distance between the human head and the camera is determined according to the pose estimation matrix.

[0118] Specifically, the target image data is corrected according to the distortion parameters and intrinsic parameter matrix of the camera. Based on the world coordinate system with the center position of the human head in the target detection image as the origin, the three-dimensional coordinates of the human head are determined according to the target detection frame. The intrinsic parameter matrix, the size of the human head and the distortion parameters are calculated according to the PNP algorithm to obtain the pose estimation matrix. The z-axis value in the translation matrix in the pose estimation matrix is used as the second distance of the human head relative to the camera.

[0119] like Figure 3 As shown, in one embodiment, the step S104 of determining the target distance according to the first distance and the second distance includes:

[0120] S1041, detecting whether the target detection frame is located within a preset area in the target image data;

[0121] S1042: When it is detected that the target detection frame is located within a preset area of the target image data, determining a ratio of the height to the width;

[0122] S1043: When it is detected that the ratio is greater than a first preset threshold, determining that the first distance is a first width relative distance;

[0123] S1044: Set a first weight of the first distance to a first preset weight value, and a second weight of the second distance to a second preset weight value; wherein the first preset weight value is greater than the second preset weight value;

[0124] S1045. Alternatively, when it is detected that the ratio is less than a first preset threshold, the first weight is correspondingly set to a second preset weight value, and the second weight is correspondingly set to the first preset weight value;

[0125] S1046: Calculate a target distance based on the first distance, the first preset weight value, the second distance, and the second preset weight value.

[0126] Specifically, the preset area is the center area of the target detection image. The location and size of the center area can be determined based on actual needs. For example, the area within 20 pixels of the four boundaries (including the upper boundary, lower boundary, left boundary, and right boundary) of the target detection image is set as the center area.

[0127] Specifically, based on the position of the target detection frame in the target detection image, it is determined whether the target detection frame is located within the preset area of the target image data. When it is detected that the four boundaries of the target image data are all within the four boundaries of the preset area, it is determined that the human head image data is complete and clear, and the ratio of the height and width of the human head in the target detection image is calculated. The above-mentioned height and width ratio is compared with a first preset threshold. The first preset threshold can be specifically set according to actual conditions. For example, the first preset threshold is set to 1.3.

[0128] Specifically, when the ratio of height to width is detected to be greater than a first preset threshold, it is determined that the detected height of the human head may be inaccurate due to reasons such as hairstyle (such as long hair) or wearing a hat. This result will affect the second distance obtained by the PNP algorithm. The triangulation result can be used to calculate the width to obtain a more accurate first distance. Correspondingly, the first distance is determined to be the relative distance of the first width. The first weight of the first distance is set to the first preset weight value, and the second weight of the second distance is set to the second preset weight value; based on the first distance, the first preset weight value, the second distance and the second preset weight value, the target distance is calculated.

[0129] The first preset weight value is greater than the second preset weight value, and the sum of the first and second weight values is 2. The first and second weight values can be set according to actual needs. For example, if the first weight value is set to 1.5 and the second weight value is set to 0.5, the corresponding target distance = (first distance (specifically, the first width relative distance) * 1.5 + second distance * 0.5) / 2.

[0130] Specifically, when the ratio is detected to be less than a first preset threshold, the second distance measured by the PNP algorithm is more accurately measured. Accordingly, the first weight is set to the second preset weight value, and the second weight is set to the first preset weight value. The target distance is calculated based on the first distance, the first preset weight value, the second distance, and the second preset weight value.

[0131] For example, if the first weight value is set to 1.5 and the second weight value is set to 0.5, the corresponding target distance = (first distance * 0.5 + PNP second distance * 1.5) / 2.

[0132] like Figure 4 As shown, in one embodiment, the step S104 of determining the target distance according to the first distance and the second distance further includes:

[0133] S1047: When it is detected that the target detection frame is not located within the preset area of the target image data, compare the width and height;

[0134] S1048. When it is detected that the width is greater than the height, determine the first distance as a first width relative distance;

[0135] S1049: When it is detected that the height is greater than the width, determine that the first distance is a first height relative distance;

[0136] S10410: Set the first weight to a third preset weight value, and the second weight to a fourth preset weight value;

[0137] S10411. Calculate the target distance based on the first distance, the third preset weight value, the second distance, and the fourth preset weight value; the third preset weight value is greater than the first preset weight value, and the fourth preset weight value is less than the second preset weight value.

[0138] Specifically, based on the position of the target detection frame in the target detection image, determine whether the target detection frame is located within the preset area of the target image data. When it is detected that the four boundaries of the target image data are not within the four boundaries of the preset area, it is determined that the target detection frame is not located within the preset area of the target image data. Because the target detection frame detected by the detector at the edge position will be deformed (if the human head exceeds the target detection image, it may result in only the side face or chin of the person being detected). At this time, the second distance measured by the PNP algorithm will be more disturbed, and the first distance obtained by selecting the longer side based on the triangulation algorithm will have higher accuracy. Correspondingly, the width and height need to be compared.

[0139] Specifically, when it is detected that the width of a human head is greater than its height, the first distance is determined to be the first width relative distance, the first weight corresponding to the first distance is set to the third preset weight value, and the second weight is set to the fourth preset weight value. The target distance is calculated based on the first distance, the third preset weight value, the second distance, and the fourth preset weight value; the third preset weight value is greater than the first preset weight value, and the fourth preset weight value is less than the second preset weight value. The sum of the third and fourth weight values is 2.

[0140] Specifically, when it is detected that the height of the human head is greater than the width, the first distance is determined to be the first height relative distance; the first weight corresponding to the first distance is set to the third preset weight value, and the second weight is set to the fourth preset weight value. According to the first distance, the third preset weight value, the second distance and the fourth preset weight value, the target distance is calculated. Among them, the third preset weight value and the fourth preset weight value can be specifically set according to actual needs. For example, the third preset weight value is 1.8 and the fourth preset weight value is 0.2. The corresponding target distance = (first distance * 1.8 + second distance * 0.2) / 2.

[0141] For example, if the height of a human head is 26 cm and the width is 20 cm, the first distance is determined to be the first height relative distance. The corresponding target distance = (first height relative distance * 1.8 + second distance * 0.2) / 2.

[0142] Based on the actual situation of the target detection image, the target distance calculation method is dynamically judged and adjusted, which improves the robustness of the distance estimation method and the stability of the measurement results.

[0143] In one embodiment, the method further comprises:

[0144] The lightweight target detection network model is optimized through the pruning algorithm to obtain a lightweight target detection network model.

[0145] Specifically, the pre-trained lightweight object detection network yolox-tiny model is optimized using the L1 filter pruning algorithm. The contribution of each filter to the model is calculated, and filters within a preset range are selected. Filters outside the preset range are filtered out to obtain a lightweight object detection network model. This reduces the model's parameter count and computational complexity, thereby improving the model's running speed. The preset range can be set based on actual needs. For example, the preset range can be set to the top 50% or top 60% of contributing filters.

[0146] In one embodiment, the method further comprises:

[0147] Acquire a plurality of training image data; wherein the training image data is image data containing a human head portrait;

[0148] Add corresponding annotations to each human head image in each training image data;

[0149] The training image data is input into the lightweight object detection network model for pre-training to obtain a pre-trained lightweight object detection network model.

[0150] This embodiment combines the first distance and the second distance calculated by the two algorithms to dynamically determine the distance from the human head to the camera, which can improve the accuracy and robustness of the distance estimation result with a low-cost device.

[0151] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0152] Corresponding to the target-based distance measurement method described in the above embodiment, Figure 5 A structural block diagram of a target-based distance measurement device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0153] Reference Figure 5 , the target-based distance measurement device 100 includes:

[0154] The data acquisition module 101 is used to acquire target image data; wherein the target image data is an image or video containing a human head;

[0155] The model processing module 102 is used to process the target detection image based on the optimized lightweight target detection network model to obtain an output target detection frame; wherein the target detection frame is used to represent the position of the human head portrait in the target image data;

[0156] A first distance determination module 103 is configured to determine a first distance and a second distance according to the target detection frame;

[0157] The second distance determining module 104 is configured to determine a target distance according to the first distance and the second distance; wherein the target distance is the relative distance from the human head to the camera.

[0158] In one embodiment, the first distance determination module includes:

[0159] A parameter determination submodule, used to determine the intrinsic parameter matrix and distortion parameters of the camera;

[0160] A coordinate system establishment submodule is used to determine the center position of the human head according to the target detection frame and establish a world coordinate system with the center position of the human head as the origin;

[0161] A first distance determination submodule is configured to calculate the intrinsic parameter matrix and the target detection frame based on the world coordinate system and a triangulation distance measurement algorithm to obtain a first distance between the human head and the camera;

[0162] The second distance determination submodule is used to calculate the intrinsic parameter matrix, the distortion parameter and the target detection frame based on the world coordinate system according to the PNP algorithm to obtain a second distance between the human head and the camera.

[0163] In one embodiment, the first distance determination submodule includes:

[0164] a data determination unit, configured to determine the width and height of the human head according to the target detection frame;

[0165] a height relative distance determining unit, configured to calculate the intrinsic parameter matrix and the height based on the world coordinate system and according to a triangulation principle to obtain a first height relative distance of the human head relative to the camera;

[0166] The width relative distance determination unit is used to calculate the internal parameter matrix and the width based on the world coordinate system and the triangulation principle to obtain a first width relative distance of the human head relative to the camera.

[0167] In one embodiment, the second distance determination submodule includes:

[0168] a correction unit, configured to correct the target image data based on the distortion parameter;

[0169] A pose estimation unit is used to calculate the intrinsic parameter matrix, the distortion parameter and the target detection frame according to the PNP algorithm to obtain a pose estimation matrix;

[0170] A second distance determining unit is used to determine a second distance between the human head and the camera according to the posture estimation matrix.

[0171] In one embodiment, the second distance determination module includes:

[0172] A position detection submodule, configured to detect whether the target detection frame is located within a preset area in the target image data;

[0173] an aspect ratio determination submodule, configured to determine a ratio of the height to the width when detecting that the target detection frame is located within a preset area of the target image data;

[0174] a ratio determination submodule, configured to determine that the first distance is a first width relative distance when detecting that the ratio is greater than a first preset threshold;

[0175] a first weight determination submodule, configured to set a first weight of the first distance to a first preset weight value, and a second weight of the second distance to a second preset weight value; wherein the first preset weight value is greater than the second preset weight value;

[0176] a second weight determination submodule, configured to, when detecting that the ratio is less than a first preset threshold, correspondingly set the first weight to a second preset weight value and the second weight to the first preset weight value;

[0177] The first target distance determination submodule is configured to calculate the target distance according to the first distance, the first preset weight value, the second distance, and the second preset weight value.

[0178] In one embodiment, the second distance determination module further includes:

[0179] a comparison submodule, configured to compare the width and height when it is detected that the target detection frame is not located within a preset area of the target image data;

[0180] a first selection submodule, configured to determine that the first distance is a first width relative distance when detecting that the width is greater than the height;

[0181] a second selection submodule, configured to determine that the first distance is a first height-relative distance when detecting that the height is greater than the width;

[0182] a third weight determination submodule, configured to set the first weight to a third preset weight value, and the second weight to a fourth preset weight value;

[0183] The second target distance determination submodule is used to calculate the target distance based on the first distance, the third preset weight value, the second distance and the fourth preset weight value; the third preset weight value is greater than the first preset weight value, and the fourth preset weight value is less than the second preset weight value.

[0184] In one embodiment, the apparatus further comprises:

[0185] The model optimization module is used to optimize the lightweight target detection network model through the pruning algorithm to obtain a lightweight target detection network model.

[0186] This embodiment dynamically determines the distance from the human head to the camera by combining the first distance and the second distance calculated by two algorithms, which can improve the accuracy and robustness of the distance estimation result with a low-cost device.

[0187] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0188] Figure 6 This is a schematic diagram of the structure of the terminal device provided in this embodiment. Figure 6 As shown, the terminal device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown), a memory 61 and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, wherein the processor 60 implements the steps of any of the above-mentioned target-based distance measurement method embodiments when executing the computer program 62.

[0189] The terminal device 6 can be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor 60 and a memory 61. It will be understood by those skilled in the art that Figure 6 It is only an example of the terminal device 6 and does not constitute a limitation on the terminal device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0190] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0191] In some embodiments, the memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. In other embodiments, the memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 6. Furthermore, the memory 61 may also include both an internal storage unit of the terminal device 6 and an external storage device. The memory 61 is used to store an operating system, application programs, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 61 may also be used to temporarily store data that has been output or is about to be output.

[0192] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0193] An embodiment of the present application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.

[0194] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0195] An embodiment of the present application provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned various method embodiments when executing the computer program product.

[0196] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process of the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0197] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0198] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0199] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0200] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0201] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A target-based distance measurement method, characterized in that: include: Acquire target image data; wherein the target image data is an image or video containing a human head; The target detection image is processed based on the optimized lightweight target detection network model to obtain an output target detection frame; wherein the target detection frame is used to represent the position of the human head portrait in the target image data; Determine a first distance and a second distance according to the target detection frame; the first distance is a first distance between the human head and the camera, and the second distance is a second distance between the human head and the camera; Determining the target distance based on the first distance and the second distance includes: detecting whether the target detection frame is located in a preset area in the target image data; determining the ratio of height to width when detecting that the target detection frame is located in the preset area of the target image data; determining the first distance as a first width relative distance when detecting that the ratio is greater than a first preset threshold; setting a first weight of the first distance to a first preset weight value, and a second weight of the second distance to a second preset weight value; wherein the first preset weight value is greater than the second preset weight value; or, when detecting that the ratio is less than the first preset threshold, correspondingly setting the first weight to the second preset weight value, and the second weight to the first preset weight value; calculating the target distance based on the first distance, the first preset weight value, the second distance and the second preset weight value; wherein the target distance is the relative distance from the human head to the camera.

2. The target-based distance measurement method according to claim 1, wherein: Determining the first distance and the second distance according to the target detection frame includes: Determining the intrinsic parameter matrix and distortion parameters of the camera; Determine the center position of the human head according to the target detection frame, and establish a world coordinate system with the center position of the human head as the origin; Based on the world coordinate system, calculating the intrinsic parameter matrix and the target detection frame according to a triangulation algorithm to obtain a first distance between the human head and the camera; Based on the world coordinate system, the intrinsic parameter matrix, the distortion parameter and the target detection frame are calculated according to the PNP algorithm to obtain a second distance of the human head relative to the camera.

3. The target-based distance measurement method according to claim 2, wherein: The step of calculating the intrinsic parameter matrix and the target detection frame based on the world coordinate system and using a triangulation algorithm to obtain a first distance between the human head and the camera includes: Determine the width and height of the human head respectively according to the target detection frame; Based on the world coordinate system, the intrinsic parameter matrix and the height are calculated according to the triangulation principle to obtain a first relative height distance of the human head relative to the camera; Based on the world coordinate system and according to the principle of triangulation, the intrinsic parameter matrix and the width are calculated to obtain a first width relative distance of the human head relative to the camera.

4. The target-based distance measurement method according to claim 2, wherein: The step of calculating the intrinsic parameter matrix, the distortion parameter, and the target detection frame according to a PNP algorithm based on the world coordinate system to obtain a second distance of the human head relative to the camera includes: Correcting the target image data based on the distortion parameter; Calculate the intrinsic parameter matrix, the distortion parameter and the target detection frame according to the PNP algorithm to obtain a pose estimation matrix; A second distance between the human head and the camera is determined according to the pose estimation matrix.

5. The target-based distance measurement method according to claim 1, wherein: The determining of the target distance according to the first distance and the second distance further includes: When it is detected that the target detection frame is not located in the preset area of the target image data, comparing the width and height; When it is detected that the width is greater than the height, determining the first distance as a first width relative distance; When it is detected that the height is greater than the width, determining the first distance to be a first height relative distance; Setting the first weight to a third preset weight value, and the second weight to a fourth preset weight value; The target distance is calculated based on the first distance, the third preset weight value, the second distance and the fourth preset weight value; the third preset weight value is greater than the first preset weight value, and the fourth preset weight value is less than the second preset weight value.

6. The target-based distance measurement method according to claim 1, wherein: The method further comprises: The lightweight target detection network model is optimized through the pruning algorithm to obtain a lightweight target detection network model.

7. A target-based distance measurement device, characterized in that: include: A data acquisition module is used to acquire target image data; wherein the target image data is an image or video containing a human head; A model processing module is used to process the target detection image based on the optimized lightweight target detection network model to obtain an output target detection frame; wherein the target detection frame is used to represent the position of the human head portrait in the target image data; A first distance determination module is configured to determine a first distance and a second distance based on the target detection frame; the first distance is a first distance between the human head and the camera, and the second distance is a second distance between the human head and the camera; a second distance determination module, configured to determine a target distance based on the first distance and the second distance; wherein the target distance is the relative distance from the human head to the camera; The second distance determination module includes: A position detection submodule, configured to detect whether the target detection frame is located within a preset area in the target image data; an aspect ratio determination submodule, configured to determine a ratio of height to width when detecting that the target detection frame is located within a preset area of the target image data; a ratio determination submodule, configured to determine that the first distance is a first width relative distance when detecting that the ratio is greater than a first preset threshold; a first weight determination submodule, configured to set a first weight of the first distance to a first preset weight value, and a second weight of the second distance to a second preset weight value; wherein the first preset weight value is greater than the second preset weight value; a second weight determination submodule, configured to, when detecting that the ratio is less than a first preset threshold, correspondingly set the first weight to a second preset weight value and the second weight to the first preset weight value; The first target distance determination submodule is configured to calculate the target distance according to the first distance, the first preset weight value, the second distance, and the second preset weight value.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images