Distance measurement method and device, electronic device and storage medium
The distance and transmission parameters in the image are obtained through image processing technology, which solves the problem of inaccurate measurement of distance between people and between objects, improves the accuracy of judging safety distances, and reduces the occurrence of safety accidents.
Patent Information
- Application Number
- CN202010613403.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-06-30
AI Technical Summary
The prior art is difficult to accurately measure the distance between people and objects, especially in specific scenarios, where the probability of safety accidents is high.
By acquiring the distance and transmission parameters in the image to be processed, using the scale information of the pixel point and object detection processing, the second distance is calculated, the accuracy of the distance measurement is improved, and an alarm is issued when the distance is too close.
Accurate measurement of the distance between people and between objects is achieved, the accuracy of judging safety distances is improved, and the probability of safety accidents is reduced.
Smart Images

Figure CN111739097B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a distance measurement method and device, an electronic device, and a storage medium. Background Art
[0002] To reduce the probability of safety accidents during specific periods or scenarios, people and objects should maintain distance from each other. For example, close proximity between people can easily lead to the spread of disease, while close proximity between vehicles can easily cause traffic accidents. Therefore, accurately determining the distance between people and / or objects is extremely important. Summary of the Invention
[0003] The present application provides a ranging method and device, an electronic device, and a storage medium.
[0004] In a first aspect, a ranging method is provided, the ranging method comprising:
[0005] Obtaining a first distance in an image to be processed and a first transmission parameter of the first distance; the first distance is the distance between a first point and a second point; both the first point and the second point belong to the image to be processed; the first transmission parameter represents a conversion relationship between the first distance and a first physical distance; the first physical distance is a physical distance corresponding to the first distance;
[0006] A second distance is obtained based on the first distance and the first transmission parameter; the second distance is negatively correlated with the scale of the first point in the image to be processed, and / or the second distance is negatively correlated with the scale of the second point in the image to be processed.
[0007] In this aspect, since the first transmission parameter carries the scale information of the first distance, the distance measuring device determines the second distance based on the first transmission parameter and the first distance, which can improve the accuracy of the second distance.
[0008] In combination with any embodiment of the present application, obtaining the first transmission parameter of the first distance includes:
[0009] Obtaining a second transmission parameter of a first pixel; the first pixel is a pixel on a line connecting the first point and the second point; the second transmission parameter represents a conversion relationship between a size of the first pixel and a size of a first object point; the first object point is an object point corresponding to the first pixel; the first ratio is negatively correlated with a scale of the first pixel in the image to be processed; the first ratio is the ratio between the size of the first pixel and the size of the first object point;
[0010] The first transmission parameter is obtained according to the second transmission parameter; the first transmission parameter is positively correlated with the second transmission parameter.
[0011] In combination with any embodiment of the present application, the first pixel point is the midpoint between the first point and the second point, and the first transmission parameter is obtained based on the second transmission parameter.
[0012] In combination with any embodiment of the present application, obtaining the second transmission parameter of the first pixel includes:
[0013] Performing object detection processing on the image to be processed to obtain the position of a first object frame and the position of a second object frame; the first object frame contains a first object; and the second object frame contains a second object;
[0014] Obtaining a first size of the first object according to the position of the first object frame, and obtaining a second size of the second object according to the position of the second object frame;
[0015] A third transmission parameter is obtained based on the first size and the third size, and a fourth transmission parameter is obtained based on the second size and the fourth size; the third size is the physical size of the first object; the third transmission parameter represents a conversion relationship between a fifth size and a sixth size; the fifth size is the size of a second pixel; the position of the second pixel in the image to be processed is determined based on the position of the first object frame; the sixth size is the size of the object point corresponding to the second pixel; the fourth size is the physical size of the second object; the fourth transmission parameter represents a conversion relationship between a seventh size and an eighth size; the seventh size is the size of a third pixel; the position of the third pixel in the image to be processed is determined based on the position of the second object frame; the eighth size is the size of the object point corresponding to the third pixel;
[0016] Performing curve fitting processing on the third transmission parameter and the fourth transmission parameter to obtain a transmission parameter map of the image to be processed; a conversion relationship between the ninth dimension and the tenth dimension is determined based on a first pixel value in the transmission parameter map; the ninth dimension is the dimension of a fourth pixel in the image to be processed; the tenth dimension is the dimension of an object point corresponding to the fourth pixel; the first pixel value is the pixel value of a fifth pixel; and the fifth pixel is the pixel corresponding to the fourth pixel in the transmission parameter map;
[0017] The second transmission parameter is obtained according to the pixel value corresponding to the first pixel point in the transmission parameter map.
[0018] In combination with any embodiment of the present application, before performing curve fitting processing on the third transmission parameter and the fourth transmission parameter to obtain the transmission parameter map of the image to be processed, the method further includes:
[0019] Acquiring a confidence map; the confidence map represents a mapping between object types and confidence levels of transmission parameters;
[0020] Obtaining a first confidence level of the third transmission parameter according to the object type of the first object and the confidence level map;
[0021] The performing curve fitting processing on the third transmission parameter and the fourth transmission parameter to obtain a transmission parameter map of the image to be processed includes:
[0022] Obtaining a fifth transmission parameter according to the first confidence level and the third transmission parameter; wherein the fifth transmission parameter is positively correlated with the first confidence level;
[0023] Curve fitting is performed on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter graph.
[0024] In combination with any embodiment of the present application, before obtaining the first confidence of the third transmission parameter based on the object type of the first object and the confidence map, the method further includes:
[0025] Performing feature extraction processing on the pixel point area within the first object frame to obtain feature data;
[0026] Obtaining a score for the first object based on the feature data; wherein the score is positively correlated with a confidence level of a size of the first object;
[0027] Obtaining a first confidence level of the third transmission parameter based on the object type of the first object and the confidence level mapping includes:
[0028] Obtaining a second confidence level of the third transmission parameter based on the object type of the first object and the confidence level map;
[0029] The first confidence level is obtained based on the score and the second confidence level; the first confidence level is correlated with the score.
[0030] In combination with any embodiment of the present application, obtaining the fifth transmission parameter based on the first confidence level and the third transmission parameter includes:
[0031] A product of the first confidence level and the third transmission parameter is determined to obtain the fifth transmission parameter.
[0032] In combination with any embodiment of the present application, before performing curve fitting processing on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter map, the method further includes:
[0033] Acquire a depth image of the image to be processed;
[0034] Obtaining first depth information of the second pixel and second depth information of the third pixel according to the depth image;
[0035] Obtaining a first data point according to the first depth information and the fifth transmission parameter, and obtaining a second data point according to the second depth information and the fourth transmission parameter;
[0036] The performing curve fitting processing on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter graph includes:
[0037] Curve fitting is performed on the first data point and the second data point to obtain the transmission parameter map.
[0038] In combination with any embodiment of the present application, the first point and the second point are both character points; and the method further includes:
[0039] When the second distance does not exceed a safety distance threshold, obtaining the position of the first point;
[0040] An alarm instruction including the position is sent to the terminal; the alarm instruction is used to instruct the terminal to output alarm information that the distance between people is too close.
[0041] In a second aspect, a distance measuring device is provided, the distance measuring device comprising:
[0042] an acquisition unit, configured to acquire a first distance in an image to be processed and a first transmission parameter of the first distance; the first distance being the distance between a first point and a second point; the first point and the second point both belonging to the image to be processed; the first transmission parameter representing a conversion relationship between the first distance and a first physical distance; the first physical distance being a physical distance corresponding to the first distance;
[0043] The first processing unit is used to obtain a second distance based on the first distance and the first transmission parameter; the second distance is negatively correlated with the scale of the first point in the image to be processed, and / or the second distance is negatively correlated with the scale of the second point in the image to be processed.
[0044] In combination with any embodiment of the present application, the acquiring unit is configured to:
[0045] Obtaining a second transmission parameter of a first pixel; the first pixel is a pixel on a line connecting the first point and the second point; the second transmission parameter represents a conversion relationship between a size of the first pixel and a size of a first object point; the first object point is an object point corresponding to the first pixel; the first ratio is negatively correlated with a scale of the first pixel in the image to be processed; the first ratio is the ratio between the size of the first pixel and the size of the first object point;
[0046] The first transmission parameter is obtained according to the second transmission parameter; the first transmission parameter is positively correlated with the second transmission parameter.
[0047] In combination with any embodiment of the present application, the first pixel point is the midpoint between the first point and the second point, and the first transmission parameter is obtained based on the second transmission parameter.
[0048] In combination with any embodiment of the present application, the acquiring unit is configured to:
[0049] Performing object detection processing on the image to be processed to obtain the position of a first object frame and the position of a second object frame; the first object frame contains a first object; and the second object frame contains a second object;
[0050] Obtaining a first size of the first object according to the position of the first object frame, and obtaining a second size of the second object according to the position of the second object frame;
[0051] A third transmission parameter is obtained based on the first size and the third size, and a fourth transmission parameter is obtained based on the second size and the fourth size; the third size is the physical size of the first object; the third transmission parameter represents a conversion relationship between a fifth size and a sixth size; the fifth size is the size of a second pixel; the position of the second pixel in the image to be processed is determined based on the position of the first object frame; the sixth size is the size of the object point corresponding to the second pixel; the fourth size is the physical size of the second object; the fourth transmission parameter represents a conversion relationship between a seventh size and an eighth size; the seventh size is the size of a third pixel; the position of the third pixel in the image to be processed is determined based on the position of the second object frame; the eighth size is the size of the object point corresponding to the third pixel;
[0052] Performing curve fitting processing on the third transmission parameter and the fourth transmission parameter to obtain a transmission parameter map of the image to be processed; a conversion relationship between the ninth dimension and the tenth dimension is determined based on a first pixel value in the transmission parameter map; the ninth dimension is the dimension of a fourth pixel in the image to be processed; the tenth dimension is the dimension of an object point corresponding to the fourth pixel; the first pixel value is the pixel value of a fifth pixel; and the fifth pixel is the pixel corresponding to the fourth pixel in the transmission parameter map;
[0053] The second transmission parameter is obtained according to the pixel value corresponding to the first pixel point in the transmission parameter map.
[0054] In conjunction with any embodiment of the present application, the acquisition unit is further configured to acquire a confidence map before performing curve fitting processing on the third transmission parameter and the fourth transmission parameter to obtain the transmission parameter map of the image to be processed; the confidence map represents a mapping between object types and confidence levels of transmission parameters;
[0055] The distance measuring device further includes:
[0056] a second processing unit, configured to obtain a first confidence level of the third transmission parameter based on the object type of the first object and the confidence level mapping;
[0057] The acquisition unit is further configured to:
[0058] Obtaining a fifth transmission parameter according to the first confidence level and the third transmission parameter; wherein the fifth transmission parameter is positively correlated with the first confidence level;
[0059] Curve fitting is performed on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter graph.
[0060] In combination with any embodiment of the present application, the distance measuring device further includes:
[0061] a feature extraction processing unit, configured to perform feature extraction processing on a pixel area within the first object frame to obtain feature data before obtaining the first confidence level of the third transmission parameter based on the object type of the first object and the confidence level mapping;
[0062] a third processing unit, configured to obtain a score of the first object based on the feature data; the score is positively correlated with a confidence level of a size of the first object;
[0063] The second processing unit is configured to:
[0064] Obtaining a second confidence level of the third transmission parameter based on the object type of the first object and the confidence level map;
[0065] The first confidence level is obtained based on the score and the second confidence level; the first confidence level is correlated with the score.
[0066] In combination with any embodiment of the present application, the acquiring unit is further configured to:
[0067] A product of the first confidence level and the third transmission parameter is determined to obtain the fifth transmission parameter.
[0068] In combination with any embodiment of the present application, the acquiring unit is configured to acquire a depth image of the image to be processed before performing curve fitting processing on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter map;
[0069] The distance measuring device further includes:
[0070] a fourth processing unit, configured to obtain first depth information of the second pixel and second depth information of the third pixel according to the depth image;
[0071] a fifth processing unit, configured to obtain a first data point according to the first depth information and the fifth transmission parameter, and obtain a second data point according to the second depth information and the fourth transmission parameter;
[0072] The acquisition unit is configured to:
[0073] Curve fitting is performed on the first data point and the second data point to obtain the transmission parameter map.
[0074] In combination with any embodiment of the present application, the first point and the second point are both character points; and the acquiring unit is further configured to:
[0075] When the second distance does not exceed a safety distance threshold, obtaining the position of the first point;
[0076] The distance measuring device further includes: a sending unit, configured to send an alarm instruction including the position to a terminal; the alarm instruction is configured to instruct the terminal to output alarm information indicating that the distance between people is too close.
[0077] In a third aspect, a processor is provided, which is used to execute the method as described in the first aspect and any possible implementation thereof.
[0078] In a fourth aspect, an electronic device is provided, comprising: a processor, a sending device, an input device, an output device and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation method thereof.
[0079] In a fifth aspect, a computer-readable storage medium is provided, in which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method as described in the first aspect above and any possible implementation method thereof.
[0080] In a sixth aspect, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is enabled to execute the method of the above-mentioned first aspect and any possible implementation thereof.
[0081] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.
[0083] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.
[0084] Figure 1 A schematic diagram of an image scale provided in an embodiment of the present application;
[0085] Figure 2 A schematic diagram of a pixel coordinate system provided in an embodiment of the present application;
[0086] Figure 3 A flow chart of a distance measurement method provided in an embodiment of the present application;
[0087] Figure 4 A flowchart of another ranging method provided in an embodiment of the present application;
[0088] Figure 5 A schematic diagram of a goal provided in an embodiment of the present application;
[0089] Figure 6 A schematic structural diagram of a distance measuring device provided in an embodiment of the present application;
[0090] Figure 7 A schematic diagram of the hardware structure of a distance measuring device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0091] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0092] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.
[0093] It should be understood that, in this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist, where A and B can be singular or plural. The character " / " can indicate that the associated objects are in an "or" relationship, referring to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or plural. The character " / " can also represent the division sign in mathematical operations, for example, a / b = a divided by b; 6 / 3 = 2. "At least one of the following" or similar expressions.
[0094] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0095] First, some concepts that will appear below are defined. In the embodiments of this application, object points refer to points in the real world, physical distances refer to distances in the real world, and physical sizes refer to sizes in the real world.
[0096] Object points correspond to pixels in an image. For example, consider image A, where a camera captures a table. The table includes object point a, and pixel point b in image A is imaged from object point a. Therefore, object point a corresponds to pixel point b.
[0097] In the embodiments of the present application, nearby objects appear large in the image, while distant objects appear small. In the embodiments of the present application, "far" refers to the distance between the real object corresponding to the object in the image and the imaging device that captured the image, and "near" refers to the distance between the real object corresponding to the object in the image and the imaging device that captured the image.
[0098] In an image, the scale of a pixel is positively correlated with the size of the object point corresponding to that pixel. Specifically, the larger the scale of a pixel in the image, the larger the size of the object point corresponding to that pixel. For example, image A contains pixel a and pixel b, where the object point corresponding to pixel a is object point 1, and the object point corresponding to pixel b is object point 2. If the scale of pixel a in image A is larger than the scale of pixel b in image A, then the size of object point 1 is larger than the size of object point 2.
[0099] In an image, the scale of a location refers to the ratio between the size of an object at that location and the physical size of the object. Figure 1 In the figure, since the scale of the position where person A is located is larger than that of person B, and the size difference between people is small (that is, the difference between the physical sizes of different people is small), the area of the pixel area covered by person A is larger than the area of the pixel area covered by person B.
[0100] In the embodiment of the present application, the position in the image refers to the position in the pixel coordinates of the image. The horizontal coordinate of the pixel coordinate system in the embodiment of the present application is used to represent the column number where the pixel point is located, and the vertical coordinate of the pixel coordinate system is used to represent the row number where the pixel point is located. For example, in Figure 2 In the image shown, the upper left corner of the image is the coordinate origin O, the direction parallel to the rows of the image is the direction of the X axis, and the direction parallel to the columns of the image is the direction of the Y axis. The pixel coordinate system is XOY. The units of the horizontal and vertical coordinates are both pixels. For example, Figure 2 Pixel A in 11 The coordinates are (1, 1), pixel A 23 The coordinates are (3, 2), pixel A 42 The coordinates are (2, 4), pixel A 34The coordinates are (4, 3).
[0101] The embodiment of the present application is executed by a distance measuring device. Optionally, the distance measuring device can be one of the following: a mobile phone, a computer, a server, or a tablet computer. The embodiment of the present application is described below in conjunction with the accompanying drawings.
[0102] The following describes the embodiments of the present application in conjunction with the accompanying drawings. Figure 3 , Figure 3 This is a flow chart of a distance measurement method provided in an embodiment of the present application.
[0103] 301. Obtain a first distance in an image to be processed and a first transmission parameter of the first distance.
[0104] In the embodiments of the present application, the image to be processed may be any image. For example, the image to be processed may include a person. The image to be processed may include only a human head, without a torso or limbs (hereinafter referred to as the torso and limbs as the human body). The image to be processed may also include only a human body, without a head. The image to be processed may also include only lower limbs or upper limbs. The present application does not limit the specific human body area included in the image to be processed. For another example, the image to be processed may include an animal. For another example, the image to be processed may include parts. The present application does not limit the content included in the image to be processed.
[0105] In the embodiment of the present application, the first distance is the distance between a first point and a second point, where both the first point and the second point belong to the image to be processed. The first point can be a pixel in the image to be processed, or a pixel region in the image to be processed, where the pixel region includes at least one pixel. Similarly, the second point can be a pixel in the image to be processed, or the first point can be a pixel region in the image to be processed, where the pixel region includes at least one pixel.
[0106] In the embodiments of the present application, the first transmission parameter represents the conversion relationship between the first distance and the first physical distance, where the first physical distance is the physical distance corresponding to the first distance. For example, the image to be processed includes Zhang San and Li Si, where the first distance is the distance between Zhang San and Li Si in the image. In this case, the first physical distance is the distance between Zhang San and Li Si in the real world. The distance measuring device can convert the first distance into the physical distance corresponding to the first distance based on the first transmission parameter.
[0107] In one implementation of obtaining a first distance in an image to be processed, a distance measuring device receives the first distance input by a user through an input component, which may include a keyboard, a mouse, a touch screen, a touch pad, an audio input device, and the like.
[0108] In another implementation of obtaining the first distance in the image to be processed, the distance measuring device receives the first distance sent by the first terminal. Optionally, the first terminal can be any of the following: a mobile phone, a computer, a tablet computer, a server, or a wearable device.
[0109] In one implementation of obtaining a first transmission parameter of a first distance, a distance measuring device receives the first transmission parameter input by a user through an input component, which includes a keyboard, a mouse, a touch screen, a touchpad, an audio input device, and the like.
[0110] In another implementation of obtaining a first transmission parameter for a first distance, a distance measuring device receives the first transmission parameter sent by a second terminal. Optionally, the second terminal can be any of the following: a mobile phone, a computer, a tablet computer, a server, or a wearable device. The second terminal can be the same as or different from the first terminal.
[0111] 302. Obtain a second distance based on the first distance and the first transmission parameter.
[0112] In the embodiment of the present application, the second distance is negatively correlated with the scale of the first point in the image to be processed, and / or the second distance is negatively correlated with the scale of the second point in the image to be processed. That is, when the first distance is constant, the larger the scale of the first point, the smaller the second distance; and when the first distance is constant, the larger the scale of the second point, the smaller the second distance.
[0113] In one possible implementation, the vertical coordinate of a pixel can be used to represent the scale of the pixel. Specifically, the larger the vertical coordinate of the pixel, the larger the scale of the pixel. In this case, the second distance is negatively correlated with the vertical coordinate of the first point, and / or the second distance is negatively correlated with the vertical coordinate of the second point.
[0114] For example, assuming the first distance is 3 and the coordinate of the second point is p1, if the second point's position is p1 and the first point's ordinate is 5, the second distance is 80 centimeters. If the second point's position is p1 and the first point's ordinate is 10, the second distance is 60 centimeters.
[0115] For another example, assuming the first distance is 3 and the coordinate of the first point is p2, if the first point is at p2 and the ordinate of the second point is 6, the second distance is 70 centimeters. If the first point is at p2 and the ordinate of the second point is 10, the second distance is 40 centimeters.
[0116] As described above, the second distance is obtained based on the first distance and the first transmission parameter. That is, when the first distance remains unchanged, the magnitude of the second distance depends on the first transmission parameter. When the first distance remains unchanged, the second distance is negatively correlated with the scale of the first point, and / or the second distance is negatively correlated with the scale of the second point. That is, in embodiments of the present application, the first transmission parameter carries scale information of the first distance. The scale information of the first distance includes at least one of the following: scale information of the first point, scale information of the second point, and scale information of any point on the line connecting the first and second points.
[0117] In the embodiment of the present application, since the first transmission parameter carries the scale information of the first distance, the distance measuring device determines the second distance based on the first transmission parameter and the first distance, which can improve the accuracy of the second distance.
[0118] As an optional implementation manner, the ranging device obtains the first transmission parameter by performing the following steps:
[0119] 1. Obtain the second transmission parameter of the first pixel.
[0120] In the embodiment of the present application, the first pixel point is a pixel point on the line connecting the first point and the second point. The second transmission parameter represents the conversion relationship between the size of the first pixel point and the size of the first object point, wherein the first object point is the object point corresponding to the first pixel point. For example, the second transmission parameter represents the conversion relationship between the length of the first pixel point and the length of the first object point. For another example, the second transmission parameter represents the conversion relationship between the height of the first pixel point and the height of the first object point. For another example, the second transmission parameter represents the conversion relationship between the width of the first pixel point and the width of the first object point.
[0121] The ratio between the size of the first pixel and the size of the first object point is referred to as the first ratio. In an embodiment of the present application, the first ratio is negatively correlated with the scale of the first pixel in the image to be processed. For example, assuming that the first ratio is the ratio of the length of the first pixel to the first object point, the larger the scale of the first pixel in the image to be processed (hereinafter referred to as the scale of the first pixel), the smaller the first ratio. Considering that the lengths of any two pixels in the image to be processed are the same, that is, the length of the first pixel is constant, then the larger the scale of the first pixel, the smaller the length of the first object point, that is, the size of the first object point is negatively correlated with the scale of the first pixel.
[0122] From the above, it can be seen that since the size of the first object point is obtained according to the size of the first pixel point and the second transmission parameter, and the size of the first pixel point is a constant, the second transmission parameter carries the scale information of the first pixel point.
[0123] In one implementation of obtaining a second transmission parameter of a first distance, a distance measuring device receives the second transmission parameter input by a user through an input component, which may include a keyboard, a mouse, a touch screen, a touchpad, an audio input device, and the like.
[0124] In another implementation of obtaining a second transmission parameter for a first distance, a distance measuring device receives the second transmission parameter sent by a second terminal. Optionally, the second terminal can be any of the following: a mobile phone, a computer, a tablet computer, a server, or a wearable device. The second terminal can be the same as or different from the first terminal.
[0125] 2. Based on the second transmission parameter, obtain the first transmission parameter.
[0126] In one possible implementation, the scale of a pixel is linearly correlated with its abscissa, and / or the scale of a pixel is linearly correlated with its ordinate. The scale of a third pixel is linearly correlated with the scale of the first point, and / or the scale of the third pixel is linearly correlated with the scale of the second point. Therefore, the ranging device can determine the first transmission parameter based on the second transmission parameter. As an optional embodiment, the first pixel is the midpoint between the first and second points.
[0127] In the embodiment of the present application, the first transmission parameter is positively correlated with the second transmission parameter. Assuming that the first transmission parameter is b1 and the second transmission parameter is b2, in one possible implementation, b1 and b2 satisfy the following formula:
[0128] b2=k×b1…Formula (1)
[0129] Wherein, k is a positive number. Optionally, k=1.
[0130] In another possible implementation, b1 and b2 satisfy the following formula:
[0131] b2=k×b1+c…Formula (2)
[0132] Where k is a positive number and c is a real number. Optionally, k=1 and c=0.
[0133] In another possible implementation, b1 and b2 satisfy the following formula:
[0134]
[0135] Where k is a positive number and c is a real number. Optionally, k=1 and c=0.
[0136] See also Figure 4 , Figure 4 This is a flowchart of a possible implementation method of step 1 provided in an embodiment of the present application.
[0137] 401 : Perform object detection on the image to be processed to obtain the position of a first object frame and the position of a second object frame.
[0138] In the embodiment of the present application, the object detection process is an object whose size is close to a certain value. For example, the average length of a human face is 20 centimeters, and the object detection process can be a human face. For another example, the average height of a person is 1.65 meters, and the object detection process can be a human body. For another example, in a football field, Figure 5 The heights of the goals shown are all fixed (eg, 2.44 meters), and the detection object of the object detection process may be the goals.
[0139] In the embodiment of the present application, the object frame can be of any shape, and the present application does not limit the shape of the object frame (including the first object frame and the second object frame mentioned above). Optionally, the shape of the object frame includes at least one of the following: rectangle, diamond, circle, ellipse, polygon.
[0140] In an embodiment of the present application, the position of the object frame (including the position of the above-mentioned first object frame and the position of the above-mentioned second object frame) is used to determine the pixel area contained in the object frame, that is, the position of the object frame in the image to be processed. For example, when the shape of the object frame is a rectangle, the position of the object frame may include the coordinates of any pair of diagonal corners in the rectangle, wherein a pair of diagonal corners refers to two vertices on the diagonal line of the rectangle. For another example, when the shape of the object frame is a rectangle, the position of the object frame may include: the position of the geometric center of the rectangle, the length of the rectangle, and the width of the rectangle. For another example, when the shape of the body frame is a circle, the position of the object frame may include: the position of the center of the object frame and the radius of the object frame.
[0141] In an embodiment of the present application, the number of detection objects processed by the object detection process is not less than 1. For example, when the detection object is a human face, the position of the face frame containing the human face can be obtained by performing object detection processing on the image to be processed. For another example, when the detection object includes a human face and a human body, the position of the face frame containing the human face and the position of the human body frame containing the human body can be obtained by performing object detection processing on the image to be processed. For another example, when the detection object includes a human face, a human body and a screw, the position of the face frame containing the human face, the position of the human body frame containing the human body and the position of the screw frame containing the screw can be obtained by performing object detection processing on the image to be processed. Optionally, the detection objects processed by the object detection process include at least one of the following: a human face, a human foot, a human body, a screw, and a goal.
[0142] In one possible implementation, object detection in processed images can be performed using a convolutional neural network. The convolutional neural network is trained using labeled images as training data, enabling the trained convolutional neural network to detect objects in images. The labeled images in the training data contain the location information of object boxes, which contain the objects to be detected.
[0143] In another possible implementation, object detection processing can be implemented by an object detection algorithm, wherein the object detection algorithm can be one of the following: you only look once (YOLO) algorithm, target detection algorithm (deformable part model, DMP), single image multi-target detection algorithm (single shot multiBox detector, SSD), Faster-RCNN algorithm, etc. This application does not limit the object detection algorithm for implementing object detection processing.
[0144] The ranging device performs object detection on the image to be processed, obtaining the position of a first object frame containing a first object and the position of a second object frame containing a second object. The detection object contained in the first object frame is different from the detection object contained in the second object frame. For example, the detection object contained in the first object frame is Zhang San's face, while the detection object contained in the second object frame is Li Si's face. For another example, the detection object contained in the first object frame is Zhang San's face, while the detection object contained in the second object frame is a sign.
[0145] 402. Obtain a first size of the first object based on the position of the first object frame, and obtain a second size of the second object based on the position of the second object frame.
[0146] The distance measuring device can determine the size of the detection object contained within the object frame based on the position of the object frame. For example, if the object frame is rectangular, the distance measuring device can determine the length and width of the object frame based on the position of the object frame, and thus determine the length and width of the detection object within the object frame.
[0147] The distance measuring device can obtain a first size of the first object according to the position of the first object frame, and obtain a second size of the second object according to the position of the second object frame.
[0148] 403. Obtain a third transmission parameter based on the first size and the third size, and obtain a fourth transmission parameter based on the second size and the fourth size.
[0149] In this embodiment of the present application, the third dimension is the physical dimension of the first object, and the fourth dimension is the physical dimension of the second object. For example, if the detection object contained in the first object frame is a human body, the third dimension can be the height of the person (e.g., 170 cm). For another example, if the detection object contained in the second object frame is a human face, the third dimension can be the length of the face (e.g., 20 cm).
[0150] The ranging device can determine a pixel point (i.e., a third pixel point) in the image to be processed based on the position of the first object frame. For example, if the shape of the first object frame is rectangular, the ranging device determines the position of the geometric center of the first object frame based on the position of the first object frame, and uses the pixel point corresponding to the geometric center as the third pixel point. For another example, if the shape of the first object frame is rectangular, the ranging device determines the position of any vertex of the first object frame based on the position of the first object frame, and uses the pixel point corresponding to the vertex as the third pixel point. For another example, if the shape of the first object frame is circular, the ranging device determines the position of the center point of the first object frame based on the position of the first object frame, and uses the pixel point corresponding to the center point as the third pixel point. Similarly, the ranging device can determine a pixel point, i.e., a fourth pixel point, in the image to be processed based on the position of the second object frame.
[0151] In the embodiments of the present application, the size of the third pixel is referred to as the fifth size, the size of the object point corresponding to the third pixel is referred to as the sixth size, the size of the fourth pixel is referred to as the seventh size, and the size of the object point corresponding to the fourth pixel is referred to as the eighth size. The conversion relationship between the fifth size and the sixth size is referred to as the third transmission parameter, and the conversion relationship between the seventh size and the eighth size is referred to as the fourth transmission parameter.
[0152] The ranging device can obtain a third transmission parameter based on the first and third dimensions, and can obtain a fourth transmission parameter based on the second and fourth dimensions. Assume that the first dimension is s1, the second dimension is s2, the third dimension is s3, and the fourth dimension is s4, the third transmission parameter is b3, and the fourth transmission parameter is b4.
[0153] In one possible implementation, s1, s3, and b3 satisfy the following formula:
[0154] b3=r×s1 / s3…Formula (4)
[0155] s2, s4, b4 satisfy the following formula:
[0156] b4=r×s2 / s4…Formula (5)
[0157] Where r is a positive number. Optionally, r=1.
[0158] In another possible implementation, s1, s3, and b3 satisfy the following formula:
[0159] b3=r×s1 / s3+c…Formula (6)
[0160] s2, s4, b4 satisfy the following formula:
[0161] b4=r×s2 / s4+c…Formula (7)
[0162] Where r is a positive number and c is a real number. Optionally, r=1 and c=0.
[0163] In another possible implementation, s1, s3, and b3 satisfy the following formula:
[0164]
[0165] s2, s4, b4 satisfy the following formula:
[0166]
[0167] Where r is a positive number and c is a real number. Optionally, r=1 and c=0.
[0168] 404 . Perform curve fitting processing on the third transmission parameter and the fourth transmission parameter to obtain a transmission parameter graph of the image to be processed.
[0169] Because the scale of a pixel in the image to be processed is linearly related to its abscissa and / or its ordinate, the distance measuring device can obtain a transmission parameter map for the image to be processed by curve fitting the third and fourth transmission parameters. The transmission parameter of any pixel in the image to be processed can be determined based on the pixel values in the transmission parameter map.
[0170] Take the sixth pixel in the transmission parameter map as an example. Assuming the sixth pixel's value is the first pixel value, and its position in the transmission parameter map is the same as the position of the fifth pixel in the image to be processed, the ranging device can determine the conversion relationship between the size of the fifth pixel (i.e., the ninth dimension) and the tenth dimension based on the first pixel value. The tenth dimension is the size of the object point corresponding to the fifth pixel.
[0171] Assume that the first pixel value is p1, the ninth size is s5, and the tenth size is s6. In one possible implementation, p1, s5, and s6 satisfy the following formula:
[0172] p1=u×s5 / s6…Formula (10)
[0173] Where u is a positive number. Optionally, u=1.
[0174] In another possible implementation, p1, s5, and s6 satisfy the following formula:
[0175] p1=u×s5 / s6+y…Formula (11)
[0176] Where u is a positive number and y is a real number. Optionally, u=1 and y=0.
[0177] In another possible implementation, p1, s5, and s6 satisfy the following formula:
[0178]
[0179] Where u is a positive number and y is a real number. Optionally, u=1 and y=0.
[0180] Similarly, the distance measuring device can determine the transmission parameter of any pixel point except the sixth pixel point in the image to be processed according to the transmission parameter map.
[0181] 405 . Obtain the second transmission parameter according to the transmission parameter map and the position of the first pixel in the image to be processed.
[0182] The distance measuring device may determine a reference pixel value from the transmission parameter map based on the position of the first pixel in the image to be processed, wherein the position of the pixel corresponding to the reference pixel value in the transmission parameter map is the same as the position of the first pixel in the image to be processed. The distance measuring device may then obtain a second transmission parameter based on the reference pixel value.
[0183] In this embodiment of the present application, the ranging device obtains a third transmission parameter based on the first and third dimensions, and obtains a fourth transmission parameter based on the second and fourth dimensions. Curve fitting is performed on the third and fourth transmission parameters to generate a transmission parameter map, which can then be used to determine the transmission parameter of any pixel in the image to be processed.
[0184] As an optional implementation, before executing step 404, the ranging device further executes the following steps:
[0185] 3. Get the confidence map.
[0186] In the embodiment of the present application, the accuracy of the transmission parameter of a pixel point is positively correlated with the accuracy of the size of the object point corresponding to the pixel point. Accordingly, the accuracy of the transmission parameter map is positively correlated with the accuracy of the size of the first object and the size of the second object.
[0187] Obviously, the accuracy of the size of an object with a fixed size is higher than the accuracy of the size of an object with a floating size.
[0188] For example, a standard soccer goal is 7.32 meters wide and 2.44 meters high. 90% of people are between 1.4 and 2 meters tall. The accuracy of a soccer goal's dimensions is higher than the accuracy of a person's height.
[0189] For example, the height of a standard basketball stand is 3.05 meters. 95% of human faces are between 17 centimeters and 30 centimeters long. The accuracy of the height of a basketball stand is higher than the accuracy of the length of a human face.
[0190] For another example, a screw with a fixed length has a length between 20 cm and 35 cm, and the length accuracy of a screw with a fixed length is higher than that of a human foot.
[0191] Optionally, the aforementioned object with a fixed size may be an object with a fixed size in a specific scenario, such as a boarding sign in a terminal, a chair in a gymnasium, or a desk in an office.
[0192] In the embodiment of the present application, the confidence map represents the mapping between the object type and the confidence of the transmission parameter. For example, the confidence map can be seen in Table 1.
[0193] Object Type Confidence Goals, basketball hoops, boarding signs 0.9 human body 0.8 Face 0.7 human feet 0.65
[0194] Table 1
[0195] In one implementation of obtaining a confidence map, a distance measuring device receives a confidence map input by a user via an input component, which may include a keyboard, a mouse, a touch screen, a touchpad, an audio input device, and the like.
[0196] In another implementation of obtaining a confidence map, the ranging device receives the confidence map sent by a third terminal. Optionally, the third terminal can be any of the following: a mobile phone, a computer, a tablet computer, a server, or a wearable device. The third terminal can be the same as or different from the first terminal.
[0197] 4. Obtain a first confidence level of the third transmission parameter based on the object type of the first object and the confidence level mapping.
[0198] After the ranging device obtains the confidence map, a first confidence level for the third transmission parameter can be obtained based on the confidence map and the object type of the first object. For example, assuming the confidence map is as shown in Table 1 above, and the object type of the first object is a human body, the first confidence level is 0.9.
[0199] Optionally, the distance measuring device may determine the object type of the first object by performing feature extraction processing on a pixel area included in the first object frame.
[0200] As an optional real-time method, the ranging device can determine the transmission parameter corresponding to each object within the object frame based on the object type of each object. For example, the ranging device can obtain the confidence level of the fourth transmission parameter (hereinafter referred to as the third confidence level) based on the object type and confidence level mapping of the second object.
[0201] After obtaining the first confidence level, the ranging device performs the following steps during step 404:
[0202] 5. Obtain a fifth transmission parameter based on the first confidence level and the third transmission parameter.
[0203] In the embodiment of the present application, the fifth transmission parameter is positively correlated with the first confidence level. Assume that the first confidence level is c1 and the fifth transmission parameter is b5. In one possible implementation, c1 and b5 satisfy the following formula:
[0204] b5=a×c1…Formula (13)
[0205] Where a is a positive number. Optionally, a=1.
[0206] In another possible implementation, c1 and b5 satisfy the following formula:
[0207] b5=a×c1+e…Formula (14)
[0208] Where a is a positive number and e is a real number. Optionally, a=1 and e=0.
[0209] In another possible implementation, c1 and b5 satisfy the following formula:
[0210]
[0211] Where a is a positive number and e is a real number. Optionally, a=1 and e=0.
[0212] 6. Perform curve fitting processing on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter graph.
[0213] The distance measuring device can improve the accuracy of the transmission parameter map by performing curve fitting processing on the fourth transmission parameter and the fifth transmission parameter.
[0214] As an optional implementation, when the ranging device obtains the third confidence level by executing step 4 and obtains the sixth transmission parameter based on the third confidence level and the fourth transmission parameter, the ranging device can obtain a transmission parameter map by performing curve fitting on the fifth transmission parameter and the sixth transmission parameter.
[0215] As an optional implementation, before executing step 4, the ranging device further executes the following steps:
[0216] 7. Perform feature extraction processing on the pixel area within the first object frame to obtain feature data.
[0217] In the embodiments of the present application, the feature extraction process may be a convolution process, a pooling process, or a combination of convolution and pooling processes. Optionally, the feature extraction process may be implemented by a trained convolutional neural network or a feature extraction model, which is not limited in this application.
[0218] The distance measuring device can extract semantic information from the pixel area within the object frame by performing feature extraction processing on the pixel area within the object frame, and obtain feature data of the object frame.
[0219] In an implementation method for performing feature extraction processing on a pixel region within a first object frame, convolution processing is performed layer by layer on the pixel region within the first object frame through at least two convolutional layers to complete the feature extraction processing on the pixel region within the first object frame. The convolutional layers in the at least two convolutional layers are connected in series, that is, the output of the previous convolutional layer is the input of the next convolutional layer, and the semantic information extracted by each convolutional layer is different. Specifically, the feature extraction processing abstracts the features of the pixel region within the first object frame step by step, and also gradually discards relatively minor feature data, wherein relatively minor feature information refers to feature information other than feature information that can be used to determine the object type of the object within the first object frame. Therefore, the size of the feature data extracted later is smaller, but the semantic information is more concentrated. By performing convolution processing on the pixel region within the first object frame step by step through multiple convolutional layers, the semantic information of the pixel region within the first object frame can be obtained.
[0220] Optionally, the distance measuring device may perform feature extraction processing on the pixel point area within each object frame respectively to obtain feature data of the pixel point area within each object frame.
[0221] 8. Based on the above feature data, obtain the score of the above first object.
[0222] Taking into account that the actual size of an object that does not have a fixed size may change, in an embodiment of the present application, the state of the object is determined based on the characteristic data of the object, and then a confidence score for characterizing the size of the object is obtained, wherein the score of the object is positively correlated with the confidence of the size of the object.
[0223] For example, assuming the object is a human body and the size of the object is the height of the person. When the person is standing upright, the person's height is equal to the person's true height, and in this case, the confidence level of the person's height is the highest. When the person is walking, there is a small error between the person's height and the person's true height, and in this case, the confidence level of the person's height is second. When the person is lowering their head (such as looking down at a mobile phone), there is a small error between the person's height and the person's true height, and in this case, the confidence level of the person's height is lower than that of the person's height when walking. When the person is sitting, there is a large error between the person's height and the person's true height, and in this case, the confidence level of the person's height is lower.
[0224] In the embodiment of the present application, the distance measuring device can determine the score of the object in the object frame based on the feature data extracted from the pixel point area in the object frame.
[0225] As an optional implementation, the distance measuring device may use a classifier (such as a support vector machine, a softmax function) to process the feature data of the object frame to obtain a score of the object in the object frame.
[0226] Optionally, the ranging device can use a neural network to process the pixel area within the object frame to obtain the score of the object within the object frame. For example, the ranging device uses a labeled image set as training data to train the neural network to obtain a trained neural network. The trained neural network is used to process an unlabeled image set to obtain a label for the unlabeled image set. The trained neural network is trained using the labeled image set, the unlabeled image set, and the labels of the unlabeled image set to obtain an image processing neural network. The information carried by the label includes the location of the object frame in the image and the score of the object within the object frame.
[0227] The distance measuring device can obtain a score of the first object based on the feature data of the first object. Optionally, the distance measuring device can obtain a score for each object in the image to be processed.
[0228] When the score of the first object is obtained, the distance measuring device performs the following steps during step 4:
[0229] 9. Obtain a second confidence level of the third transmission parameter based on the object type of the first object and the confidence level mapping.
[0230] The implementation process of this step can be referred to step 4, but in this step, the ranging device obtains not the first confidence but the second confidence based on the object type and confidence mapping of the first object.
[0231] 10. Based on the above score and the above second confidence level, obtain the above first confidence level.
[0232] In the embodiment of the present application, the first confidence level is correlated with the score. Assume that the first confidence level is c1, the second confidence level is c2, and the score is s. In one possible implementation, c1, c2, and s satisfy the following formula:
[0233] c2=α×s×c1…Formula (16)
[0234] Wherein, α is a positive number. Optionally, α=1.
[0235] In another possible implementation, c1, c2, and s satisfy the following formula:
[0236] c2=α×s×c1+σ…Formula (17)
[0237] Where α is a positive number and σ is a real number. Optionally, α=1 and σ=0.
[0238] In another possible implementation, c1, c2, and s satisfy the following formula:
[0239]
[0240] Where α is a positive number and σ is a real number. Optionally, α=1 and σ=0.
[0241] The distance measuring device obtains the first confidence level according to the score of the first object and the second confidence level, thereby improving the accuracy of the first confidence level.
[0242] Optionally, the ranging device may obtain a fourth confidence level of the fourth transmission parameter by executing step 9, and the ranging device may obtain the third confidence level according to the score of the second object and the fourth confidence level.
[0243] As an optional implementation, before executing step 6, the ranging device further executes the following steps:
[0244] 11. Obtain a depth image of the image to be processed.
[0245] In an embodiment of the present application, a depth image of an image to be processed carries depth information for pixels in the image to be processed. In one possible implementation, a ranging device receives a depth image input by a user via an input component. Such input components include a keyboard, mouse, touch screen, touchpad, and audio input device.
[0246] In another possible implementation, the ranging device is equipped with an RGB camera and a depth camera. While the ranging device uses the RGB camera to capture the image to be processed, it uses the depth camera to capture a depth image of the image to be processed. The depth camera can be any of the following: a structured light camera, a time-of-flight camera, or a binocular stereo vision camera.
[0247] In another possible implementation, the distance measuring device receives a depth image sent by a fourth terminal, where the fourth terminal includes a mobile phone, a computer, a tablet computer, a server, etc. In this embodiment, the fourth terminal may be the same as or different from the first terminal.
[0248] 12. Obtain first depth information of the third pixel and second depth information of the fourth pixel based on the depth image.
[0249] As described above, the depth image carries depth information of pixels in the image to be processed. After acquiring the depth image, the ranging device can determine the depth information of the third pixel (i.e., the first depth information) and the depth information of the fourth pixel (i.e., the second depth information) based on the depth image.
[0250] 13. Obtain a first data point based on the first depth information and the fifth transmission parameter, and obtain a second data point based on the second depth information and the fourth transmission parameter.
[0251] In one possible implementation, the abscissa of the first data point is the first depth information, the abscissa of the second data point is the second depth information, the ordinate of the first data point is the fifth transmission parameter, and the ordinate of the second data point is the fourth transmission parameter. That is, the ranging device uses the depth information of the pixel point as the abscissa and the transmission parameter of the pixel point as the ordinate.
[0252] In one possible implementation, the ordinate of the first data point is the first depth information, the ordinate of the second data point is the second depth information, the abscissa of the first data point is the fifth transmission parameter, and the abscissa of the second data point is the fourth transmission parameter. That is, the ranging device uses the depth information of the pixel point as the ordinate and the transmission parameter of the pixel point as the abscissa.
[0253] After obtaining the first data point and the second data point, the distance measuring device performs the following steps during step 6:
[0254] 14. Perform curve fitting processing on the first data point and the second data point to obtain the transmission parameter graph.
[0255] Since both the first data point and the second data point carry depth information of the pixel point, the transmission parameter map obtained by the distance measuring device by performing curve fitting processing on the first data point and the second data point also carries depth information.
[0256] Because the scale of the pixel in the image to be processed is determined based on the depth information of the pixel, the accuracy of the scale of the pixel in the image to be processed can be improved, so the ranging device obtains the transmission parameter map by executing step 14, which can improve the accuracy of the transmission parameter map, and further improve the accuracy of the transmission parameters of the pixel in the image to be processed, thereby improving the accuracy of the second distance.
[0257] Based on the technical solution provided in the embodiment of the present application, the embodiment of the present application also provides a possible application scenario. When a respiratory infectious disease breaks out, in order to reduce the infection rate and the number of infected people, people should keep a distance from each other.
[0258] Based on the technical solution provided in the embodiments of the present application, the distance between people can be determined by processing images. If the distance does not exceed the safe distance, a prompt can be issued to the relevant management personnel to enable the relevant management personnel to guide the crowd.
[0259] As an optional embodiment, the first point and the second point are both character points. In the embodiment of the present application, the pixel area covered by the character point can be regarded as a character area, wherein the character area is the pixel area covered by the human body. For example, the area covered by the first point belongs to the pixel area covered by the head. For another example, the area covered by the first point belongs to the pixel area covered by the arm. For another example, the area covered by the first point belongs to the pixel area covered by the torso.
[0260] The second distance does not exceed the safety distance threshold, indicating that the distance between the person corresponding to the first point (hereinafter referred to as the first person) and the person corresponding to the second point (hereinafter referred to as the second person) does not exceed the safety distance threshold. At this point, the ranging device can obtain the location of the first point and send an alert instruction including the location of the first point (hereinafter referred to as the first location) to the terminal.
[0261] The warning instruction may be a voice prompt, such as "The distance between a person at the first location and another person is too close." The warning instruction may also be a text prompt, such as a pop-up window on the terminal display interface containing the first location and a text prompt, such as "The distance between a person at the first location and another person is too close." This application does not limit the specific form of the warning instruction.
[0262] In the embodiment of the present application, the alarm instruction is used to instruct the terminal to output an alarm message. After receiving the alarm instruction, the terminal outputs the corresponding alarm message to prompt the relevant management personnel to arrive in time to evacuate the crowd.
[0263] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.
[0264] The above describes in detail the method of the embodiment of the present application, and the following provides an apparatus of the embodiment of the present application.
[0265] See also Figure 6 , Figure 6 This is a structural diagram of a distance measuring device provided in an embodiment of the present application, which includes: an acquisition unit 11 and a first processing unit 12.
[0266] An acquisition unit 11 is configured to acquire a first distance in an image to be processed and a first transmission parameter of the first distance; the first distance is the distance between a first point and a second point; both the first point and the second point belong to the image to be processed; the first transmission parameter represents a conversion relationship between the first distance and a first physical distance; the first physical distance is a physical distance corresponding to the first distance;
[0267] The first processing unit 12 is used to obtain a second distance based on the first distance and the first transmission parameter; the second distance is negatively correlated with the scale of the first point in the image to be processed, and / or the second distance is negatively correlated with the scale of the second point in the image to be processed.
[0268] In combination with any embodiment of the present application, the acquiring unit 11 is configured to:
[0269] Obtaining a second transmission parameter of a first pixel; the first pixel is a pixel on a line connecting the first point and the second point; the second transmission parameter represents a conversion relationship between a size of the first pixel and a size of a first object point; the first object point is an object point corresponding to the first pixel; the first ratio is negatively correlated with a scale of the first pixel in the image to be processed; the first ratio is the ratio between the size of the first pixel and the size of the first object point;
[0270] The first transmission parameter is obtained according to the second transmission parameter; the first transmission parameter is positively correlated with the second transmission parameter.
[0271] In combination with any embodiment of the present application, the first pixel point is the midpoint between the first point and the second point, and the first transmission parameter is obtained based on the second transmission parameter.
[0272] In combination with any embodiment of the present application, the acquiring unit 11 is configured to:
[0273] Performing object detection processing on the image to be processed to obtain the position of a first object frame and the position of a second object frame; the first object frame contains a first object; and the second object frame contains a second object;
[0274] Obtaining a first size of the first object according to the position of the first object frame, and obtaining a second size of the second object according to the position of the second object frame;
[0275] A third transmission parameter is obtained based on the first size and the third size, and a fourth transmission parameter is obtained based on the second size and the fourth size; the third size is the physical size of the first object; the third transmission parameter represents a conversion relationship between a fifth size and a sixth size; the fifth size is the size of a second pixel; the position of the second pixel in the image to be processed is determined based on the position of the first object frame; the sixth size is the size of the object point corresponding to the second pixel; the fourth size is the physical size of the second object; the fourth transmission parameter represents a conversion relationship between a seventh size and an eighth size; the seventh size is the size of a third pixel; the position of the third pixel in the image to be processed is determined based on the position of the second object frame; the eighth size is the size of the object point corresponding to the third pixel;
[0276] Performing curve fitting processing on the third transmission parameter and the fourth transmission parameter to obtain a transmission parameter map of the image to be processed; a conversion relationship between the ninth dimension and the tenth dimension is determined based on a first pixel value in the transmission parameter map; the ninth dimension is the dimension of a fourth pixel in the image to be processed; the tenth dimension is the dimension of an object point corresponding to the fourth pixel; the first pixel value is the pixel value of a fifth pixel; and the fifth pixel is the pixel corresponding to the fourth pixel in the transmission parameter map;
[0277] The second transmission parameter is obtained according to the pixel value corresponding to the first pixel point in the transmission parameter map.
[0278] In conjunction with any embodiment of the present application, the acquisition unit 11 is further configured to acquire a confidence map before performing curve fitting processing on the third transmission parameter and the fourth transmission parameter to obtain the transmission parameter map of the image to be processed; the confidence map represents a mapping between object types and confidence levels of transmission parameters;
[0279] The distance measuring device 1 further includes: a second processing unit 13, configured to obtain a first confidence level of the third transmission parameter based on the object type of the first object and the confidence level mapping;
[0280] The acquisition unit 11 is further configured to:
[0281] Obtaining a fifth transmission parameter according to the first confidence level and the third transmission parameter; wherein the fifth transmission parameter is positively correlated with the first confidence level;
[0282] Curve fitting is performed on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter graph.
[0283] In combination with any embodiment of the present application, the distance measuring device 1 further includes:
[0284] a feature extraction processing unit 14 configured to perform feature extraction processing on a pixel area within the first object frame to obtain feature data before obtaining the first confidence level of the third transmission parameter based on the object type of the first object and the confidence level mapping;
[0285] a third processing unit 15 for obtaining a score of the first object based on the feature data; the score is positively correlated with a confidence level of a size of the first object;
[0286] The second processing unit 13 is configured to:
[0287] Obtaining a second confidence level of the third transmission parameter based on the object type of the first object and the confidence level map;
[0288] The first confidence level is obtained based on the score and the second confidence level; the first confidence level is correlated with the score.
[0289] In combination with any embodiment of the present application, the acquiring unit 11 is further configured to:
[0290] A product of the first confidence level and the third transmission parameter is determined to obtain the fifth transmission parameter.
[0291] In combination with any embodiment of the present application, the acquiring unit is configured to acquire a depth image of the image to be processed before performing curve fitting processing on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter map;
[0292] The distance measuring device 1 further includes:
[0293] a fourth processing unit 16, configured to obtain first depth information of the second pixel and second depth information of the third pixel according to the depth image;
[0294] a fifth processing unit 17, configured to obtain a first data point according to the first depth information and the fifth transmission parameter, and obtain a second data point according to the second depth information and the fourth transmission parameter;
[0295] The acquisition unit 11 is configured to:
[0296] Curve fitting is performed on the first data point and the second data point to obtain the transmission parameter map.
[0297] In combination with any embodiment of the present application, the first point and the second point are both character points; and the acquiring unit 11 is further configured to:
[0298] When the second distance does not exceed a safety distance threshold, obtaining the position of the first point;
[0299] The distance measuring device 1 further includes: a sending unit 18, configured to send an alarm instruction including the position to a terminal; the alarm instruction is configured to instruct the terminal to output an alarm message indicating that the distance between people is too close.
[0300] In the embodiment of the present application, since the first transmission parameter carries the scale information of the first distance, the distance measuring device determines the second distance based on the first transmission parameter and the first distance, which can improve the accuracy of the second distance.
[0301] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.
[0302] Figure 7 A schematic diagram of the hardware structure of a distance measuring device provided in an embodiment of the present application. The distance measuring device 2 includes a processor 21, a memory 22, an input device 23, and an output device 24. The processor 21, the memory 22, the input device 23, and the output device 24 are coupled via a connector, which may include various interfaces, transmission lines, or buses, etc., and the embodiments of the present application do not limit this. It should be understood that in the various embodiments of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, connection through various interfaces, transmission lines, buses, etc.
[0303] The processor 21 may be one or more graphics processing units (GPUs). If the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Alternatively, the processor 21 may be a processor group consisting of multiple GPUs, with the multiple processors coupled to each other via one or more buses. Alternatively, the processor may be another type of processor, etc., which is not limited in the present embodiment.
[0304] The memory 22 can be used to store computer program instructions and various computer program codes, including program codes for executing the solution of the present application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable compact disc read-only memory (CD-ROM), which is used for related instructions and data.
[0305] The input device 23 is used to input data and / or signals, and the output device 24 is used to output data and / or signals. The input device 23 and the output device 24 can be independent devices or an integrated device.
[0306] It can be understood that in the embodiment of the present application, the memory 22 can be used not only to store relevant instructions, but also to store relevant data. For example, the memory 22 can be used to store the first distance obtained through the input device 23, or the memory 22 can also be used to store the second distance obtained by the processor 21, etc. The embodiment of the present application does not limit the specific data stored in the memory.
[0307] It is understandable that Figure 7 Only a simplified design of a distance measuring device is shown. In actual applications, the distance measuring device may also include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all distance measuring devices that can implement the embodiments of the present application are within the scope of protection of this application.
[0308] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0309] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here. Those skilled in the art will also clearly understand that the descriptions of the various embodiments of this application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, reference can be made to the descriptions of other embodiments.
[0310] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0311] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0312] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0313] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).
[0314] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by a computer program instructing related hardware to perform the processes. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A distance measurement method, characterized in that: The method comprises: Acquire a first distance in an image to be processed and a first transmission parameter of the first distance; the first distance is the distance between a first point and a second point; the first point and the second point both belong to the image to be processed; the first transmission parameter represents a conversion relationship between the first distance and a first physical distance; the first physical distance is a physical distance corresponding to the first distance; The obtaining a first transmission parameter of the first distance includes: Acquire a second transmission parameter of a first pixel point; the first pixel point is a pixel point on a line connecting the first point and the second point; the second transmission parameter represents a conversion relationship between a size of the first pixel point and a size of a first object point; the first object point is an object point corresponding to the first pixel point; the first ratio is negatively correlated with a scale of the first pixel point in the image to be processed; the first ratio is a ratio between a size of the first pixel point and a size of the first object point; The first transmission parameter is obtained according to the second transmission parameter; the first transmission parameter is positively correlated with the second transmission parameter; A second distance is obtained based on the first distance and the first transmission parameter; the second distance is negatively correlated with the scale of the first point in the image to be processed, and / or the second distance is negatively correlated with the scale of the second point in the image to be processed; the first transmission parameter carries scale information of the first distance, wherein the scale information of the first distance includes at least one of the following: scale information of the first point, scale information of the second point, and scale information of any point on the line connecting the first point and the second point.
2. The method according to claim 1, characterized in that The first pixel point is the midpoint between the first point and the second point, and the first transmission parameter is obtained according to the second transmission parameter.
3. The method according to claim 1 or 2, characterized in that: The obtaining of the second transmission parameter of the first pixel point includes: Performing object detection processing on the image to be processed to obtain the position of a first object frame and the position of a second object frame; the first object frame contains a first object; the second object frame contains a second object; Obtaining a first size of the first object according to the position of the first object frame, and obtaining a second size of the second object according to the position of the second object frame; A third transmission parameter is obtained according to the first size and the third size, and a fourth transmission parameter is obtained according to the second size and the fourth size; the third size is the physical size of the first object; the third transmission parameter represents the conversion relationship between the fifth size and the sixth size; the fifth size is the size of the second pixel; the position of the second pixel in the image to be processed is determined according to the position of the first object frame; the sixth size is the size of the object point corresponding to the second pixel; the fourth size is the physical size of the second object; the fourth transmission parameter represents the conversion relationship between the seventh size and the eighth size; the seventh size is the size of the third pixel; the position of the third pixel in the image to be processed is determined according to the position of the second object frame; the eighth size is the size of the object point corresponding to the third pixel; The third transmission parameter and the fourth transmission parameter are subjected to curve fitting processing to obtain a transmission parameter map of the image to be processed; the conversion relationship between the ninth dimension and the tenth dimension is determined according to the first pixel value in the transmission parameter map; the ninth dimension is the dimension of a fourth pixel point in the image to be processed; the tenth dimension is the dimension of an object point corresponding to the fourth pixel point; the first pixel value is the pixel value of a fifth pixel point; the fifth pixel point is the pixel point in the transmission parameter map corresponding to the fourth pixel point; The second transmission parameter is obtained according to the pixel value corresponding to the first pixel point in the transmission parameter map.
4. The method according to claim 3, characterized in that Before performing curve fitting processing on the third transmission parameter and the fourth transmission parameter to obtain the transmission parameter map of the image to be processed, the method further includes: Acquire a confidence map; the confidence map represents a mapping between object types and confidences of transmission parameters; Obtaining a first confidence level of the third transmission parameter according to the object type of the first object and the confidence level mapping; The performing curve fitting processing on the third transmission parameter and the fourth transmission parameter to obtain a transmission parameter graph of the image to be processed includes: According to the first confidence level and the third transmission parameter, a fifth transmission parameter is obtained; the fifth transmission parameter is positively correlated with the first confidence level; Curve fitting is performed on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter graph.
5. The method according to claim 4, characterized in that Before obtaining the first confidence of the third transmission parameter according to the object type of the first object and the confidence mapping, the method further includes: Performing feature extraction processing on the pixel point area within the first object frame to obtain feature data; Obtaining a score of the first object according to the feature data; the score is positively correlated with the confidence level of the size of the first object; The obtaining, according to the object type of the first object and the confidence mapping, a first confidence of the third transmission parameter comprises: Obtaining a second confidence level of the third transmission parameter according to the object type of the first object and the confidence level mapping; The first confidence level is obtained according to the score and the second confidence level; the first confidence level is correlated with the score.
6. The method according to claim 4 or 5, characterized in that: The obtaining of a fifth transmission parameter according to the first confidence level and the third transmission parameter comprises: The product of the first confidence level and the third transmission parameter is determined to obtain the fifth transmission parameter.
7. The method according to claim 4, characterized in that Before performing curve fitting processing on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter graph, the method further includes: Acquire a depth image of the image to be processed; Obtaining first depth information of the second pixel and second depth information of the third pixel according to the depth image; Obtaining a first data point according to the first depth information and the fifth transmission parameter, and obtaining a second data point according to the second depth information and the fourth transmission parameter; The performing curve fitting processing on the fourth transmission parameter and the fifth transmission parameter to obtain the transmission parameter graph includes: Curve fitting is performed on the first data point and the second data point to obtain the transmission parameter map.
8. The method according to claim 1 or 2, characterized in that: The first point and the second point are both character points; and the method further includes: When the second distance does not exceed a safety distance threshold, obtaining the position of the first point; An alarm instruction including the position is sent to the terminal; the alarm instruction is used to instruct the terminal to output alarm information that the distance between the persons is too close.
9. A distance measuring device, characterized in that: The device comprises: an acquisition unit, configured to acquire a first distance in an image to be processed and a first transmission parameter of the first distance; the first distance is a distance between a first point and a second point; the first point and the second point both belong to the image to be processed; the first transmission parameter represents a conversion relationship between the first distance and a first physical distance; the first physical distance is a physical distance corresponding to the first distance; The obtaining a first transmission parameter of the first distance includes: Acquire a second transmission parameter of a first pixel point; the first pixel point is a pixel point on a line connecting the first point and the second point; the second transmission parameter represents a conversion relationship between a size of the first pixel point and a size of a first object point; the first object point is an object point corresponding to the first pixel point; the first ratio is negatively correlated with a scale of the first pixel point in the image to be processed; the first ratio is a ratio between a size of the first pixel point and a size of the first object point; The first transmission parameter is obtained according to the second transmission parameter; the first transmission parameter is positively correlated with the second transmission parameter; A first processing unit is used to obtain a second distance based on the first distance and the first transmission parameter; the second distance is negatively correlated with the scale of the first point in the image to be processed, and / or the second distance is negatively correlated with the scale of the second point in the image to be processed; the first transmission parameter carries scale information of the first distance, wherein the scale information of the first distance includes at least one of the following: scale information of the first point, scale information of the second point, and scale information of any point on the line connecting the first point and the second point.
10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and when the processor executes the computer instructions, the electronic device executes the method according to any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is enabled to execute the method according to any one of claims 1 to 8.
