A target localization method, apparatus and electronic device

Abstract

A target localization method, apparatus, and electronic device are disclosed. The method includes: determining an initial image frame corresponding to an initial image of a target object acquired by an image acquisition device; determining the completeness of the target object within the initial image frame; determining an initial preset image frame corresponding to a preset completeness consistent with the initial completeness based on a preset correspondence between completeness and preset image frames; determining a first target coordinate point of the initial preset image frame in a first preset coordinate system corresponding to the image acquisition device; converting the first target coordinate point into a second target coordinate point in a spatial Cartesian coordinate system established with reference to the world coordinate system; and using the second target coordinate point as the localization coordinate point of the target object. By converting the first target coordinate point in the first preset coordinate system into a second target coordinate point in a spatial Cartesian coordinate system in the real-time scene, the accuracy of the determined localization coordinate point of the target object is ensured.

Description

Technical Field

[0001] This application relates to the field of target positioning technology, and in particular to a target positioning method, apparatus and electronic device. Background Technology

[0002] With the development of target positioning technology, its application in daily life is becoming increasingly widespread. To promptly determine the accurate location of a target object within a target area, the following method is employed: First, the scaling ratio between the park model and the park site is obtained based on the ratio of the park-level distance to the model-level distance between two reference points in the park site. Second, the distance between the target object and the reference point in the park site is calculated based on the park-level latitude and longitude coordinates of the target object and the reference point. Third, the relative azimuth angle between the target object and the reference point in the park site is calculated based on the park-level latitude and longitude coordinates of the target object and the reference point. Finally, the coordinates of the target object relative to the reference point in the model level are calculated based on the park-level direction and distance of the target object relative to the reference point, as well as the park scaling ratio.

[0003] Based on the above description, determining the coordinates of the target object requires setting a reference point and using latitude and longitude information to locate the target object. When the latitude and longitude information of the target object is inaccurate due to building obstruction, the obtained latitude and longitude information of the target object will differ significantly from the actual situation, resulting in inaccurate coordinates of the target object. Summary of the Invention

[0004] This application provides a target positioning method, apparatus, and electronic device for improving the accuracy of the coordinates of the determined target object.

[0005] Firstly, this application provides a target localization method, the method comprising:

[0006] Determine the initial image frame corresponding to the initial image of the target object acquired by the image acquisition device;

[0007] The completeness of the target object in the initial image frame is determined, and based on the correspondence between the preset completeness and the preset image frame, an initial preset image frame corresponding to the preset completeness that is consistent with the completeness is determined;

[0008] The first target coordinate point of the initial preset image frame in the first preset coordinate system corresponding to the image acquisition device is determined, wherein the first preset coordinate system is a spatial rectangular coordinate system established with the center point of the lens of the image acquisition device as the origin;

[0009] The first target coordinate point is converted into a second target coordinate point in a spatial rectangular coordinate system established with the world coordinate system as a reference, and the second target coordinate point is used as the positioning coordinate point of the target object.

[0010] Using the above method, the completeness of the initial image frame of the initial image acquired by the image acquisition device is determined. Based on the completeness of the initial image frame, different processing is performed on the initial image frame of the initial image, thereby realizing the positioning of the target object based on the completeness of the initial image. Then, the first target coordinate point of the initial image is converted into the second target coordinate point in the spatial rectangular coordinate system of the real scene, ensuring the accuracy of the determined second target coordinate point, and thus enabling accurate positioning of the target object.

[0011] In one possible design, determining the initial preset image frame before the first target coordinate point in the first preset coordinate system corresponding to the image acquisition device includes:

[0012] An initial parameter set of the image acquisition device is determined, and each initial parameter in the initial parameter set of the image acquisition device is adjusted to its corresponding first parameter. The initial parameter set includes at least: focal length, principal point coordinates, and lens distortion coefficient. The distortion coefficient characterizes the degree of distortion of the lens of the image acquisition device when acquiring images.

[0013] An initial image of the target object is acquired using an image acquisition device adjusted to the first parameter.

[0014] In one possible design, each initial parameter in the initial parameter set of the image acquisition device is adjusted to its corresponding first parameter, including:

[0015] Determining the sharpness of the initial image, in response to the image acquisition device acquiring an initial image whose sharpness is less than a preset sharpness threshold, adjusting each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter; and / or

[0016] The actual position of the target object is determined, and in response to the inconsistency between the actual position and the preset position, each initial parameter in the initial parameter set of the image acquisition device is adjusted to its corresponding first parameter.

[0017] In one possible design, determining the first target coordinate point of the initial preset image frame in the first preset coordinate system corresponding to the image acquisition device includes:

[0018] When the initial image frame is a complete human image frame, the midpoint of the lower edge of the initial image frame is taken as the first coordinate point, and the first coordinate point is processed according to a preset algorithm to obtain the first target coordinate point corresponding to the initial image frame; or

[0019] When the initial image frame is a non-complete human image frame, the height information and head image frame of the target object are determined. Based on the height information and the head image frame, the initial image frame of the target object is determined. The midpoint of the upper edge of the head image frame is taken as the second coordinate point, and the second coordinate point is processed according to a preset algorithm to obtain the first target coordinate point corresponding to the initial image frame.

[0020] In one possible design, converting the first target coordinate point into a second target coordinate point in a spatial Cartesian coordinate system established with reference to the world coordinate system includes:

[0021] When the initial image is a complete human image, the first target coordinate point of the initial image and the distortion coefficient of the image acquisition device are determined. The first target coordinate point and the distortion coefficient are then input into the preset camera imaging model to obtain the second target coordinate point corresponding to the first target coordinate point in the spatial rectangular coordinate system; or

[0022] When the initial image is an incomplete human body image, the height information of the target object corresponding to the initial image is determined, and the incomplete human body image and the height information are input into the preset camera imaging model to obtain the second target coordinate point corresponding to the first target coordinate point in the spatial rectangular coordinate system.

[0023] Secondly, this application provides a target positioning device, the device comprising:

[0024] The determination module is used to determine the initial image frame corresponding to the initial image of the target object acquired by the image acquisition device.

[0025] The corresponding module is used to determine the completeness of the target object in the initial image frame, and based on the correspondence between the preset completeness and the preset image frame, to determine the initial preset image frame corresponding to the preset completeness that is consistent with the completeness.

[0026] The positioning module is used to determine the first target coordinate point of the initial preset image frame in the first preset coordinate system corresponding to the image acquisition device;

[0027] The conversion module is used to convert the first target coordinate point into a second target coordinate point in a spatial rectangular coordinate system established with the world coordinate system as a reference, and to use the second target coordinate point as the positioning coordinate point of the target object.

[0028] In one possible design, the determining module is specifically used to determine the initial parameter set of the image acquisition device, adjust each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter, and acquire an initial image of the target object based on the image acquisition device adjusted to the first parameter.

[0029] In one possible design, the determining module is further configured to determine the sharpness of the initial image, and in response to the initial image acquired by the image acquisition device having a sharpness less than a preset sharpness threshold, adjust each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter, and / or determine the actual position of the target object, and in response to the actual position being inconsistent with the preset position, adjust each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter.

[0030] In one possible design, the corresponding module is specifically used to, when the initial image frame is a complete human image frame, take the midpoint of the lower edge of the initial image frame as the first coordinate point, and process the first coordinate point according to a preset algorithm to obtain the first target coordinate point corresponding to the initial image frame; or when the initial image frame is a non-complete human image frame, determine the height information and head image frame of the target object, determine the initial image frame of the target object based on the height information and the head image frame, take the midpoint of the upper edge of the head image frame as the second coordinate point, and process the second coordinate point according to a preset algorithm to obtain the first target coordinate point corresponding to the initial image frame.

[0031] In one possible design, the conversion module is specifically used to determine the first target coordinate point of the initial image and the distortion coefficient of the image acquisition device when the initial image is a complete human image, and input the first target coordinate point and the distortion coefficient into the preset camera imaging model to obtain the second target coordinate point corresponding to the first target coordinate point in the spatial rectangular coordinate system; or when the initial image is a non-complete human image, determine the height information of the target object corresponding to the initial image, input the non-complete human image and the height information into the preset camera imaging model to obtain the second target coordinate point corresponding to the first target coordinate point in the spatial rectangular coordinate system.

[0032] Thirdly, this application provides an electronic device, comprising:

[0033] Memory, used to store computer programs;

[0034] When a processor executes a computer program stored in the memory, it implements the above-described target localization method steps.

[0035] Fourthly, a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the target localization method described above.

[0036] For details on each of the above-mentioned aspects one through four, and the technical effects that each aspect may achieve, please refer to the above description of the technical effects that can be achieved for the first aspect or the various possible solutions in the first aspect. These details will not be repeated here. Attached Figure Description

[0037] Figure 1 A flowchart of the steps of a target localization method provided in this application;

[0038] Figure 2 A schematic diagram illustrating the initial image of the target object acquired by the image acquisition device provided in this application;

[0039] Figure 3 A schematic diagram illustrating the geometric relationship between the target object and the image acquisition device provided in this application;

[0040] Figure 4 This application provides a schematic diagram of the structure of a target positioning device;

[0041] Figure 5 This application provides a schematic diagram of the structure of an electronic device. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The specific operational methods in the method embodiments can also be applied to the device embodiments or system embodiments. It should be noted that in the description of this application, "multiple" is understood as "at least two". "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A connected to B can represent: A and B directly connected, and A and B connected through C. Furthermore, in the description of this application, terms such as "first" and "second" are used only for distinguishing the purpose of description and should not be construed as indicating or implying relative importance or order.

[0043] In previous technologies, in order to obtain the location information of a target object in a target area, it is necessary to set two reference points in the target area, obtain the scaling ratio between the target area model and the target area, and then calculate the real-time distance between the target object and the reference points and the relative azimuth angle between the target object and the reference points based on the latitude and longitude coordinates of the target object in the target area and the latitude and longitude coordinates of the reference points. Based on the direction and distance of the target object relative to the reference points in the target area and the scaling ratio, the coordinates of the target object relative to the reference points at the target area model level are calculated.

[0044] However, the above method for determining the coordinates of a target object requires obtaining a reference point in the target area and requires latitude and longitude information to locate the target object. When the latitude and longitude information of the target object is inaccurate due to building obstruction, the obtained latitude and longitude information of the target object will differ significantly from the actual situation, thus resulting in inaccurate coordinates of the target object.

[0045] To address the problems described above, embodiments of this application provide a target localization method to improve the accuracy of the determined coordinates of a target object. The methods and apparatus described in these embodiments are based on the same technical concept. Since the principles by which the methods and apparatus solve the problems are similar, embodiments of the apparatus and methods can be referred to interchangeably, and repeated details will not be elaborated further.

[0046] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0047] Reference Figure 1 This application provides a target localization method, which can improve the accuracy of the determined coordinates of the target object. The implementation process of this method is as follows:

[0048] Step S1: Determine the initial image frame corresponding to the initial image of the target object acquired by the image acquisition device.

[0049] The embodiments of this application aim to improve the accuracy of the determined coordinates of the target object. First, an image acquisition device is needed to acquire an initial image of the target object. After obtaining the initial image, an initial image frame can be determined based on the initial image.

[0050] When acquiring an initial image using an image acquisition device, in order to make the acquired initial image clearer, an initial parameter set of the image acquisition device will be determined. This initial parameter set includes at least the focal length and principal point coordinates. After determining the initial parameter set, the sharpness of the initial image needs to be determined. This sharpness is obtained based on the parameters recorded by the image acquisition device. When the sharpness is less than a preset sharpness threshold, the initial parameter set of the image acquisition device will be adjusted to the first parameter corresponding to each initial parameter.

[0051] In addition, when the image acquisition device acquires the initial image of the target object, it obtains the actual position of the target object. When the target object is too far away or too close to the lens of the image acquisition device, the actual position of the target object is not consistent with the preset position. At this time, it is necessary to adjust each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter.

[0052] Step S2: Determine the completeness of the target object in the initial image frame, and based on the correspondence between the preset completeness and the preset image frame, determine the initial preset image frame corresponding to the preset completeness that is consistent with the completeness.

[0053] After determining the first target coordinate point of the initial image, it is necessary to determine the completeness of the target object within the initial image frame. After determining the completeness of the initial image frame, the correspondence between preset completeness and preset image frames is obtained. This correspondence can be stored in a preset list. When the preset list contains this correspondence, the preset list is as shown in Table 1 below:

[0054] Preset completeness Preset image frame (0，50％] Preset image frame 2 (50％，80％] Preset image frame 3 (80％，99％] Preset image frame 4 100％ Preset image frame 1 …… ……

[0055] Table 1

[0056] In Table 1 above, the preset completeness is divided into 4 levels, and each level corresponds to a preset image frame. When the preset completeness is not 100%, it means that the completeness of the target object in the initial image frame is not 100%. Then, the specified position of the target object is determined in the initial image frame based on the preset image frame. This specified position can be the head, feet, etc. of the target object. When the preset completeness is 100%, the preset image frame is the initial image frame. The preset completeness level division in Table 1 above can be adjusted according to the actual situation. The preset completeness level division in Table 1 is just an example and will not be explained in detail here.

[0057] Based on the above determination of the completeness of the target object in the initial image frame, and based on the above Table 1, the preset completeness to which the completeness belongs can be determined, and the preset image frame corresponding to the preset completeness is obtained. The preset image frame is used as the initial preset image frame of the initial image. For example, if the completeness is 89%, then 89% corresponds to the preset image frame 4.

[0058] Step S3: Determine the first target coordinate point of the initial preset image frame in the first preset coordinate system corresponding to the image acquisition device.

[0059] Based on the above description, when the image acquisition device is acquiring the initial image of the target object, the farther the target object is from the center point of the lens of the image acquisition device, the more obvious the distortion of the initial image will be. If the initial image is distorted when the image acquisition device is acquiring the initial image, the position of the target object in the obtained initial image will be the position after the distortion, rather than the true position of the target object in the initial image.

[0060] To reduce the impact of image acquisition device distortion on the initial image, a first parameter set for the image acquisition device is determined. Based on this first parameter set, the various acquisition parameters in the image acquisition device are adjusted, and the adjusted acquisition parameters are then used to control the image acquisition device to acquire the initial image of the target object. The specific process for adjusting the acquisition parameters of the image acquisition device is as follows:

[0061] The checkerboard pattern is fixed on a plane, the orientation of the image acquisition device is adjusted, and multiple checkerboard images from different angles are acquired using the image acquisition device. The first parameter set of the image acquisition device is calculated using Zhang's calibration method. The first parameter set includes: the focal length of the image acquisition device, the radial distortion coefficient and tangential distortion coefficient of the lens of the image acquisition device, and the principal point coordinates. The principal point coordinates are the points mapped from the center point of the lens of the image acquisition device onto the acquisition plane to which the target object belongs. Since the calibration of the image acquisition device based on the checkerboard pattern and Zhang's calibration method is a well-known technique to those skilled in the art, the process of calibrating the image acquisition device based on the checkerboard pattern and Zhang's calibration method will not be described in detail here.

[0062] After calibrating the image acquisition device using the above method, the influence of the image acquisition device's distortion on the initial image of the target object can be effectively reduced, ensuring the accuracy of the center point coordinates of the obtained initial image of the target object.

[0063] After explaining the distortion of the image acquisition device, since a first preset coordinate system can be established based on the center point of the lens of the image acquisition device, it is necessary to determine the first target coordinate point of the initial image frame in the first preset coordinate system. The specific process for determining the first target coordinate point is as follows:

[0064] When the initial image frame is a complete human body image frame, since the initial image frame contains a complete human body image, the position of the target object can be determined based on the initial image frame. After obtaining the initial image frame, the midpoint of the lower edge of the initial image frame is taken as the first coordinate point. Then, the first coordinate point is processed by a preset algorithm, and the first coordinate point after processing by the preset algorithm is taken as the first target coordinate point.

[0065] When the initial image frame is an incomplete human body image frame, since the human body image in the human body image frame is incomplete, in order to determine the position of the target object based on the incomplete human body image, it is necessary to locate the target object based on a part of the human body image in the human body image frame. In the embodiments of this application, the part of the human body image is the head image or the foot image of the target object. Other parts of the human body image refer to the embodiments of this application, and will not be described in detail here.

[0066] Based on the above description, when the initial image is an incomplete human body image, the midpoint of the upper edge of the initial image is taken as the second coordinate point, and the second coordinate point is processed according to a preset algorithm. The second coordinate point after processing by the preset algorithm is taken as the first target coordinate point corresponding to the initial image.

[0067] A schematic diagram of the initial image of the target object acquired by the image acquisition device is shown below. Figure 2 As shown, in Figure 2 In this paper, a spatial rectangular coordinate system is established with the center point of the lens of the image acquisition device as the origin. The point with coordinates (a, b, c) in the spatial rectangular coordinate system is the first target coordinate point corresponding to the initial image of the target object. This first target coordinate point can be the midpoint of the upper edge of the initial image or the midpoint of the lower edge of the initial image, depending on the completeness of the human body image frame in the initial image of the target object acquired by the image acquisition device. Since the determination of the first target coordinate point has been explained in detail above, it will not be explained in detail here.

[0068] The first coordinate point described above represents the midpoint of the lower edge of the target object's feet, and the second coordinate point represents the midpoint of the upper edge of the target object's head. The first and second coordinate points can also represent other positions of the human body image edge. Other human body image edge points are illustrated in the examples in the embodiments of this application, and will not be specifically described here.

[0069] It should be noted that the specific process of obtaining the first target coordinate point from the first coordinate point based on the preset algorithm is a well-known technique to those skilled in the art, and therefore will not be described in detail here.

[0070] Based on the above method, in order to avoid the influence of image acquisition device distortion on the initial image, the image acquisition device is calibrated. Furthermore, the initial image is classified based on the integrity of the human image frame in the initial image, and different methods are used to determine the corresponding first coordinate point and second coordinate point. Then, based on the preset algorithm, the first target coordinate point corresponding to the initial image is obtained, which ensures the accuracy of the obtained first target coordinate point and ensures that the first target coordinate point corresponding to the initial image can be obtained in various real-time scenarios.

[0071] Step S4: Convert the first target coordinate point into a second target coordinate point in a spatial rectangular coordinate system established with the world coordinate system as a reference, and use the second target coordinate point as the positioning coordinate point of the target object.

[0072] After determining the first target coordinate point of the initial image, since the first coordinate point is a coordinate point in a spatial rectangular coordinate system established with the center point of the lens of the image acquisition device as the origin, and the first target coordinate point is a coordinate point obtained based on the image acquisition device, in order to obtain the coordinate point of the target object in the real-time scene, it is necessary to convert the first target coordinate point into a second target coordinate point in a spatial rectangular coordinate system established with the world coordinate system as a reference through a preset camera imaging model. This spatial rectangular coordinate system can be the world coordinate system or a coordinate system established based on latitude and longitude, which is not limited here.

[0073] After obtaining the spatial rectangular coordinate system, converting the first target coordinate point in the first coordinate system into the second target coordinate point in the spatial rectangular coordinate system requires translating and / or rotating the first coordinate point. The preset camera imaging model described above can be a camera imaging model, which will not be explained in detail here.

[0074] When the human image frame in the initial image is a complete human image frame, the first target coordinate point and the distortion coefficient of the image acquisition device are determined. The first target coordinate point and the distortion coefficient are input into the preset camera imaging model to obtain the second target coordinate point corresponding to the first target coordinate point in the spatial rectangular coordinate system. The distortion coefficient is the degree of distortion of the lens of the image acquisition device when acquiring the image.

[0075] For example: A schematic diagram of the initial image of the target object at various coordinate points in the first preset coordinate system, as shown below. Figure 3 As shown, in Figure 3 In the initial image, the coordinates are A(u1, v1) and B(u2, v2). The coordinates of the head image frame in the initial image are C(u3, v3) and D(u4, v4). When the initial image is a complete human image frame, the midpoint E of the lower edge of the initial image is taken as the foot position, and point E is taken as the first target coordinate point, E(u5, v5), u5=uu(1 / 2 2), v5=v2.

[0076] When image acquisition equipment exhibits distortion, its distortion coefficients are divided into radial distortion coefficients and tangential distortion coefficients. The radial distortion coefficients are k1 and k2, and the tangential distortion coefficients are p1 and p2. Since the target object is significantly affected by distortion when it appears at the edge of the image acquisition equipment, it is necessary to calculate the corresponding position of the first target coordinate point of the target object on the distorted image based on the distortion coefficients of the image acquisition equipment. If the coordinates of the image acquisition equipment to be distorted in the first preset coordinate system are (a, b), the principal point coordinates in the first parameter set are (u0, v0), the focus is f, and (x, y) are the normalized coordinates in the first preset coordinate system of the image acquisition equipment, the specific process of obtaining the second target coordinate system based on the above coordinates is as follows:

[0077] x=(u5-u0) / f,y=(v5-v0) / f (1)

[0078] a=x(1+k1r 2 +k2r 4 )+2p1xy+p2(r 2 +2x 2 (2)

[0079] b=y(1+k1r 2 +k2r 4 )+2p2xy+p1(r 2 +2y 2 (3)

[0080] r 2 =x*x+y*y (4)

[0081] Substituting formula (1) into formulas (2) and (3) respectively, we calculate the value of (x, y). Then, substituting (x, y) into formula (4) below, we obtain the coordinate point (u) corresponding to (x, y) in the first preset coordinate system. d v d ):

[0082]

[0083] Based on the above method, (u) is obtained d v d After that, the nearest neighbor algorithm can be used to determine the coordinates (c, d) of the image after distortion correction. After determining the coordinates after distortion correction, (c, d) needs to be transformed into the first preset coordinate system. Then, based on the rotation and / or translation relationship between the first preset coordinate system and the spatial rectangular coordinate system, the second target coordinate point is determined. The specific process of determining the second target coordinate point is as follows:

[0084]

[0085] Substituting the coordinates described above into formula (5), we can calculate matrix R, which is shown below:

[0086]

[0087] Based on formula (5) and matrix R, the second target coordinate point of the target object in the actual scene can be calculated. The second target coordinate point can be represented as:

[0088]

[0089] The above Z map Z represents the vertical height of the target object's feet relative to the ground in the actual scene. map Once the spatial rectangular coordinate system is constructed, these are known quantities. Then, based on the formula for the second target coordinate point mentioned above, we construct a formula about X. map With Y map X can be calculated from a linear equation in two variables. map With Y map The value of is obtained, thus the coordinates of the second target point are obtained.

[0090] When the human body image in the initial image is an incomplete human body image, the height information of the target object corresponding to the initial image is determined, and the incomplete human body image and height information are input into the preset model to obtain the second target coordinate point corresponding to the first target coordinate point in the second preset coordinate system. The specific process of obtaining the second target coordinate point is as follows:

[0091] A head image frame is obtained, and the height information of the target object is determined based on the head image frame. In this embodiment, the height information can be obtained based on an image acquisition device or based on a stored database. The database is used to record the initial image of each target object and the height information of the target object, which will not be explained in detail here.

[0092] Since the initial image is an incomplete human image, the midpoint of the top edge of the head image frame is taken as the position (q, w) of the head. Then, the head image is distorted using the distortion correction method described above, and the coordinates of the head in the distorted head image are denoted as (e, r), as follows. Figure 3 The diagram shows the geometric relationship between the target object and the image acquisition device. Figure 3 In the diagram, point A is the position of the image acquisition device, point B is the top of the target object's head, and point C is the intersection of the line connecting the main point of the image acquisition device and the top of the target object's head with the ground. The height of the image acquisition device is H, and the height information of the target object is... The coordinates of point A in the second preset coordinate system are: The coordinates of point A can be obtained using the following formula:

[0093] A = -R T T

[0094] R and T have already been obtained in the above description, so they will not be explained further here.

[0095] After obtaining the coordinates of points A and C, we can obtain H based on these coordinates, and then calculate the included angle α. The specific formula is as follows:

[0096]

[0097]

[0098] a = asin(H / L1)

[0099] Next, the direction vector from the image acquisition device's projection point E to point C on the ground is calculated and normalized. The specific process is as follows:

[0100]

[0101]

[0102] Based on the above calculation results, the projected coordinates D of the head on the ground are calculated. Point D is the second target coordinate point of the target object. The specific process of obtaining the second target coordinate point of the target object is as follows:

[0103]

[0104]

[0105]

[0106] In the above formula, since This provides the height information of the target object in the actual scene, therefore, it is possible to calculate... and That is, the coordinates of the second target point can be determined.

[0107] Based on the above description, the first target coordinate point in the first preset coordinate system is converted into the second target coordinate point in the spatial rectangular coordinate system, realizing the conversion from the first preset coordinate system to the spatial rectangular coordinate system. This enables the imaging model and geometric relationship to be converted into the real-time scene through the image acquisition device. When the target object is occluded, the head image and height information are used to locate the target object, thereby ensuring the accuracy of the determined target object coordinates.

[0108] Based on the above method, the first target coordinate point of the target object in the first preset coordinate system is determined, and then the first target coordinate point is converted into the second target coordinate point in the second spatial rectangular coordinate system. This realizes the conversion of the first target coordinate point into the second target coordinate point in the real-time scene. The second spatial rectangular coordinate system is established with reference to the world coordinate system. Furthermore, the initial image is distorted before determining the positioning coordinate point of the target object, thereby ensuring the accuracy of the obtained positioning coordinate point of the target object.

[0109] Based on the same inventive concept, this application also provides a target positioning device, which implements the function of a target positioning method, referring to... Figure 4 The device includes:

[0110] The determining module 401 is used to determine the initial image frame corresponding to the initial image of the target object acquired by the image acquisition device;

[0111] The corresponding module 402 is used to determine the completeness of the target object in the initial image frame, and based on the correspondence between the preset completeness and the preset image frame, determine the initial preset image frame corresponding to the preset completeness that is consistent with the completeness.

[0112] The positioning module 403 is used to determine the first target coordinate point of the initial preset image frame in the first preset coordinate system corresponding to the image acquisition device;

[0113] The conversion module 404 is used to convert the first target coordinate point into a second target coordinate point in a spatial rectangular coordinate system established with the world coordinate system as a reference, and to use the second target coordinate point as the positioning coordinate point of the target object.

[0114] In one possible design, the determining module 401 is specifically used to determine the initial parameter set of the image acquisition device, adjust each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter, and acquire the initial image of the target object based on the image acquisition device adjusted to the first parameter.

[0115] In one possible design, the determining module 401 is further configured to determine the sharpness of the initial image, and in response to the initial image acquired by the image acquisition device having a sharpness less than a preset sharpness threshold, adjust each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter, and / or determine the actual position of the target object, and in response to the actual position being inconsistent with the preset position, adjust each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter.

[0116] In one possible design, the corresponding module 402 is specifically used to: when the initial image frame is a complete human image frame, take the midpoint of the lower edge of the initial image frame as the first coordinate point, and process the first coordinate point according to a preset algorithm to obtain the first target coordinate point corresponding to the initial image frame; or when the initial image frame is a non-complete human image frame, determine the height information and head image frame of the target object, determine the initial image frame of the target object based on the height information and the head image frame, take the midpoint of the upper edge of the head image frame as the second coordinate point, and process the second coordinate point according to a preset algorithm to obtain the first target coordinate point corresponding to the initial image frame. In one possible design, the conversion module 404 is specifically used to determine the first target coordinate point of the initial image and the distortion coefficient of the image acquisition device when the initial image is a complete human image, input the first target coordinate point and the distortion coefficient into the preset camera imaging model to obtain the second target coordinate point corresponding to the first target coordinate point in the spatial rectangular coordinate system; or when the initial image is a non-complete human image, determine the height information of the target object corresponding to the initial image, input the non-complete human image and the height information into the preset camera imaging model to obtain the second target coordinate point corresponding to the first target coordinate point in the spatial rectangular coordinate system.

[0117] Based on the same inventive concept, this application also provides an electronic device that can realize the function of the aforementioned target positioning device. (Refer to...) Figure 5 The electronic device includes:

[0118] At least one processor 501 and a memory 502 connected to at least one processor 501. In this embodiment, the specific connection medium between the processor 501 and the memory 502 is not limited. Figure 5 The example shown is the connection between processor 501 and memory 502 via bus 500. Bus 500 is... Figure 5 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. The Bus 500 can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 5 The term 501 is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, the processor 501 can also be called a controller; there is no restriction on the name.

[0119] In this embodiment, memory 502 stores instructions executable by at least one processor 501. By executing the instructions stored in memory 502, at least one processor 501 can execute a target localization method described above. Processor 501 can implement... Figure 4 The functions of each module in the device shown.

[0120] The processor 501 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 502 and calling data stored in memory 502, the processor can perform various functions and process data, thereby monitoring the device as a whole.

[0121] In one possible design, processor 501 may include one or more processing units. Processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 501. In some embodiments, processor 501 and memory 502 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0122] Processor 501 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of a target localization method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0123] Memory 502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 502 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 502 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 502 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0124] By designing and programming the processor 501, the code corresponding to the target localization method described in the foregoing embodiments can be embedded into the chip, thereby enabling the chip to execute the code during runtime. Figure 1 The illustrated embodiment presents a target localization step. How to design and program the processor 501 is a technique well-known to those skilled in the art and will not be described further here.

[0125] Based on the same inventive concept, embodiments of this application also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform a target localization method as described above.

[0126] In some possible implementations, various aspects of the target positioning method provided by this application can also be implemented in the form of a program product, which includes program code that, when the program product is run on a device, causes the control device to perform the steps in a target positioning method according to various exemplary embodiments of this application as described above.

[0127] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0131] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A target localization method, characterized in that, include: Determine the initial image frame corresponding to the initial image of the target object acquired by the image acquisition device; The completeness of the target object in the initial image frame is determined, and based on the correspondence between the preset completeness and the preset image frame, an initial preset image frame corresponding to the preset completeness that is consistent with the completeness is determined; Based on the completeness of the human body image frame in the initial image, after determining the first coordinate point corresponding to the complete human body image frame and the second coordinate point corresponding to the incomplete human body image frame using different methods, the first target coordinate point of the initial preset image frame in the first preset coordinate system corresponding to the image acquisition device is determined. The first preset coordinate system is a spatial rectangular coordinate system established with the center point of the lens of the image acquisition device as the origin. The first target coordinate point is converted into a second target coordinate point in a spatial rectangular coordinate system established with the world coordinate system as a reference, and the second target coordinate point is used as the positioning coordinate point of the target object.

2. The method as described in claim 1, characterized in that, Before determining the initial preset image frame before the first target coordinate point in the first preset coordinate system corresponding to the image acquisition device, the process includes: An initial parameter set of the image acquisition device is determined, and each initial parameter in the initial parameter set of the image acquisition device is adjusted to its corresponding first parameter. The initial parameter set includes at least: focal length, principal point coordinates, and lens distortion coefficient. The distortion coefficient characterizes the degree of distortion of the lens of the image acquisition device when acquiring images. An initial image of the target object is acquired using an image acquisition device adjusted to the first parameter.

3. The method as described in claim 2, characterized in that, Adjusting each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter includes: Determining the sharpness of the initial image, in response to the image acquisition device acquiring an initial image whose sharpness is less than a preset sharpness threshold, adjusting each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter; and / or The actual position of the target object is determined, and in response to the inconsistency between the actual position and the preset position, each initial parameter in the initial parameter set of the image acquisition device is adjusted to its corresponding first parameter.

4. The method as described in claim 1, characterized in that, Determining the first target coordinate point of the initial preset image frame in the first preset coordinate system corresponding to the image acquisition device includes: When the initial image frame is a complete human image frame, the midpoint of the lower edge of the initial image frame is taken as the first coordinate point, and the first coordinate point is processed according to a preset algorithm to obtain the first target coordinate point corresponding to the initial image frame; or When the initial image frame is a non-complete human body image frame, the height information of the target object and the head image frame are determined. Based on the height information and the head image frame, the initial image frame of the target object is determined. The midpoint of the upper edge of the head image frame is taken as the second coordinate point, and the second coordinate point is processed according to a preset algorithm to obtain the first target coordinate point corresponding to the initial image frame.

5. The method as described in claim 1, characterized in that, Converting the first target coordinate point into a second target coordinate point in a spatial rectangular coordinate system established with the world coordinate system as a reference includes: When the initial image is a complete human image, the first target coordinate point of the initial image and the distortion coefficient of the image acquisition device are determined. The first target coordinate point and the distortion coefficient are then input into a preset camera imaging model to obtain the second target coordinate point corresponding to the first target coordinate point in the spatial rectangular coordinate system; or When the initial image is an incomplete human body image, the height information of the target object corresponding to the initial image is determined, and the incomplete human body image and the height information are input into the preset camera imaging model to obtain the second target coordinate point corresponding to the first target coordinate point in the spatial rectangular coordinate system.

6. A target positioning device, characterized in that, The device includes: The determination module is used to determine the initial image frame corresponding to the initial image of the target object acquired by the image acquisition device. The corresponding module is used to determine the first coordinate point corresponding to the complete human body image frame and the second coordinate point corresponding to the incomplete human body image frame by different methods according to the completeness of the human body image frame in the initial image, and then determine the completeness of the target object in the initial image frame. Based on the correspondence between the preset completeness and the preset image frame, the module determines the initial preset image frame corresponding to the preset completeness that is consistent with the completeness. The positioning module is used to determine the first target coordinate point of the initial preset image frame in the first preset coordinate system corresponding to the image acquisition device, wherein the first preset coordinate system is a spatial rectangular coordinate system established with the center point of the lens of the image acquisition device as the origin; The conversion module is used to convert the first target coordinate point into a second target coordinate point in a spatial rectangular coordinate system established with the world coordinate system as a reference, and to use the second target coordinate point as the positioning coordinate point of the target object.

7. The apparatus as claimed in claim 6, characterized in that, The determining module is specifically used to determine the initial parameter set of the image acquisition device, adjust each initial parameter in the initial parameter set of the image acquisition device to its corresponding first parameter, and acquire the initial image of the target object based on the image acquisition device adjusted to the first parameter.

8. The apparatus as claimed in claim 6, characterized in that, The corresponding module is specifically used to: when the initial image frame is a complete human image frame, take the midpoint of the lower edge of the initial image frame as the first coordinate point, and process the first coordinate point according to a preset algorithm to obtain the first target coordinate point corresponding to the initial image frame; or when the initial image frame is a non-complete human image frame, determine the height information and head image frame of the target object, determine the initial image frame of the target object based on the height information and the head image frame, take the midpoint of the upper edge of the head image frame as the second coordinate point, and process the second coordinate point according to a preset algorithm to obtain the first target coordinate point corresponding to the initial image frame.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a computer program stored in the memory, implements the steps of the method according to any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-4.

Images