Positioning method, device, equipment, medium and product based on monocular camera

Abstract

The present application discloses a method, device, equipment, medium and product for positioning based on a monocular camera, which relates to the field of visual technology, and includes: if a target object image is detected in a current video stream, obtaining the size data of the target object image; obtaining the current camera parameters of the monocular camera; calculating the target distance between the camera of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle; positioning the current position of the target object according to the target distance and a matrix set of the monocular camera; in the above manner, after the target object image is detected, the target distance between the camera and the target object is calculated in combination with the current camera parameters of the monocular camera, and then the current position of the target object is positioned in combination with the matrix set of the monocular camera, thereby effectively improving the efficiency of positioning and reducing the positioning cost, and being applicable to different camera devices.

Description

Technical Field

[0001] The present application relates to the field of visual technology, and in particular to a method, device, equipment, medium and product for positioning based on a monocular camera. Background Art

[0002] Using video for positioning has become the current development trend. The commonly used positioning solution is to build the relationship between the object and the camera in the current world coordinates, obtain the relationship between the object and the camera in the video image, and then determine the position of the object in the world coordinates. However, this solution requires the construction of multiple parameter sets and the need to fix the camera, resulting in high positioning costs and limitations. In addition, there is a solution that requires the use of the camera's elevation information to determine the geographic coordinates of the target object. The process of obtaining the elevation information is relatively complicated and will take a lot of time to calculate, resulting in low efficiency of positioning.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a method, device, equipment, medium and product for positioning based on a monocular camera, aiming to solve the technical problems of high cost and low efficiency of positioning in the prior art.

[0005] To achieve the above objectives, the present application proposes a position positioning method based on a monocular camera, the method comprising:

[0006] If a target object image is detected in the current video stream, obtaining size data of the target object image;

[0007] Get the current camera parameters of the monocular camera;

[0008] Calculate the target distance between the camera head of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle;

[0009] Locate the current position of the target object based on the target distance and the matrix set of the monocular camera

[0010] In one embodiment, if a target object image is detected in the current video stream, the step of obtaining size data of the target object image includes:

[0011] Determine characteristic information of the target object;

[0012] Divide the current video stream into frames to obtain image data of each frame;

[0013] Based on the target detection algorithm, detect each frame of image data according to the feature information;

[0014] If a target object image is detected in the target frame image data in each frame image data, size data of the target object image is acquired.

[0015] In one embodiment, the step of calculating the target distance between the camera head of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle includes:

[0016] Get the frame number of the target frame image data;

[0017] When the frame number of the target frame image data is less than a preset detection frame threshold, repeatedly performing the step of detecting the next frame image data based on the target detection algorithm according to the feature information;

[0018] Real-time monitoring of the frame number of the current frame image data;

[0019] When the frame sequence number of the current frame image data is equal to a preset detection frame threshold, stopping detection;

[0020] Calculate the distance between the cameras of the plurality of monocular cameras and the target object according to the current camera parameters and the size data;

[0021] The distances between the cameras of the multiple monocular cameras and the target object are averaged to obtain the target distance.

[0022] In one embodiment, the step of calculating the distances between the cameras of the plurality of monocular cameras and the target object according to the current camera parameters and the size data comprises:

[0023] Determine the number of height pixels according to the height of the anchor box detected by the target detection algorithm and the size data;

[0024] Calculate the height pixel distance of the target object image according to the height pixel number and the height pixel size;

[0025] Acquire an actual height of the target object, wherein the actual height is pre-acquired and stored in a dimension database;

[0026] Obtaining the current focal length of the camera according to the current camera parameters;

[0027] The distances between the cameras of the multiple monocular cameras and the target object are calculated respectively according to the current focal length, the actual height and the height pixel distance.

[0028] In one embodiment, the step of locating the current position of the target object according to the target distance and the matrix set of the monocular camera includes:

[0029] Get the pixel coordinates of the center point of the anchor box detected by the target detection algorithm;

[0030] Calculating the coordinates of the target object in the first coordinate system according to the target distance, the pixel coordinates and the matrix set of the monocular camera based on a target coordinate algorithm;

[0031] Get the directions of the axes of the second coordinate system;

[0032] Rearranging the coordinates of the target object in the world coordinate system according to the directions of the axes to obtain the coordinates of the target object in the second coordinate system;

[0033] Determine the north distance and the east distance between the target object and the camera head of the monocular camera according to the coordinates of the target object in the second coordinate system;

[0034] Get the geographic coordinates of the monocular camera;

[0035] Locating the geographic coordinates of the target object according to the geographic coordinates of the camera head of the monocular camera, the north distance, and the east distance;

[0036] The current position of the target object is obtained according to the geographic coordinates.

[0037] In one embodiment, before the step of locating the current position of the target object according to the target distance and the matrix set of the monocular camera, the method further includes:

[0038] Determine the focal length in each axis direction according to the current focal length of the monocular camera;

[0039] Generate an intrinsic parameter matrix of the camera according to the focal length in each axis direction through a preset calibration program;

[0040] Get the current pitch angle and yaw angle of the monocular camera;

[0041] Generate a target rotation matrix according to the current pitch angle and the current yaw angle by using a rotation matrix template;

[0042] Get the coordinates of the camera of the monocular camera in the first coordinate system;

[0043] Generate an external parameter matrix of the camera according to the coordinates of the camera in the first coordinate system and the target rotation matrix;

[0044] A matrix set of a monocular camera is obtained according to the intrinsic parameter matrix and the extrinsic parameter matrix.

[0045] In addition, to achieve the above purpose, the present application also proposes a position positioning device based on a monocular camera, and the position positioning device based on a monocular camera includes:

[0046] A detection module, configured to obtain size data of a target object image if a target object image is detected in a current video stream;

[0047] The acquisition module is used to obtain the current camera parameters of the monocular camera;

[0048] A calculation module, used for calculating the target distance between the camera head of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle;

[0049] A positioning module is used to locate the current position of the target object according to the target distance and the matrix set of the monocular camera.

[0050] In addition, to achieve the above-mentioned objectives, the present application also proposes a positioning device based on a monocular camera, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the positioning method based on a monocular camera as described above.

[0051] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the monocular camera-based positioning method as described above are implemented.

[0052] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the position positioning method based on a monocular camera as described above.

[0053] One or more technical solutions proposed in the present application have at least the following technical effects: if a target object image is detected in the current video stream, the size data of the target object image is obtained; the current camera parameters of the monocular camera are obtained; based on the target imaging principle, the target distance between the camera of the monocular camera and the target object is calculated according to the current camera parameters and the size data; the current position of the target object is located according to the target distance and the matrix set of the monocular camera; in the above manner, when the target object image is detected, the target distance between the camera and the target object is calculated in combination with the current camera parameters of the monocular camera, and then the current position of the target object is located in combination with the matrix set of the monocular camera, thereby effectively improving the efficiency of position positioning and reducing the positioning cost, and being applicable to different camera devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0056] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the method for positioning based on a monocular camera of the present application;

[0057] Figure 2 A schematic diagram of a flow chart provided for the second embodiment of the method for positioning based on a monocular camera of the present application;

[0058] Figure 3 This is a schematic diagram of the module structure of a position positioning device based on a monocular camera according to an embodiment of the present application;

[0059] Figure 4 Schematic diagram of the device structure of the hardware operating environment involved in the position positioning method based on a monocular camera in an embodiment of the present application.

[0060] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0061] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a positioning controller, etc. The following takes the positioning controller as an example to illustrate this embodiment and the following embodiments.

[0062] Based on this, the embodiment of the present application provides a position positioning method based on a monocular camera, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the position positioning method based on a monocular camera of the present application.

[0063] In this embodiment, the position positioning method based on a monocular camera includes steps S10 to S40:

[0064] Step S10: If a target object image is detected in the current video stream, the size data of the target object image is obtained.

[0065] It should be noted that the current video stream refers to the video stream captured by the monocular camera. The format of the current video stream includes but is not limited to MP4, FLV, WebM and MOV, etc. The target object image refers to the image of the object to be detected, and the size data of the target object image includes but is not limited to the actual height, actual width, pixel size, etc.

[0066] Further, step S10 includes: determining characteristic information of the target object; dividing the current video stream into frames to obtain image data of each frame; detecting each frame image data separately according to the characteristic information based on a target detection algorithm; if a target object image is detected in the target frame image data in each frame image data, obtaining size data of the target object image.

[0067] It should be understood that characteristic information refers to information that can accurately identify the target object. In order to effectively improve the accuracy of detection, this embodiment adopts a frame-by-frame detection strategy to detect image data, that is, before detection, the current video stream needs to be framed. At this time, the current video stream is framed into frames of image data, and then the target detection algorithm is used to learn the characteristic information of the target object. The target detection algorithm can be the Yolov5 detection algorithm. After the learning is completed, each frame of image data is detected separately. The target frame image data is the frame image data in which the target object image is detected for the first time in each frame of image data.

[0068] Step S20, obtaining the current imaging parameters of the monocular camera.

[0069] It can be understood that the current camera parameters refer to parameters for video recording using a monocular camera, and the current camera parameters include but are not limited to focal length, resolution, aperture, etc.

[0070] Step S30, calculating the target distance between the camera head of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle.

[0071] It should be understood that the target imaging principle refers to the imaging principle of an object in a camera. After acquiring the current camera parameters, the target distance between the camera of the monocular camera and the target object is calculated in combination with the size data of the target object image. The target distance may be an average of multiple distances. In this embodiment, the distance is calculated using a monocular camera, which can effectively reduce the cost of calculating the distance compared to binocular distance measurement.

[0072] Step S40: locating the current position of the target object according to the target distance and the matrix set of the monocular camera.

[0073] It can be understood that the matrix set of the monocular camera includes an intrinsic parameter matrix and an extrinsic parameter matrix. After obtaining the target distance between the camera of the monocular camera and the target object, the current position of the target object is located in combination with the matrix set of the monocular camera. The current position can be represented by the geographic coordinates of the target object. For this embodiment, the elevation information of the camera and multiple parameter sets are not required during the positioning process. The preconditions are simple and the positioning efficiency can be effectively improved. In addition, the scheme of this embodiment does not limit the type of camera, that is, it can also be applied to mobile camera equipment such as drones.

[0074] Further, step S40 includes: obtaining the pixel coordinates of the center point of the anchor frame detected by the target detection algorithm; calculating the coordinates of the target object in the first coordinate system based on the target distance, the pixel coordinates and the matrix set of the monocular camera based on the target coordinate algorithm; obtaining the axis directions of the second coordinate system; rearranging the coordinates of the target object in the world coordinate system according to the axis directions to obtain the coordinates of the target object in the second coordinate system; determining the north distance and east distance between the target object and the camera of the monocular camera according to the coordinates of the target object in the second coordinate system; obtaining the geographic coordinates of the camera of the monocular camera; locating the geographic coordinates of the target object according to the geographic coordinates of the camera of the monocular camera, the north distance and the east distance; and obtaining the current position of the target object according to the geographic coordinates.

[0075] It should be understood that the center point of the anchor frame refers to the center point of the anchor frame detected by the target detection algorithm for each frame of image data. The center point of the anchor frame may be the intersection of the lines connecting the diagonal endpoints of the anchor frame. The first coordinate system may be the world coordinate system, and the second coordinate system may be the north-east coordinate system, specifically: the X axis points to the north, the Y axis points to the east, and the Z axis points to the ground. The target coordinate algorithm refers to an algorithm for calculating the coordinates of the object in the first coordinate system, specifically:

[0076]

[0077] Among them, X w Indicates the horizontal coordinate of the target object in the first coordinate system, Y w Represents the vertical coordinate of the target object in the first coordinate system, Z w Represents the vertical axis coordinate of the target object in the first coordinate system, represents the target distance, K1 represents the intrinsic parameter matrix, K2 represents the extrinsic parameter matrix, and (u, v) represents the pixel coordinates of the center point of the anchor box.

[0078] It can be understood that after calculating the coordinates of the target object in the first coordinate system, it is necessary to use the axis directions of the second coordinate system to rearrange the coordinates of the target object in the world coordinate system to achieve the conversion of the coordinate system, and use the coordinates of the target object in the second coordinate system to further determine the north distance and east distance between the target object and the camera of the monocular camera. Since the coordinates of the final position positioning are geographic coordinates, it is necessary to use the geographic coordinates of the camera of the monocular camera, and combine the north distance and the east distance to determine the geographic coordinates of the target object. At this time, the current position of the target object can be obtained, and the final position positioning is completed.

[0079] Furthermore, before the step of locating the current position of the target object according to the target distance and the matrix set of the monocular camera, it also includes: determining the focal length in each axis direction according to the current focal length of the monocular camera; generating the camera's intrinsic parameter matrix according to the focal length in each axis direction through a preset calibration program; obtaining the current pitch angle and the current yaw angle of the monocular camera; generating a target rotation matrix according to the current pitch angle and the current yaw angle through a rotation matrix template; obtaining the coordinates of the monocular camera in the first coordinate system; generating the camera's extrinsic parameter matrix according to the coordinates of the camera in the first coordinate system and the target rotation matrix; and obtaining the matrix set of the monocular camera according to the intrinsic parameter matrix and the extrinsic parameter matrix.

[0080] It can be understood that the focal lengths in the directions of the axes include the focal lengths on the X-axis and the Y-axis in the camera coordinate system, wherein the focal length in the direction of the X-axis can be expressed as f x , the focal length in the Y-axis direction can be expressed as f y , where f x and f y will change with the current focal length of the camera. After obtaining the focal length in each axis direction, the intrinsic parameter matrix of the camera is generated through the preset calibration program, which is specifically expressed as Among them, c x and c y Indicates the offset of the monocular camera optical axis in the camera coordinate system.

[0081] It should be understood that the current pitch angle refers to the angle between the axis of the monocular camera on the vertical plane and the horizontal plane, and the current yaw angle refers to the angle between the projection of the monocular camera on the horizontal plane and the earth's axis. After obtaining the current pitch angle and the current yaw angle of the monocular camera, the current pitch angle and the current yaw angle are input into the rotation matrix template to generate the target rotation matrix, and then the external parameter matrix of the camera is generated in combination with the coordinates of the monocular camera in the first coordinate system, which is specifically expressed as Among them, R represents the target rotation matrix, and t represents the coordinates of the camera of the monocular camera in the first coordinate system, which can also be called the translation vector.

[0082] This embodiment obtains size data of the target object image if a target object image is detected in the current video stream; obtains current camera parameters of a monocular camera; calculates a target distance between the camera of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle; locates the current position of the target object according to the target distance and a matrix set of the monocular camera; in the above manner, after the target object image is detected, the target distance between the camera and the target object is calculated in combination with the current camera parameters of the monocular camera, and then the current position of the target object is located in combination with the matrix set of the monocular camera, thereby effectively improving the efficiency of position positioning and reducing the positioning cost, and being applicable to different camera devices.

[0083] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , step S30 includes steps S301 to S306:

[0084] Step S301, obtaining the frame number of the target frame image data.

[0085] It should be noted that the frame number indicates the order of each frame image data. For example, when the frame number is 1, it indicates that the target frame image data is the first to be detected by the target detection algorithm, and the detection frame number is 1 at this time.

[0086] Step S302, when the frame number of the target frame image data is less than a preset detection frame threshold, repeatedly executing the step of detecting the next frame image data based on the target detection algorithm according to the feature information.

[0087] It should be understood that in order to avoid the situation where the target object moves too fast or the position is too offset, it is necessary to set a preset detection frame threshold. When it is determined that the frame number of the target frame image data is less than the preset detection frame threshold, it indicates that detection needs to continue, that is, repeat the steps of detecting the next frame image data based on the feature information based on the target detection algorithm.

[0088] Step S303: monitor the frame number of the current frame image data in real time.

[0089] It can be understood that the frame number of the current frame image data refers to the frame image data being detected using the target detection algorithm. During repeated detection, the frame number of the current frame image data will be monitored in real time.

[0090] Step S304, when the frame sequence number of the current frame image data is equal to a preset detection frame threshold, stop the detection.

[0091] It can be understood that when it is determined that the frame number of the current frame image data is equal to the preset detection frame threshold, it indicates that the number of detections is sufficient, and the detection is stopped at this time, that is, the frame image data is no longer detected.

[0092] Step S305 , calculating the distances between the cameras of the plurality of monocular cameras and the target object according to the current camera parameters and the size data.

[0093] It can be understood that since there are multiple frames of image data detected, the calculated distances between the camera of the monocular camera and the target object are also multiple. For example, the distance calculated after detecting the first frame of image data is L1, and the distance calculated after detecting the second frame of image data is L2.

[0094] Further, step S305 includes: determining the number of height pixels according to the height of the anchor frame detected by the target detection algorithm and the size data; calculating the height pixel distance of the target object image according to the height pixel number and the height pixel size; obtaining the actual height of the target object, wherein the actual height is pre-acquired and stored in the size database; obtaining the current focal length of the camera according to the current camera parameters; and calculating the distances between the cameras of multiple monocular cameras and the target object according to the current focal length, the actual height and the height pixel distance.

[0095] It can be understood that the height pixel distance refers to the pixel distance of the height of the target object in the image. The height pixel size can be detected by the sensor set in the monocular camera. After determining the number of height pixels using the height and size data of the anchor frame, the height pixel distance of the target object image is calculated in combination with the height pixel size, specifically:

[0096] h p =A·B.

[0097] Among them, h p Represents the height pixel distance of the target object image, A represents the number of height pixels, and B represents the height pixel size.

[0098] It should be noted that in this embodiment, all actual size data related to the target object are pre-acquired and stored in the size database, that is, they can be directly read from the size database when needed, for example, the actual length, actual width and actual height of the target object.

[0099] It should be understood that after obtaining the height pixel distance of the target object image, the distance between the camera of the monocular camera and the target object is calculated in combination with the actual height of the target object and the current focal length of the camera, specifically:

[0100]

[0101] Where D is the distance between the monocular camera and the target object, h t Indicates the actual height of the target object, h p represents the height pixel distance of the target object image, and f represents the current focal length of the camera.

[0102] It should be noted that the above steps are repeated to calculate the distances between the cameras of multiple monocular cameras and the target object. In addition, the distance between the camera of the monocular camera and the target object can also be calculated by the width parameter. The width parameter can be the actual width of the target object and the width pixel distance of the target object image, which is specifically:

[0103]

[0104] Where D is the distance between the monocular camera and the target object, w t Indicates the actual width of the target object, w p represents the width pixel distance of the target object image, and f represents the current focal length of the camera.

[0105] Step S306: averaging the distances between the multiple monocular cameras and the target object to obtain a target distance.

[0106] It can be understood that the target distance represents the average value of multiple distances. After obtaining the distances between the cameras of multiple monocular cameras and the target object, the average of the above multiple distances is calculated to obtain the target distance.

[0107] In this embodiment, the frame number of the target frame image data is obtained; when the frame number of the target frame image data is less than the preset detection frame threshold, the step of detecting the next frame image data according to the feature information is repeatedly executed based on the target detection algorithm; the frame number of the current frame image data is monitored in real time; when the frame number of the current frame image data is equal to the preset detection frame threshold, the detection is stopped; the distances between the cameras of the multiple monocular cameras and the target object are calculated according to the current camera parameters and the size data; the distances between the cameras of the multiple monocular cameras and the target object are averaged to obtain the target distance; in the above manner, when it is determined that the frame number of the target frame image data is less than the preset detection frame threshold, the detection step is repeatedly executed, and the frame number of the current frame image data is monitored in real time during the execution process; when it is greater than the preset detection frame threshold, the distances between the cameras of the multiple monocular cameras and the target object are calculated, and the average value of the above multiple distances is calculated, and the preset detection frame threshold is used to reduce the calculation error, thereby effectively improving the accuracy of calculating the target distance.

[0108] This application also provides a position positioning device based on a monocular camera, please refer to Figure 3 , the position positioning device based on the monocular camera includes:

[0109] The detection module 10 is used to obtain size data of the target object image if a target object image is detected in the current video stream.

[0110] The acquisition module 20 is used to acquire the current imaging parameters of the monocular camera.

[0111] The calculation module 30 is used to calculate the target distance between the camera head of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle.

[0112] The positioning module 40 is used to locate the current position of the target object according to the target distance and the matrix set of the monocular camera.

[0113] This embodiment obtains size data of the target object image if a target object image is detected in the current video stream; obtains current camera parameters of a monocular camera; calculates a target distance between the camera of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle; locates the current position of the target object according to the target distance and a matrix set of the monocular camera; in the above manner, after the target object image is detected, the target distance between the camera and the target object is calculated in combination with the current camera parameters of the monocular camera, and then the current position of the target object is located in combination with the matrix set of the monocular camera, thereby effectively improving the efficiency of position positioning and reducing the positioning cost, and being applicable to different camera devices.

[0114] The monocular camera-based position positioning device provided by the present application adopts the monocular camera-based position positioning method in the above-mentioned embodiment, which can solve the technical problems of high cost and low efficiency of position positioning in the prior art. Compared with the prior art, the beneficial effects of the monocular camera-based position positioning device provided by the present application are the same as the beneficial effects of the monocular camera-based position positioning method provided in the above-mentioned embodiment, and the other technical features of the monocular camera-based position positioning device are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0115] In one embodiment, the detection module 10 is also used to determine the characteristic information of the target object; divide the current video stream into frames to obtain each frame image data; detect each frame image data according to the characteristic information based on the target detection algorithm; if the target object image is detected in the target frame image data in each frame image data, the size data of the target object image is obtained.

[0116] In one embodiment, the calculation module 30 is also used to obtain the frame number of the target frame image data; when the frame number of the target frame image data is less than a preset detection frame threshold, repeatedly execute the steps of detecting the next frame image data based on the feature information based on the target detection algorithm; monitor the frame number of the current frame image data in real time; stop detecting when the frame number of the current frame image data is equal to the preset detection frame threshold; calculate the distance between the cameras of the multiple monocular cameras and the target object according to the current camera parameters and the size data; and average the distances between the cameras of the multiple monocular cameras and the target object to obtain the target distance.

[0117] In one embodiment, the calculation module 30 is further used to determine the number of height pixels based on the height of the anchor frame detected by the target detection algorithm and the size data; calculate the height pixel distance of the target object image based on the height pixel number and the height pixel size; obtain the actual height of the target object, wherein the actual height is pre-acquired and stored in the size database; obtain the current focal length of the camera based on the current camera parameters; and calculate the distances between the cameras of multiple monocular cameras and the target object based on the current focal length, the actual height and the height pixel distance.

[0118] In one embodiment, the positioning module 40 is also used to obtain the pixel coordinates of the center point of the anchor frame detected by the target detection algorithm; calculate the coordinates of the target object in the first coordinate system based on the target distance, the pixel coordinates and the matrix set of the monocular camera based on the target coordinate algorithm; obtain the orientation of each axis of the second coordinate system; rearrange the coordinates of the target object in the world coordinate system according to the orientation of each axis to obtain the coordinates of the target object in the second coordinate system; determine the north distance and east distance between the target object and the camera of the monocular camera according to the coordinates of the target object in the second coordinate system; obtain the geographic coordinates of the camera of the monocular camera; locate the geographic coordinates of the target object according to the geographic coordinates of the camera of the monocular camera, the north distance and the east distance; obtain the current position of the target object according to the geographic coordinates.

[0119] In one embodiment, the positioning module 40 is also used to determine the focal length in each axis direction according to the current focal length of the monocular camera; generate the camera's intrinsic parameter matrix according to the focal length in each axis direction through a preset calibration program; obtain the current pitch angle and the current yaw angle of the monocular camera; generate a target rotation matrix according to the current pitch angle and the current yaw angle through a rotation matrix template; obtain the coordinates of the monocular camera in the first coordinate system; generate the camera's extrinsic parameter matrix according to the coordinates of the camera in the first coordinate system and the target rotation matrix; and obtain a matrix set of the monocular camera according to the intrinsic parameter matrix and the extrinsic parameter matrix.

[0120] The present application provides a position positioning device based on a monocular camera, and the position positioning device based on a monocular camera includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the position positioning method based on the monocular camera in the above-mentioned embodiment 1.

[0121] Reference below Figure 4 , which shows a schematic diagram of the structure of a monocular camera-based position positioning device suitable for implementing the embodiment of the present application. The monocular camera-based position positioning device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4The monocular camera-based positioning device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0122] like Figure 4 As shown, the position positioning device based on the monocular camera may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the position positioning device based on the monocular camera are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the monocular camera-based position positioning device to communicate wirelessly or wired with other devices to exchange data. Although the monocular camera-based position positioning device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0123] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0124] The monocular camera-based positioning device provided by the present application adopts the monocular camera-based positioning method in the above-mentioned embodiment, which can solve the technical problems of high cost and low efficiency of positioning in the prior art. Compared with the prior art, the beneficial effects of the monocular camera-based positioning device provided by the present application are the same as the beneficial effects of the monocular camera-based positioning method provided in the above-mentioned embodiment, and the other technical features of the monocular camera-based positioning device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0125] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0126] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0127] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the position positioning method based on a monocular camera in the above-mentioned embodiment.

[0128] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-ReadOnly Memory), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0129] The computer-readable storage medium may be included in a position positioning device based on a monocular camera; or may exist independently without being assembled into a position positioning device based on a monocular camera.

[0130] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0131] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0132] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0133] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned position positioning method based on a monocular camera, and can solve the technical problems of high cost and low efficiency of position positioning in the prior art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the position positioning method based on a monocular camera provided in the above-mentioned embodiment, and will not be repeated here.

[0134] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned monocular camera-based position positioning method.

[0135] The computer program product provided by the present application can solve the technical problems of high cost and low efficiency of position positioning in the prior art. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the position positioning method based on a monocular camera provided in the above embodiment, and will not be repeated here.

[0136] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for positioning based on a monocular camera, characterized in that: The method comprises: If a target object image is detected in the current video stream, obtaining size data of the target object image; Get the current camera parameters of the monocular camera; Calculate the target distance between the camera head of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle; Locating the current position of the target object according to the target distance and the matrix set of the monocular camera; The step of calculating the target distance between the camera head of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle includes: Get the frame number of the target frame image data; When the frame number of the target frame image data is less than a preset detection frame threshold, repeatedly performing the step of detecting the next frame image data based on the target detection algorithm according to the feature information; Real-time monitoring of the frame number of the current frame image data; When the frame sequence number of the current frame image data is equal to a preset detection frame threshold, stopping detection; Calculate the distance between the cameras of the plurality of monocular cameras and the target object according to the current camera parameters and the size data; Calculating the average distance between the multiple monocular cameras and the target object to obtain the target distance; The step of locating the current position of the target object according to the target distance and the matrix set of the monocular camera includes: Get the pixel coordinates of the center point of the anchor box detected by the target detection algorithm; Calculating the coordinates of the target object in a first coordinate system according to the target distance, the pixel coordinates and the matrix set of the monocular camera based on a target coordinate algorithm; wherein the first coordinate system is a world coordinate system; Obtain the directions of the axes of the second coordinate system; wherein the second coordinate system is the north-east coordinate system; Rearranging the coordinates of the target object in the world coordinate system according to the directions of the axes to obtain the coordinates of the target object in the second coordinate system; Determine the north distance and the east distance between the target object and the camera head of the monocular camera according to the coordinates of the target object in the second coordinate system; Get the geographic coordinates of the monocular camera; Locating the geographic coordinates of the target object according to the geographic coordinates of the camera head of the monocular camera, the north distance, and the east distance; Obtaining the current position of the target object according to the geographic coordinates; Before the step of locating the current position of the target object according to the target distance and the matrix set of the monocular camera, the method further includes: Determine the focal length in each axis direction according to the current focal length of the monocular camera; Generate an intrinsic parameter matrix of the camera according to the focal length in each axis direction through a preset calibration program; Get the current pitch angle and yaw angle of the monocular camera; Generate a target rotation matrix according to the current pitch angle and the current yaw angle by using a rotation matrix template; Get the coordinates of the camera of the monocular camera in the first coordinate system; Generate an external parameter matrix of the camera according to the coordinates of the camera in the first coordinate system and the target rotation matrix; A matrix set of a monocular camera is obtained according to the intrinsic parameter matrix and the extrinsic parameter matrix.

2. The method according to claim 1, characterized in that If a target object image is detected in the current video stream, the step of obtaining size data of the target object image comprises: Determine characteristic information of the target object; Divide the current video stream into frames to obtain image data of each frame; Based on the target detection algorithm, detect each frame of image data according to the feature information; If a target object image is detected in the target frame image data in each frame image data, size data of the target object image is acquired.

3. The method according to claim 1, characterized in that The step of calculating the distances between the cameras of the plurality of monocular cameras and the target object according to the current camera parameters and the size data comprises: Determine the number of height pixels according to the height of the anchor box detected by the target detection algorithm and the size data; Calculate the height pixel distance of the target object image according to the height pixel number and the height pixel size; Acquire an actual height of the target object, wherein the actual height is pre-acquired and stored in a dimension database; Obtaining the current focal length of the camera according to the current camera parameters; The distances between the cameras of the multiple monocular cameras and the target object are calculated respectively according to the current focal length, the actual height and the height pixel distance.

4. A position positioning device based on a monocular camera, characterized in that: The device comprises: A detection module, configured to obtain size data of a target object image if a target object image is detected in a current video stream; The acquisition module is used to obtain the current camera parameters of the monocular camera; A calculation module, used for calculating the target distance between the camera head of the monocular camera and the target object according to the current camera parameters and the size data based on the target imaging principle; A positioning module, used to locate the current position of the target object according to the target distance and the matrix set of the monocular camera; The calculation module is further used to obtain the frame number of the target frame image data; when the frame number of the target frame image data is less than a preset detection frame threshold, repeatedly perform the steps of detecting the next frame image data based on the target detection algorithm according to the feature information; monitor the frame number of the current frame image data in real time; stop the detection when the frame number of the current frame image data is equal to the preset detection frame threshold; calculate the distance between the cameras of the multiple monocular cameras and the target object according to the current camera parameters and the size data; average the distances between the cameras of the multiple monocular cameras and the target object to obtain the target distance; The positioning module is also used to obtain the pixel coordinates of the center point of the anchor frame detected by the target detection algorithm; calculate the coordinates of the target object in the first coordinate system based on the target coordinate algorithm according to the target distance, the pixel coordinates and the matrix set of the monocular camera; wherein the first coordinate system is the world coordinate system; obtain the directions of each axis of the second coordinate system; wherein the second coordinate system is the north-east coordinate system; rearrange the coordinates of the target object in the world coordinate system according to the directions of each axis to obtain the coordinates of the target object in the second coordinate system; determine the north distance and east distance between the target object and the camera of the monocular camera according to the coordinates of the target object in the second coordinate system; obtain the geographic coordinates of the camera of the monocular camera; locate the geographic coordinates of the target object according to the geographic coordinates of the camera of the monocular camera, the north distance and the east distance; obtain the current position of the target object according to the geographic coordinates; The positioning module is also used to determine the focal length in each axis direction according to the current focal length of the monocular camera; generate the camera's internal parameter matrix according to the focal length in each axis direction through a preset calibration program; obtain the current pitch angle and the current yaw angle of the monocular camera; generate a target rotation matrix according to the current pitch angle and the current yaw angle through a rotation matrix template; obtain the coordinates of the monocular camera in the first coordinate system; generate the camera's external parameter matrix according to the coordinates of the camera in the first coordinate system and the target rotation matrix; and obtain the matrix set of the monocular camera according to the internal parameter matrix and the external parameter matrix.

5. A position positioning device based on a monocular camera, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the monocular camera-based positioning method according to any one of claims 1 to 3.

6. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the position positioning method based on a monocular camera as described in any one of claims 1 to 3 are implemented.

7. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the position positioning method based on a monocular camera as claimed in any one of claims 1 to 3 are implemented.

Images