Target positioning method and device, computer device, storage medium and program product

Abstract

The application relates to a target positioning method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring a video frame collected by a shooting device located at the top of a limited space; identifying a target object in the video frame to obtain a target pixel coordinate of the target object; converting the target pixel coordinate of the target object to a world coordinate system based on a first conversion relationship between a pixel coordinate system and the world coordinate system obtained through pre-calibration to obtain a coordinate of the target object in the world coordinate system; and converting the coordinate of the target object in the world coordinate system to a latitude and longitude coordinate system based on a second conversion relationship between the world coordinate system and the latitude and longitude coordinate system obtained through pre-calibration to obtain a latitude and longitude coordinate of the target object in the latitude and longitude coordinate system. The method can improve the positioning accuracy.

Description

Technical Field

[0001] This application relates to the field of positioning technology, and in particular to a target positioning method, apparatus, computer equipment, storage medium, and program product. Background Technology

[0002] With increasing emphasis on traffic safety, there is a need for real-time location tracking and detection of vehicles on highways, intersections, tunnels, and bridges.

[0003] In related technologies, vehicle positioning systems such as GPS or BeiDou can be used to locate vehicles. However, when a vehicle is in a confined space such as a tunnel, the signals of GPS and BeiDou positioning systems are weak, resulting in a significant difference between the vehicle's positioning information and its actual location, leading to low positioning accuracy.

[0004] Therefore, how to improve the accuracy of positioning in confined spaces such as tunnels has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] Therefore, it is necessary to provide a target positioning method, device, computer equipment, storage medium, and program product that can improve the positioning accuracy in a limited space, in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a target localization method. The method includes:

[0007] Acquire video frames captured by a camera located at the top of a confined space;

[0008] The target object in the video frame is identified, and the target pixel coordinates of the target object are obtained;

[0009] Based on the first transformation relationship between the pre-calibrated pixel coordinate system and the world coordinate system, the target pixel coordinates of the target object are transformed to the world coordinate system to obtain the coordinates of the target object in the world coordinate system.

[0010] Based on the second transformation relationship between the pre-calibrated world coordinate system and latitude and longitude coordinate system, the coordinates of the target object in the world coordinate system are transformed to the latitude and longitude coordinate system to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system. The second transformation relationship between the world coordinate system and latitude and longitude coordinate system is determined based on the latitude and longitude coordinates and world coordinates of the calibration point on the plane of the road at the entrance and exit of the limited space. The latitude and longitude coordinates of the calibration point are obtained using positioning equipment.

[0011] In one embodiment, the method for determining the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system includes:

[0012] Based on the mapping data of the finite space, obtain the coordinates of each point in the finite space in the world coordinate system;

[0013] The latitude and longitude coordinates of the first calibration point on the plane of the road at the entrance and exit of the confined space are obtained using a positioning device;

[0014] Based on the latitude and longitude coordinates of the first calibration point and its coordinates in the world coordinate system, the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system is obtained.

[0015] In one embodiment, the method for defining the first transformation relationship between the pixel coordinate system and the world coordinate system includes:

[0016] Based on the mapping data of the finite space, obtain the coordinates of each point in the finite space in the world coordinate system;

[0017] Acquire calibration images from a camera located at the top of a confined space;

[0018] Obtain the coordinates of at least four second calibration points in the calibration image in the pixel coordinate system;

[0019] Based on the coordinates of the second calibration point in the pixel coordinate system and its coordinates in the world coordinate system, the first transformation relationship between the pixel coordinate system and the world coordinate system is obtained.

[0020] In one embodiment, identifying the target object in the video frame and obtaining the target pixel coordinates of the target object includes:

[0021] The target object in the video frame is identified to obtain the target detection box of the target object in the video frame;

[0022] The target pixel coordinates of the target object are determined based on the target detection box.

[0023] In one embodiment, determining the target pixel coordinates of the target object based on the target detection bounding box includes:

[0024] Obtain the coverage area of ​​the target detection box;

[0025] Obtain the pixel coordinates of the center point of the coverage area, and use the pixel coordinates of the center point as the target pixel coordinates of the target object.

[0026] In one embodiment, the method further includes:

[0027] When the signal strength of the positioning device carried by the target object is lower than the set requirement, the step of acquiring the video frame captured by the shooting device located at the top of the confined space is executed;

[0028] When the signal strength of the positioning device carried by the target object meets the set requirements, the positioning device is used to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system.

[0029] Secondly, this application also provides a target positioning device. The device includes:

[0030] The video frame acquisition module is used to acquire video frames captured by a shooting device located at the top of a confined space.

[0031] The recognition module is used to identify target objects in the video frame and obtain the target pixel coordinates of the target objects;

[0032] The world coordinate transformation module is used to transform the target pixel coordinates of the target object to the world coordinate system based on the first transformation relationship between the pre-calibrated pixel coordinate system and the world coordinate system, so as to obtain the coordinates of the target object in the world coordinate system.

[0033] The latitude and longitude coordinate transformation module is used to transform the coordinates of the target object in the world coordinate system to the latitude and longitude coordinate system based on the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system obtained in advance, so as to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system; the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system is determined based on the latitude and longitude coordinates and world coordinates of the calibration point on the plane of the limited space entrance and exit road, and the latitude and longitude coordinates of the calibration point are obtained using positioning equipment.

[0034] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the aforementioned target localization method.

[0035] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the aforementioned target localization method.

[0036] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the aforementioned target localization method.

[0037] The aforementioned target positioning method, apparatus, computer equipment, storage medium, and program product, when the target object is located in a limited space with low signal strength of the positioning device, captures video frames through a shooting device located at the top of the limited space, then performs target recognition on the video frames to determine the target pixel coordinates of the target object, and then transforms the target pixel coordinates to latitude and longitude coordinates in the latitude and longitude coordinate system according to the pre-calibrated first and second transformation relationships, thereby improving the accuracy of target object positioning in limited spaces with low signal strength of the positioning device. Attached Figure Description

[0038] Figure 1 This is a diagram illustrating the application environment of the target localization method in one embodiment;

[0039] Figure 2 This is a flowchart illustrating a target localization method in one embodiment;

[0040] Figure 3 This is a schematic diagram of a finite space in one embodiment;

[0041] Figure 4 This is a flowchart illustrating the step of determining the second transformation relationship in one embodiment;

[0042] Figure 5 This is a flowchart illustrating the step of determining the first transformation relationship in one embodiment;

[0043] Figure 6 This is a flowchart illustrating the target localization method in another embodiment;

[0044] Figure 7 This is a structural block diagram of a target positioning device in one embodiment;

[0045] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0047] The target localization method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed in the cloud or on another network server. The terminal acquires video frames captured by a camera located at the top of a confined space, then identifies the target object in the video frame, obtains the target pixel coordinates of the target object, and then, based on a pre-calibrated first transformation relationship between the pixel coordinate system and the world coordinate system, transforms the target pixel coordinates of the target object into the world coordinate system to obtain the target object's coordinates in the world coordinate system. Finally, based on a pre-calibrated second transformation relationship between the world coordinate system and the latitude and longitude coordinate system, transforms the target object's coordinates in the world coordinate system into the latitude and longitude coordinate system to obtain the target object's latitude and longitude coordinates in the latitude and longitude coordinate system. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0048] Taking the application of target localization in traffic safety scenarios as an example, the terminal communicates with the camera. The camera, located at the top of a confined space, sends the acquired video stream to the terminal. The terminal then extracts video frames from the video stream and identifies the target object in the video frames to obtain the target pixel coordinates of the target object. Based on the first transformation relationship between the pre-calibrated pixel coordinate system and the world coordinate system, the terminal transforms the target pixel coordinates of the target object into the world coordinate system to obtain the target object's coordinates in the world coordinate system. Then, based on the second transformation relationship between the pre-calibrated world coordinate system and the latitude and longitude coordinate system, the terminal transforms the target object's coordinates in the world coordinate system into the latitude and longitude coordinate system to obtain the target object's latitude and longitude coordinates in the latitude and longitude coordinate system, thereby achieving the localization of the target object.

[0049] It should be noted that when the shooting device is an electronic device with processing capabilities, the target localization method can be directly implemented by the shooting device.

[0050] In one embodiment, such as Figure 2 As shown, a target localization method is provided, which can be applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:

[0051] Step 202: Acquire video frames captured by the camera located at the top of the confined space.

[0052] Confined spaces can refer to enclosed or partially enclosed spaces that are relatively isolated from the outside world, have narrow entrances and exits, prevent workers from working inside for extended periods, have poor natural ventilation, and are prone to the accumulation of toxic, harmful, flammable, or explosive substances or insufficient oxygen content. Such confined spaces can be tunnels, caves, etc.

[0053] Filming equipment can refer to electronic devices capable of filming. This filming equipment can be a camera, mobile phone, camcorder, etc. This filming equipment is installed on the ceiling within a limited space. For example, as... Figure 3 As shown, when the shooting device 302 is a camera, the camera can be installed at the top of the tunnel 6 to 8 meters above the ground.

[0054] Video frames can be captured in real time, pre-stored on a server, or input into a terminal device by a user or administrator.

[0055] For example, a video stream can be captured in real time by a camera installed at the top of a confined space, and then the captured video stream can be extracted to obtain video frames. For instance, a camera can be installed at the top of a tunnel at a certain distance from the ground, and then the video stream inside the tunnel can be captured in real time by the camera, and then the video stream can be extracted to obtain video frames.

[0056] In some embodiments, the terminal can obtain video frames pre-stored in a data storage system on a server via a network. These video frames are captured by a camera located at the top of the tunnel.

[0057] Step 204: Identify the target object in the video frame and obtain the target pixel coordinates of the target object.

[0058] The target pixel coordinates can refer to the coordinates of the target object in the pixel coordinate system.

[0059] The target object can refer to the object that needs to be located. This target object can be a vehicle, pedestrian, etc.

[0060] For example, a target recognition algorithm is used to identify target objects in a video frame and obtain the target pixel coordinates of the target objects.

[0061] Step 206: Based on the first transformation relationship between the pre-calibrated pixel coordinate system and the world coordinate system, transform the target pixel coordinates of the target object to the world coordinate system to obtain the coordinates of the target object in the world coordinate system.

[0062] The first transformation relationship refers to the mapping relationship between coordinates in the pixel coordinate system and coordinates in the world coordinate system. This first transformation relationship can be pre-input by the user or administrator, or it can be obtained through pre-processing by the terminal. This first transformation relationship can be represented by the homography matrix H.

[0063] For example, after obtaining the target pixel coordinates of the target object, the target pixel coordinates are mapped and transformed according to the first transformation relationship to obtain the coordinates of the target object in the world coordinate system.

[0064] Step 208: Based on the second transformation relationship between the pre-calibrated world coordinate system and latitude and longitude coordinate system, the coordinates of the target object in the world coordinate system are transformed to the latitude and longitude coordinate system to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system. The second transformation relationship between the world coordinate system and latitude and longitude coordinate system is determined based on the latitude and longitude coordinates of the calibration point on the plane of the limited space entrance and exit road and the world coordinates. The latitude and longitude coordinates of the calibration point are obtained using positioning equipment.

[0065] The second transformation relationship refers to the mapping relationship between coordinates in the world coordinate system and coordinates in the latitude and longitude coordinate system. This second transformation relationship can be pre-input by the user or administrator, or it can be pre-processed by the terminal. This second transformation relationship can be represented by a transformation matrix M.

[0066] A positioning device refers to a device capable of directly obtaining the latitude and longitude coordinates of a target object. This device can be based on GPS (Global Positioning System), BeiDou Navigation Satellite System, or other positioning systems. When a positioning device is inside a confined space such as a tunnel, its signal strength decreases significantly, leading to reduced positioning accuracy. However, when the device is located at the level of a road at the entrance or exit of a confined space, its signal strength increases to meet the set requirements, resulting in significantly improved positioning accuracy. Therefore, a calibration point located at the level of a road at the entrance or exit of a confined space can directly obtain its latitude and longitude coordinates using a positioning device.

[0067] In some embodiments, determining the second transformation relationship by using the latitude and longitude coordinates and world coordinates of a calibration point located on the road plane at the entrance and exit of a confined space can improve the accuracy of the second transformation relationship, thereby improving the positioning accuracy of the target object within the confined space.

[0068] For example, after obtaining the coordinates of the target object in the world coordinate system through the aforementioned steps, the coordinates of the target object in the world coordinate system are transformed according to the second transformation relationship to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system, thereby realizing the positioning of the target object.

[0069] The target positioning method of this application embodiment, when the target object is located in a limited space with low signal strength of the positioning device, captures a video frame by a shooting device located at the top of the limited space, then performs target recognition on the video frame to determine the target pixel coordinates of the target object, and then transforms the target pixel coordinates to latitude and longitude coordinates in the latitude and longitude coordinate system according to the pre-calibrated first transformation relationship and second transformation relationship, thereby improving the accuracy of positioning the target object in a limited space with low signal strength of the positioning device.

[0070] See Figure 3 and Figure 4 , Figure 3 These are schematic diagrams of the structure of a limited space in some embodiments of this application. Figure 4 This is a flowchart illustrating the steps for determining the second transformation relationship in some embodiments of this application. In some embodiments, the method for determining the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system includes, but is not limited to, the following steps:

[0071] Step 402: Based on the mapping data of the finite space, obtain the coordinates of each point in the finite space in the world coordinate system.

[0072] Surveying data can refer to data obtained by surveying a finite space. This surveying data can be represented by point cloud data, such as using coordinates (x, y). n ,y n ,z n The mapping data can be obtained from a server or by surveying a limited space using surveying instruments. For example, a laser surveying instrument can be used to survey a tunnel to obtain tunnel mapping data.

[0073] For example, a laser surveying instrument is used to survey a finite space to obtain surveying data of the finite space. Then, the coordinates of each point in the finite space in the world coordinate system are obtained based on the surveying data.

[0074] Step 404: Use positioning equipment to obtain the latitude and longitude coordinates of the first calibration point on the plane of the road at the entrance and exit of the confined space.

[0075] The first calibration point can refer to a point used to calibrate and determine the transformation relationship between the latitude and longitude coordinate system and the world coordinate system. At least two such first calibration points should be set.

[0076] For example, the latitude and longitude coordinates of two first calibration points located on the plane of the entrance and exit road in a confined space are obtained by a GPS positioning system. The latitude and longitude coordinates of the first calibration point P1 can be represented by (a1, b1), and the latitude and longitude coordinates of the second calibration point P2 can be represented by (a2, b2).

[0077] Step 406: Based on the latitude and longitude coordinates of the first calibration point and its coordinates in the world coordinate system, obtain the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system.

[0078] For example, take Figure 3 Points P1 and P2 are taken as two first calibration points. The coordinates of the first calibration point P1 in the world coordinate system can be represented by (x1, y1, z1), and the coordinates of the second calibration point P2 in the world coordinate system can be represented by (x2, y2, z2). Since the first calibration points are located on the road plane, z1 = z2 = 0 can be taken. Therefore, the second transformation relationship M can be calculated by the following formula (1), which is as follows:

[0079]

[0080] By substituting the latitude and longitude coordinates of the two first calibration points and their coordinates in the world coordinate system into formula (1), the values ​​of c11, c21, c12 and c22 can be calculated.

[0081]

[0082]

[0083] That is, the second transformation relation M can be expressed by formula (2), which is as follows:

[0084]

[0085] Then we have:

[0086]

[0087] The technical solution of this application embodiment determines the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system by using two first calibration points located on the plane of the road at the entrance and exit of a confined space. This facilitates the direct use of the second transformation relationship to determine the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system when locating the target object.

[0088] Please see Figure 3 and Figure 5 , Figure 5This is a flowchart illustrating the step of determining the first transformation relationship in some embodiments. In some embodiments, the method of determining the first transformation relationship between the pixel coordinate system and the world coordinate system includes, but is not limited to, the following steps:

[0089] Step 502: Based on the mapping data of the finite space, obtain the coordinates of each point in the finite space in the world coordinate system.

[0090] Surveying data can refer to data obtained by surveying a finite space. This surveying data can be represented by point cloud data, such as using coordinates (x, y). n ,y n ,z n The mapping data can be obtained from a server or by surveying a limited space using surveying instruments. These instruments can be laser surveyors, 3D scanning equipment, etc. For example, a laser surveyor can be used to survey a tunnel to obtain tunnel mapping data.

[0091] For example, a laser surveying instrument is used to survey a finite space to obtain surveying data of the finite space. Then, the coordinates of each point in the finite space in the world coordinate system are obtained based on the surveying data.

[0092] Step 504: Obtain the calibration image captured by the imaging device located at the top of the confined space.

[0093] The calibration image can refer to an image used to calibrate and determine the transformation relationship between the pixel coordinate system and the world coordinate system. This calibration image is an image of the interior of the finite space captured by an imaging device located at the top of the finite space. For example, the calibration image could be... Figure 3 The diagram shows a finite space structure. This calibration image can be acquired in real-time or obtained by the terminal from a server via a network; this application does not impose specific limitations on this.

[0094] For example, calibration images of the interior of a confined space are acquired in real time by a camera located at the top of the confined space.

[0095] Step 506: Obtain the coordinates of at least four second calibration points in the calibration image in the pixel coordinate system.

[0096] The second calibration point can refer to the point used to calibrate and determine the transformation relationship between the pixel coordinate system and the world coordinate system. These four second calibration points can be selected... Figure 3 Points P3, P4, P5, and P6 in the diagram.

[0097] For example, the coordinates of at least four second calibration points in the calibration image in the pixel coordinate system are obtained using a target recognition algorithm. (The text repeats itself here, so the translation will only include the first instance.) Figure 3The pixel coordinates of points P3, P4, P5, and P6 in the second calibration are as follows: P3(u3, v3), P4(u4, v4), P5(u5, v5), and P6(u6, v6).

[0098] Step 508: Based on the coordinates of the second calibration point in the pixel coordinate system and the coordinates in the world coordinate system, obtain the first transformation relationship between the pixel coordinate system and the world coordinate system.

[0099] For example, the coordinates of P3 to P6 in the world coordinate system are: P3(x3,y3,z3), P4(x4,y4,z4), P5(x5,y5,z5), P6(x6,y6,z6).

[0100] The first transformation relationship H can be calculated using the following formula (4), which is as follows:

[0101]

[0102] Since a homogeneous coordinate system is used, scaling at any scale is possible. We can set h33 = 1 and substitute the pixel coordinates of P3 to P6 and the coordinates in the world coordinate system into formula (4) to obtain formula (5):

[0103]

[0104] The first transformation relation H can be obtained from formula (5).

[0105] The technical solution of this application embodiment determines the first transformation relationship between the pixel coordinate system and the world coordinate system by calibrating four second calibration points in the image. This facilitates the direct use of the first transformation relationship when locating a target object to determine the world coordinates of the target object in the world coordinate system, and thus determine the latitude and longitude coordinates of the target object.

[0106] In some embodiments, step 204 includes, but is not limited to, the following steps: identifying the target object in the video frame to obtain the target detection box of the target object in the video frame; and determining the target pixel coordinates of the target object based on the target detection box.

[0107] A bounding box can be used to represent the region of interest where a target object is located when identifying the target object.

[0108] For example, a target recognition algorithm (such as the YOLO5 algorithm) can be used to identify target objects in a video frame, obtain the target detection box of the target object in the video frame, and then determine the target pixel coordinates of the target object based on the target detection box.

[0109] The technical solution of this application embodiment identifies the target object in the video frame to determine the target detection box of the target object in the video frame, thereby facilitating the determination of the target pixel coordinates of the target object based on the target detection box.

[0110] In some embodiments, the step "determine the target pixel coordinates of the target object based on the target detection box" includes, but is not limited to, the following steps: obtaining the coverage area of ​​the target detection box; obtaining the pixel coordinates of the center point of the coverage area, and using the pixel coordinates of the center point as the target pixel coordinates of the target object.

[0111] The coverage area can refer to the region defined by the target detection box.

[0112] For example, the coverage area of ​​the target detection box is obtained, then the pixel coordinates of the center point of the coverage area are obtained, and then the pixel coordinates of the center point are used as the target pixel coordinates of the target object.

[0113] For example, when the target detection box is a rectangle, the center point of the rectangle can be directly used as the center point of the coverage area, and then the pixel coordinates of the center point can be used as the target pixel coordinates of the target object.

[0114] The technical solution of this application embodiment obtains a coverage area by determining the area bounded by the target detection box, then obtains the pixel coordinates of the center point of the coverage area, and uses the pixel coordinates of the center point as the target pixel coordinates of the target object, thereby facilitating the determination of the target pixel coordinates of the target object.

[0115] In some embodiments, the target positioning method of this application includes, but is not limited to, the following steps: when the signal strength of the positioning device carried by the target object is lower than the set requirement, performing the step of acquiring video frames captured by the shooting device located at the top of the confined space; when the signal strength of the positioning device carried by the target object meets the set requirement, using the positioning device to acquire the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system.

[0116] The setting requirement refers to a pre-defined signal strength requirement. This setting requirement can be entered by the user or administrator, or it can be set by the terminal itself; this application does not impose specific restrictions on this.

[0117] A positioning device is a device carried by the target object. For example, when the target object is a vehicle, the positioning device can be a positioning system installed on the vehicle; when the target object is a pedestrian, the positioning device can be a positioning system carried by the pedestrian's smartphone.

[0118] For example, when the signal strength of the positioning device carried by the target object is lower than the set requirement, it indicates that the target object may be in a confined space with poor signal, such as a tunnel. In this case, to improve the accuracy of the target object's positioning, the step of acquiring video frames captured by a camera located at the top of the confined space is performed to obtain the target object's latitude and longitude coordinates in the latitude and longitude coordinate system, thereby achieving the positioning of the target object. That is, in this case, the aforementioned target positioning method is used to locate the target object and obtain its latitude and longitude coordinates in the latitude and longitude coordinate system to improve the accuracy of the target object's positioning. When the signal strength of the positioning device carried by the target object meets the set requirement, it indicates that the target object is in an environment with good signal. In this case, the target object's latitude and longitude coordinates in the latitude and longitude coordinate system can be directly obtained through the positioning device to achieve the positioning of the target object.

[0119] The technical solution of this application embodiment judges the signal strength of the positioning device carried by the target object. When the signal strength of the positioning device is lower than the set requirement, the step of acquiring video frames captured by the shooting device located at the top of the confined space is executed to realize the positioning of the target object and improve the accuracy of the target object positioning.

[0120] Please see Figure 6 In some embodiments, the target localization method may include two stages: a first stage is a calibration stage and a second stage is a measurement stage. The calibration stage includes steps 602 to 612, and the measurement stage includes steps 614 to 622.

[0121] Step 602: Based on the mapping data of the finite space, obtain the coordinates of each point in the finite space in the world coordinate system.

[0122] Step 604: Obtain the calibration image captured by the imaging device located at the top of the confined space.

[0123] Step 606: Obtain the coordinates of at least four second calibration points in the calibration image in the pixel coordinate system.

[0124] Step 608: Based on the coordinates of the second calibration point in the pixel coordinate system and the coordinates in the world coordinate system, obtain the first transformation relationship between the pixel coordinate system and the world coordinate system.

[0125] Step 610: Use positioning equipment to obtain the latitude and longitude coordinates of the first calibration point on the road plane at the entrance and exit of the confined space.

[0126] Step 612: Based on the latitude and longitude coordinates of the first calibration point and its coordinates in the world coordinate system, obtain the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system.

[0127] Step 614: Identify the target object in the video frame to obtain the target detection box of the target object in the video frame.

[0128] Step 616: Obtain the coverage area of ​​the target detection box.

[0129] Step 618: Obtain the pixel coordinates of the center point of the covered area, and use the pixel coordinates of the center point as the target pixel coordinates of the target object.

[0130] Step 620: Based on the first transformation relationship between the pixel coordinate system and the world coordinate system, the target pixel coordinates of the target object are transformed into the world coordinate system to obtain the coordinates of the target object in the world coordinate system.

[0131] Step 622: Based on the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system, the coordinates of the target object in the world coordinate system are transformed to the latitude and longitude coordinate system to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system. The second transformation relationship between the world coordinate system and the latitude and longitude coordinate system is determined based on the latitude and longitude coordinates of the calibration point on the plane of the road at the entrance and exit of the limited space and the world coordinates. The latitude and longitude coordinates of the calibration point are obtained using positioning equipment.

[0132] It should be noted that for detailed explanations of steps 602 to 622, please refer to the aforementioned embodiments.

[0133] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0134] Based on the same inventive concept, this application also provides a target positioning device for implementing the target positioning method described above. The solution provided by this device is similar to the solution described in the above method.

[0135] In one embodiment, such as Figure 7 As shown, a target positioning device is provided, including: a video frame acquisition module 702, a recognition module 704, a world coordinate transformation module 706, and a latitude and longitude coordinate transformation module 708, wherein:

[0136] The video frame acquisition module 702 is used to acquire video frames captured by a shooting device located at the top of a confined space.

[0137] The recognition module 704 is used to identify target objects in video frames and obtain the target pixel coordinates of the target objects.

[0138] The world coordinate transformation module 706 is used to transform the target pixel coordinates of the target object to the world coordinate system based on the first transformation relationship between the pre-calibrated pixel coordinate system and the world coordinate system, so as to obtain the coordinates of the target object in the world coordinate system.

[0139] The latitude and longitude coordinate transformation module 708 is used to transform the coordinates of the target object in the world coordinate system to the latitude and longitude coordinate system based on the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system obtained in advance, so as to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system. The second transformation relationship between the world coordinate system and the latitude and longitude coordinate system is determined based on the latitude and longitude coordinates and world coordinates of the calibration point on the plane of the limited space entrance and exit road. The latitude and longitude coordinates of the calibration point are obtained using positioning equipment.

[0140] In one embodiment, the target positioning device further includes:

[0141] The first world coordinate acquisition module is used to obtain the coordinates of each point in a finite space in the world coordinate system based on the survey data of the finite space.

[0142] The latitude and longitude coordinate acquisition module is used to obtain the latitude and longitude coordinates of the first calibration point on the road plane at the entrance and exit of a limited space using positioning equipment.

[0143] The second transformation relationship determination module is used to obtain the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system based on the latitude and longitude coordinates of the first calibration point and its coordinates in the world coordinate system.

[0144] In one embodiment, the target positioning device further includes:

[0145] The second world coordinate acquisition module is used to obtain the coordinates of each point in a finite space in the world coordinate system based on the mapping data of the finite space.

[0146] The calibration image acquisition module is used to acquire calibration images captured by the imaging device located at the top of a confined space.

[0147] The pixel coordinate acquisition module is used to obtain the coordinates of at least four second calibration points in the calibration image in the pixel coordinate system.

[0148] The first transformation relationship determination module is used to obtain the first transformation relationship between the pixel coordinate system and the world coordinate system based on the coordinates of the second calibration point in the pixel coordinate system and the coordinates in the world coordinate system.

[0149] In one embodiment, the identification module 704 includes:

[0150] The recognition unit is used to identify target objects in video frames and obtain the target detection box of the target object in the video frame.

[0151] The target pixel coordinate determination unit is used to determine the target pixel coordinates of the target object based on the target detection box.

[0152] In one embodiment, the target pixel coordinate determination unit includes:

[0153] The coverage area acquisition sub-unit is used to acquire the coverage area of ​​the target detection box.

[0154] The target pixel coordinate determination subunit is used to obtain the pixel coordinates of the center point of the coverage area and use the pixel coordinates of the center point as the target pixel coordinates of the target object.

[0155] In one embodiment, the target positioning device further includes:

[0156] The first processing module is used to execute the step of acquiring video frames captured by the shooting device located at the top of the confined space when the signal strength of the positioning device carried by the target object is lower than the set requirement.

[0157] The second processing module is used to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system when the signal strength of the positioning device carried by the target object meets the set requirements.

[0158] Each module in the aforementioned target positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0159] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a target localization method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0160] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0161] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring video frames captured by a camera located at the top of a confined space; identifying a target object in the video frames to obtain the target pixel coordinates of the target object; transforming the target pixel coordinates of the target object into the world coordinate system based on a pre-calibrated first transformation relationship between the pixel coordinate system and the world coordinate system to obtain the coordinates of the target object in the world coordinate system; transforming the coordinates of the target object into the latitude and longitude coordinate system based on a pre-calibrated second transformation relationship between the world coordinate system and the latitude and longitude coordinate system to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system; the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system is determined based on the latitude and longitude coordinates of a calibration point on the plane of the entrance and exit road of the confined space and the world coordinates, and the latitude and longitude coordinates of the calibration point are obtained using a positioning device.

[0162] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining the coordinates of each point in the limited space in the world coordinate system based on the mapping data of the limited space; obtaining the latitude and longitude coordinates of the first calibration point of the road plane at the entrance and exit of the limited space using a positioning device; and obtaining the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system based on the latitude and longitude coordinates of the first calibration point and its coordinates in the world coordinate system.

[0163] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining the coordinates of each point in the finite space in the world coordinate system based on the mapping data of the finite space; obtaining a calibration image captured by an imaging device located at the top of the finite space; obtaining the coordinates of at least four second calibration points in the calibration image in the pixel coordinate system; and obtaining a first transformation relationship between the pixel coordinate system and the world coordinate system based on the coordinates of the second calibration points in the pixel coordinate system and their coordinates in the world coordinate system.

[0164] In one embodiment, when the processor executes the computer program, it further performs the following steps: identifying a target object in a video frame to obtain a target detection box of the target object in the video frame; and determining the target pixel coordinates of the target object based on the target detection box.

[0165] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining the coverage area of ​​the target detection box; obtaining the pixel coordinates of the center point of the coverage area, and using the pixel coordinates of the center point as the target pixel coordinates of the target object.

[0166] In one embodiment, when the processor executes the computer program, it further implements the following steps: when the signal strength of the positioning device carried by the target object is lower than the set requirement, it performs the step of acquiring video frames captured by the shooting device located at the top of the confined space; when the signal strength of the positioning device carried by the target object meets the set requirement, it uses the positioning device to acquire the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system.

[0167] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program performs the following steps: acquiring video frames captured by a camera located at the top of a confined space; identifying a target object in the video frames and obtaining the target pixel coordinates of the target object; based on a first transformation relationship between a pre-calibrated pixel coordinate system and a world coordinate system, transforming the target pixel coordinates of the target object into the world coordinate system to obtain the coordinates of the target object in the world coordinate system; based on a second transformation relationship between a pre-calibrated world coordinate system and a latitude and longitude coordinate system, transforming the coordinates of the target object in the world coordinate system into the latitude and longitude coordinate system to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system; the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system is determined based on the latitude and longitude coordinates and world coordinates of a calibration point on the plane of the entrance and exit road of the confined space, and the latitude and longitude coordinates of the calibration point are obtained using a positioning device.

[0168] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the coordinates of each point in the limited space in the world coordinate system based on the mapping data of the limited space; obtaining the latitude and longitude coordinates of the first calibration point of the road plane at the entrance and exit of the limited space using a positioning device; and obtaining the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system based on the latitude and longitude coordinates of the first calibration point and its coordinates in the world coordinate system.

[0169] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the coordinates of each point in the finite space in the world coordinate system based on the mapping data of the finite space; obtaining a calibration image captured by a camera located at the top of the finite space; obtaining the coordinates of at least four second calibration points in the calibration image in the pixel coordinate system; and obtaining a first transformation relationship between the pixel coordinate system and the world coordinate system based on the coordinates of the second calibration points in the pixel coordinate system and their coordinates in the world coordinate system.

[0170] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: identifying a target object in a video frame to obtain a target detection box of the target object in the video frame; and determining the target pixel coordinates of the target object based on the target detection box.

[0171] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the coverage area of ​​the target detection box; obtaining the pixel coordinates of the center point of the coverage area, and using the pixel coordinates of the center point as the target pixel coordinates of the target object.

[0172] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: when the signal strength of the positioning device carried by the target object is lower than the set requirement, it performs the step of acquiring video frames captured by the shooting device located at the top of the confined space; when the signal strength of the positioning device carried by the target object meets the set requirement, it uses the positioning device to acquire the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system.

[0173] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: acquiring video frames captured by a camera located at the top of a confined space; identifying a target object in the video frames to obtain the target pixel coordinates of the target object; based on a first transformation relationship between a pre-calibrated pixel coordinate system and a world coordinate system, transforming the target pixel coordinates of the target object to the world coordinate system to obtain the coordinates of the target object in the world coordinate system; based on a second transformation relationship between a pre-calibrated world coordinate system and a latitude and longitude coordinate system, transforming the coordinates of the target object in the world coordinate system to the latitude and longitude coordinate system to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system; the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system is determined based on the latitude and longitude coordinates and world coordinates of a calibration point on the plane of the entrance and exit road of the confined space, and the latitude and longitude coordinates of the calibration point are obtained using a positioning device.

[0174] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the coordinates of each point in the limited space in the world coordinate system based on the mapping data of the limited space; obtaining the latitude and longitude coordinates of the first calibration point of the road plane at the entrance and exit of the limited space using a positioning device; and obtaining the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system based on the latitude and longitude coordinates of the first calibration point and its coordinates in the world coordinate system.

[0175] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the coordinates of each point in the finite space in the world coordinate system based on the mapping data of the finite space; obtaining a calibration image captured by a camera located at the top of the finite space; obtaining the coordinates of at least four second calibration points in the calibration image in the pixel coordinate system; and obtaining a first transformation relationship between the pixel coordinate system and the world coordinate system based on the coordinates of the second calibration points in the pixel coordinate system and their coordinates in the world coordinate system.

[0176] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: identifying a target object in a video frame to obtain a target detection box of the target object in the video frame; and determining the target pixel coordinates of the target object based on the target detection box.

[0177] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the coverage area of ​​the target detection box; obtaining the pixel coordinates of the center point of the coverage area, and using the pixel coordinates of the center point as the target pixel coordinates of the target object.

[0178] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: when the signal strength of the positioning device carried by the target object is lower than the set requirement, it performs the step of acquiring video frames captured by the shooting device located at the top of the confined space; when the signal strength of the positioning device carried by the target object meets the set requirement, it uses the positioning device to acquire the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system.

[0179] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0180] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0181] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A target localization method, characterized in that, The method includes: Acquire video frames captured by a camera located at the top of a confined space; The target object in the video frame is identified, and the target pixel coordinates of the target object are obtained; Based on the first transformation relationship between the pre-calibrated pixel coordinate system and the world coordinate system, the target pixel coordinates of the target object are transformed to the world coordinate system to obtain the coordinates of the target object in the world coordinate system. Based on the second transformation relationship between the pre-calibrated world coordinate system and latitude and longitude coordinate system, the coordinates of the target object in the world coordinate system are transformed to the latitude and longitude coordinate system to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system; the second transformation relationship between the world coordinate system and latitude and longitude coordinate system is determined based on the latitude and longitude coordinates and world coordinates of the calibration point on the plane of the road at the entrance and exit of the limited space, and the latitude and longitude coordinates of the calibration point are obtained using positioning equipment; The method for obtaining the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system includes: obtaining the coordinates of each point in the limited space in the world coordinate system based on the survey data of the limited space; obtaining the latitude and longitude coordinates of the first calibration point of the road plane at the entrance and exit of the limited space using a positioning device; and calculating the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system based on the latitude and longitude coordinates of the first calibration point and its coordinates in the world coordinate system. The method further includes: when the signal strength of the positioning device carried by the target object is lower than the set requirement, performing the step of acquiring video frames captured by the shooting device located at the top of the confined space; when the signal strength of the positioning device carried by the target object meets the set requirement, using the positioning device to acquire the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system.

2. The method according to claim 1, characterized in that, The method for defining the first transformation relationship between the pixel coordinate system and the world coordinate system includes: Based on the mapping data of the finite space, obtain the coordinates of each point in the finite space in the world coordinate system; Acquire calibration images from a camera located at the top of a confined space; Obtain the coordinates of at least four second calibration points in the calibration image in the pixel coordinate system; Based on the coordinates of the second calibration point in the pixel coordinate system and its coordinates in the world coordinate system, the first transformation relationship between the pixel coordinate system and the world coordinate system is obtained.

3. The method according to claim 1 or 2, characterized in that, The step of identifying the target object in the video frame and obtaining the target pixel coordinates of the target object includes: The target object in the video frame is identified to obtain the target detection box of the target object in the video frame; The target pixel coordinates of the target object are determined based on the target detection box.

4. The method according to claim 3, characterized in that, Determining the target pixel coordinates of the target object based on the target detection box includes: Obtain the coverage area of ​​the target detection box; Obtain the pixel coordinates of the center point of the coverage area, and use the pixel coordinates of the center point as the target pixel coordinates of the target object.

5. A target positioning device, characterized in that, The device includes: The video frame acquisition module is used to acquire video frames captured by a shooting device located at the top of a confined space. The recognition module is used to identify target objects in the video frame and obtain the target pixel coordinates of the target objects; The world coordinate transformation module is used to transform the target pixel coordinates of the target object to the world coordinate system based on the first transformation relationship between the pre-calibrated pixel coordinate system and the world coordinate system, so as to obtain the coordinates of the target object in the world coordinate system. The latitude and longitude coordinate transformation module is used to transform the coordinates of the target object in the world coordinate system to the latitude and longitude coordinate system based on the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system obtained in a pre-calibrated manner, thereby obtaining the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system. The second transformation relationship between the world coordinate system and the latitude and longitude coordinate system is determined based on the latitude and longitude coordinates and world coordinates of the calibration point on the plane of the limited space entrance and exit road. The latitude and longitude coordinates of the calibration point are obtained using a positioning device. The first world coordinate acquisition module is used to acquire the coordinates of each point in the finite space in the world coordinate system based on the mapping data of the finite space. The latitude and longitude coordinate acquisition module is used to obtain the latitude and longitude coordinates of the first calibration point of the road plane at the entrance and exit of the confined space using positioning equipment; The second transformation relationship determination module is used to calculate the second transformation relationship between the world coordinate system and the latitude and longitude coordinate system based on the latitude and longitude coordinates of the first calibration point and its coordinates in the world coordinate system. The first processing module is used to execute the step of acquiring video frames captured by the shooting device located at the top of the confined space when the signal strength of the positioning device carried by the target object is lower than the set requirement. The second processing module is used to obtain the latitude and longitude coordinates of the target object in the latitude and longitude coordinate system using the positioning device when the signal strength of the positioning device carried by the target object meets the set requirements.

6. The apparatus according to claim 5, characterized in that, The device further includes: The second world coordinate acquisition module is used to acquire the coordinates of each point in the finite space in the world coordinate system based on the mapping data of the finite space. The calibration image acquisition module is used to acquire calibration images captured by the imaging device located at the top of a confined space; A pixel coordinate acquisition module is used to acquire the coordinates of at least four second calibration points in the calibration image in the pixel coordinate system; The first transformation relationship determination module is used to obtain the first transformation relationship between the pixel coordinate system and the world coordinate system based on the coordinates of the second calibration point in the pixel coordinate system and the coordinates in the world coordinate system.

7. The apparatus according to claim 5 or 6, characterized in that, The identification module further includes: The recognition unit is used to recognize the target object in the video frame and obtain the target detection box of the target object in the video frame; The target pixel coordinate determination unit is used to determine the target pixel coordinates of the target object based on the target detection box.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images