A method, device, equipment and storage medium for locating a target object

Abstract

The present invention discloses a method, device, equipment and storage medium for positioning a target object. After detecting a target object from an environmental image, a first depth prediction value of the target object in a camera coordinate system is calculated, then the first depth prediction value is corrected to obtain a second depth prediction value, and then the coordinates of the target object in the camera coordinate system are calculated based on the center point of the target object and the second depth prediction value. By correcting the depth value of the target object in the camera coordinate system and calculating the coordinates of the target object in the camera coordinate system based on the corrected depth value, the position accuracy of the target object is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of autonomous driving technology, and in particular to a target object positioning method, device, equipment and storage medium. Background Art

[0002] An autonomous vehicle is a new type of smart car that uses sensors to sense its surroundings and collect environmental information. The control device (i.e., the onboard intelligent brain) accurately calculates and analyzes the environmental information, and finally sends instructions to the ECU (Electronic Control Unit) to control different devices in the autonomous vehicle, thereby achieving fully automatic operation of the vehicle and achieving the purpose of unmanned driving.

[0003] The perception of external target objects (e.g., obstacles) is one of the most important functions of autonomous driving, so that autonomous vehicles can avoid obstacles in time based on the perceived target objects to avoid accidents. Currently, the most commonly used sensors for target object perception are lidar and cameras. Lidar has the disadvantage of high cost, while the position error of the target object obtained by camera-based target object detection is large. Summary of the invention

[0004] The present invention provides a target object positioning method, device, equipment and storage medium to improve the position accuracy of the target object.

[0005] In a first aspect, an embodiment of the present invention provides a method for locating a target object, comprising:

[0006] Calculating a first depth prediction value of the target object in a camera coordinate system based on the target object detected from the environment image;

[0007] Correcting the first depth prediction value to obtain a second depth prediction value;

[0008] The coordinates of the target object in the camera coordinate system are calculated based on the center point of the target object and the second depth prediction value.

[0009] In a second aspect, an embodiment of the present invention further provides a target object positioning device, comprising:

[0010] A first depth prediction value calculation module, configured to calculate a first depth prediction value of the target object in a camera coordinate system based on the target object detected from the environment image;

[0011] A correction module, used for correcting the first depth prediction value to obtain a second depth prediction value;

[0012] A coordinate determination module is used to calculate the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value.

[0013] In a third aspect, an embodiment of the present invention further provides a computer device, including:

[0014] one or more processors;

[0015] A storage device for storing one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the target object positioning method provided in the first aspect of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target object positioning method provided in the first aspect of the present invention.

[0018] The target object positioning method provided by the embodiment of the present invention calculates a first depth prediction value of the target object in the camera coordinate system after detecting the target object from the environment image, then corrects the first depth prediction value to obtain a second depth prediction value, and then calculates the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value. By correcting the depth value of the target object in the camera coordinate system and calculating the coordinates of the target object in the camera coordinate system based on the corrected depth value, the position accuracy of the target object is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a target object positioning method provided in Embodiment 1 of the present invention;

[0020] Figure 2A A flowchart of a target object positioning method provided in Embodiment 2 of the present invention;

[0021] Figure 2B This is the imaging principle diagram of the camera;

[0022] Figure 3 A schematic diagram of the structure of a target object positioning device provided in Embodiment 3 of the present invention;

[0023] Figure 4 A schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0025] Embodiment 1

[0026] Figure 1 This is a flow chart of a target object positioning method provided in the first embodiment of the present invention. This embodiment is applicable to the case where the camera coordinates of the target object are calculated using the environment image captured by the camera. The method can be executed by the target object positioning device provided in the embodiment of the present invention. The device can be implemented in software and / or hardware, and is usually configured in a computer device, such as Figure 1 As shown, the method specifically comprises the following steps:

[0027] S101 . Calculate a first depth prediction value of a target object in a camera coordinate system based on a target object detected from an environment image.

[0028] In an embodiment of the present invention, the environmental image is an image collected by the unmanned device during its movement and within the visible range of the camera. Exemplarily, in an embodiment of the present invention, the camera is a monocular camera. Exemplarily, in an embodiment of the present invention, common target detection methods such as Fast-RCNN or YOLO (You Only Look Once) series models can be used for target detection to detect the target object from the environmental image. The target object may be an obstacle in the process of the unmanned device moving. For example, for an unmanned vehicle, the target object may be other vehicles, pedestrians, or other objects that temporarily appear on the road surface. The embodiment of the present invention is not limited here.

[0029] After detecting the target object, the pixel coordinates of the target object in the environment image are determined. Based on the imaging principle of the camera and combined with the camera parameters, the first depth prediction value of the target object in the camera coordinate system can be calculated. Specifically, it can be assumed that the road surface is a horizontal plane, and the maximum point of the Y axis of the target object in the pixel coordinate system is the grounding point. Combined with the camera parameters and the height of the camera, the first depth prediction value of the target object in the camera coordinate system can be calculated. The depth of the target object in the camera coordinate system is the horizontal distance of the target object from the origin of the camera coordinate system.

[0030] S102: Modify the first depth prediction value to obtain a second depth prediction value.

[0031] In the calculation process of the first depth prediction value, it is calculated based on the ground point of the target object. However, the ground point is not the center point of the Y axis of the target object in the pixel coordinate system, which leads to a certain error between the calculated first depth prediction value and the actual depth value. In order to eliminate this error, the first depth prediction value needs to be corrected to obtain the second depth prediction value.

[0032] S103 : Calculate the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value.

[0033] The center point of the target object can be the center point of the target object in the pixel coordinate system, that is, the 2D projection of the center point of the 3D target object in the environment image. Based on the imaging principle of the camera, combined with the center point of the target object, the second depth prediction value and the camera parameters, the X-axis coordinate and the Y-axis coordinate of the target object in the camera coordinate system can be calculated, thereby obtaining the 3D coordinates of the target object in the camera coordinate system.

[0034] The target object positioning method provided by the embodiment of the present invention calculates a first depth prediction value of the target object in the camera coordinate system after detecting the target object from the environment image, then corrects the first depth prediction value to obtain a second depth prediction value, and then calculates the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value. By correcting the depth value of the target object in the camera coordinate system and calculating the coordinates of the target object in the camera coordinate system based on the corrected depth value, the position accuracy of the target object is improved.

[0035] Embodiment 2

[0036] Figure 2A This is a flow chart of a target object positioning method provided in the second embodiment of the present invention. This embodiment is refined on the basis of the above-mentioned first embodiment, and describes in detail the specific implementation process of each step in the target object positioning method, such as Figure 2A As shown, the method includes:

[0037] S201, obtaining an environment image captured by a camera of an unmanned device.

[0038] In an embodiment of the present invention, the unmanned device may be an unmanned vehicle equipped with a camera, and the environment image is an image collected by the unmanned device during movement and within the visible range of the camera. Exemplarily, in an embodiment of the present invention, the camera is a monocular camera.

[0039] S202: Detect a target object from the environment image, and indicate the target object with a rectangular frame in the environment image.

[0040] For example, in the embodiments of the present invention, common target detection methods such as Fast-RCNN or YOLO (You Only Look Once) series models can be used to detect the target object from the environment image. The target object can be an obstacle in the process of the unmanned equipment. For example, for an unmanned vehicle, the target object can be other vehicles, pedestrians, and other objects that temporarily appear on the road surface. The embodiments of the present invention are not limited here. The detected target object is usually selected and displayed with a rectangular box. The pixel coordinates of the rectangular box in the environment image can be used (x min ,y min , x max ,y max ) is used to represent the min and x max is the left and right borders of the rectangular box, y min and max The upper and lower boundaries of the rectangular box.

[0041] S203 : Calculate a first depth prediction value of the target object in the camera coordinate system based on the pixel coordinates of the rectangular frame and the camera parameters.

[0042] Figure 2B For the imaging principle diagram of the camera, refer to Figure 2B After obtaining the pixel coordinates of the rectangular frame, the first depth prediction value of the target object in the camera coordinate system is calculated based on the imaging principle of the camera, the pixel coordinates of the rectangular frame and the camera parameters. Among them, the camera parameters are the intrinsic parameters of the camera, which can be expressed by the following matrix:

[0043]

[0044] Among them, f x is the focal length in the X direction of the camera coordinate system, f y is the focal length in the Y direction of the camera coordinate system, c x is the offset of the camera's optical axis relative to the coordinate center of the projection plane on the X axis, c y The offset of the camera's optical axis relative to the coordinate center of the projection plane on the Y axis

[0045] Specifically, the maximum value of the Y-axis coordinate in the pixel coordinates of the rectangular frame (i.e., y max ) minus the offset c of the camera's optical axis relative to the coordinate center of the projection plane on the Y axis y , get the first value. Calculate the focal length f of the camera on the Y axis yThe quotient of the third value and the first value is calculated to obtain a second value. The product of the second value and the height h of the camera from the ground is calculated to obtain a third value. The product of the second value and the tangent value of the inclination angle α of the camera relative to the ground is calculated to obtain a fourth value. The quotient of the third value and the fourth value is calculated to obtain a fifth value. The sum of the third value and the fifth value is calculated to obtain a first depth prediction value D of the target object in the camera coordinate system. ref .

[0046] The mathematical expression of the above calculation process is:

[0047]

[0048] t=f y / (y max -c y )

[0049] S204 , pre-training a correction parameter output model based on a first depth prediction value of the target object in the camera coordinate system and a known actual depth value.

[0050] Since the lower boundary y of the rectangular box predicted by the target detection algorithm in the previous step max It is not necessarily accurate, and the target object has a certain height, which causes the center point of the target object to not be the grounding point of the target object. Therefore, the first depth prediction value D calculated in the above step needs to be ref Make corrections.

[0051] Exemplarily, the embodiment of the present invention uses a deep learning model to calculate a correction parameter, and then uses the correction parameter to adjust the first depth prediction value D ref Make corrections.

[0052] Specifically, a correction parameter output model is established based on the initial model parameters, and a large number of data samples are collected in advance, the data samples include the first depth prediction value of the target object and the known actual depth value of the target object. The correction parameter output model is iteratively trained using a large number of data samples until the model converges (that is, the error between the first depth prediction value and the actual depth value meets the requirements).

[0053] S205 , inputting the first depth prediction value of the target object in the camera coordinate system into the trained training correction parameter output model for processing to obtain the correction parameter.

[0054] In practical application, the first depth prediction value D of the target object in the camera coordinate system obtained in the above steps is ref Input the trained training correction parameter output model for processing to obtain the correction parameter δ. The correction parameter is defined as follows:

[0055] δ=D gt / D ref

[0056] Among them, D gt The actual depth value of the target object.

[0057] S206: Use a correction parameter to correct the first depth prediction value to obtain a second depth prediction value.

[0058] Exemplarily, in this embodiment of the present invention, the first depth prediction value D ref Multiply it with the modified δ to get the second depth prediction value D pre The specific calculation formula is as follows:

[0059] D pre =δ*D ref

[0060] S207: Detect the center point of the target object from the environment image.

[0061] In an embodiment of the present invention, a deep neural network can be used to directly predict the center point of the target object, or the center point of the rectangular box obtained in the above steps can be corrected to obtain the center point of the target object, which is not limited in the embodiment of the present invention.

[0062] Exemplarily, in some embodiments of the present invention, the CenterNet target detection model is used to process the environment image to obtain the center point of the target object.

[0063] The traditional YOLO model and Fast-RCNN rely on a large number of anchors for target detection. Due to the huge number of anchors and the artificially designed size of the anchors, the detection speed and accuracy are not high. CenterNet is an anchor-free target detection model that can directly predict the center point of the target object, improving the target detection efficiency and the position accuracy of the center point.

[0064] Exemplarily, in another embodiment of the present invention, a deep learning model is used to calculate the deviation between the center point of the rectangular box and the actual center point, and then the deviation is used to correct the position of the center point of the rectangular box.

[0065] Specifically, a deviation output model is established based on the initial model parameters, and a large number of data samples are collected in advance, including the center point coordinates of the rectangular box of the target object and the actual center point coordinates of the known target object. The deviation output model is iteratively trained with a large number of data samples until the model converges (that is, the error between the center point coordinates of the rectangular box of the target object and the actual center point coordinates meets the requirements).

[0066] In practical applications, the pixel coordinates of the center point of the rectangular frame obtained in the above target detection step (x max +x min ) / 2,(y max +y min ) / 2) Input the trained deviation output model for processing to obtain the deviation (x offset ,y offset ).

[0067] The center point of the rectangular frame is corrected based on the deviation to obtain the center point of the target object. Specifically, the calculation formula is as follows:

[0068]

[0069]

[0070] Among them, x p is the X-axis coordinate of the center point of the corrected target object in the pixel coordinate system, y p is the Y-axis coordinate of the center point of the corrected target object in the pixel coordinate system.

[0071] S208 , calculating the X-axis coordinate of the center point of the target object in the camera coordinate system based on the X-axis coordinate of the pixel coordinate of the center point of the target object, the camera parameters, and the second depth prediction value.

[0072] After obtaining the pixel coordinates and the second depth prediction value of the center point of the target object, based on the imaging principle of the camera, the X-axis coordinate of the pixel coordinates of the center point of the target object, the camera parameters and the second depth prediction value are calculated.

[0073] Specifically, the difference between the X-axis coordinate of the pixel coordinate of the center point of the target object and the offset of the optical axis of the camera relative to the coordinate center of the projection plane on the X-axis is calculated to obtain a sixth value. The product of the sixth value and the focal length of the camera on the X-axis is calculated to obtain a seventh value. The quotient of the seventh value and the second depth prediction value is calculated to obtain the X-axis coordinate of the center point of the target object in the camera coordinate system.

[0074] The mathematical expression of the above calculation process is:

[0075] x d =f x *(x p -c x ) / D pre

[0076] Among them, x d The X-axis coordinate of the center point of the target object in the camera coordinate system.

[0077] S209 , calculating the Y-axis coordinate of the center point of the target object in the camera coordinate system based on the Y-axis coordinate of the pixel coordinate of the center point of the target object, the camera parameters, and the second depth prediction value.

[0078] Similarly, after obtaining the pixel coordinates of the center point of the target object and the second depth prediction value, based on the imaging principle of the camera, the Y-axis coordinate of the center point of the target object in the camera coordinate system is calculated according to the Y-axis coordinate of the pixel coordinate of the center point of the target object, the camera parameters and the second depth prediction value.

[0079] Specifically, the difference between the Y-axis coordinate of the pixel coordinate of the center point of the target object and the offset of the optical axis of the camera relative to the coordinate center of the projection plane on the Y-axis is calculated to obtain a ninth value. The product of the ninth value and the focal length of the camera on the Y-axis is calculated to obtain a tenth value. The quotient of the tenth value and the second depth prediction value is calculated to obtain the Y-axis coordinate of the center point of the target object in the camera coordinate system.

[0080] The mathematical expression of the above calculation process is:

[0081] y d =f y *(y p -c y ) / D pre

[0082] Among them, y d The Y-axis coordinate of the center point of the target object in the camera coordinate system.

[0083] So far, through the above steps, the coordinates of the final target object in the camera coordinate system are (x d ,y d ,D pre ).

[0084] The target object positioning method provided by the embodiment of the present invention, after detecting the target object from the environment image, calculates the first depth prediction value of the target object in the camera coordinate system based on the pixel coordinates of the rectangular frame and the camera parameters, then uses the deep learning model to calculate a correction parameter, and then uses the correction parameter to correct the first depth prediction value to obtain a second depth prediction value, and then calculates the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value. The depth value of the target object in the camera coordinate is corrected by the deep learning model, and the coordinates of the target object in the camera coordinate system are calculated based on the corrected depth value, thereby improving the position accuracy of the target object.

[0085] Embodiment 3

[0086] Figure 3A schematic diagram of a target object positioning device provided in Embodiment 3 of the present invention is shown in FIG. Figure 3 As shown, the device comprises:

[0087] A first depth prediction value calculation module 301 is used to calculate a first depth prediction value of the target object in a camera coordinate system based on the target object detected from the environment image;

[0088] A correction module 302, configured to correct the first depth prediction value to obtain a second depth prediction value;

[0089] The coordinate determination module 303 is configured to calculate the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value.

[0090] In some embodiments of the present invention, the first depth prediction value calculation module 301 includes:

[0091] The image acquisition submodule is used to obtain the environment image collected by the camera of the unmanned equipment;

[0092] A target object detection submodule, used to detect a target object from the environment image, wherein the target object is shown in the environment image with a rectangular frame;

[0093] The first depth prediction value calculation submodule is used to calculate a first depth prediction value of the target object in a camera coordinate system based on the pixel coordinates of the rectangular frame and camera parameters.

[0094] In some embodiments of the present invention, the first depth prediction value calculation submodule includes:

[0095] A first numerical value calculation unit, configured to obtain a first numerical value by subtracting an offset of the optical axis of the camera relative to the coordinate center of the projection plane on the Y axis from a maximum value of the Y axis coordinate in the pixel coordinates of the rectangular frame;

[0096] A second numerical value calculation unit, used for calculating the quotient of the focal length of the camera on the Y axis and the first value to obtain a second value;

[0097] A third value calculation unit, used for calculating the product of the second value and the height of the camera from the ground to obtain a third value;

[0098] a fourth value calculation unit, configured to calculate the product of the second value and the tangent value of the inclination angle of the camera relative to the ground to obtain a fourth value;

[0099] a fifth value calculation unit, configured to calculate a quotient of the third value and the fourth value to obtain a fifth value;

[0100] The first depth prediction value calculation unit is used to calculate the sum of the third value and the fifth value to obtain a first depth prediction value of the target object in the camera coordinate system.

[0101] In some embodiments of the present invention, the correction module 302 includes:

[0102] A correction parameter output model training submodule, used to pre-train a correction parameter output model based on a first depth prediction value of a target object in a camera coordinate system and a known actual depth value;

[0103] A correction parameter calculation submodule, used for inputting the first depth prediction value of the target object in the camera coordinate system into a trained training correction parameter output model for processing to obtain a correction parameter;

[0104] The correction submodule is used to correct the first depth prediction value using the correction parameter to obtain a second depth prediction value.

[0105] In some embodiments of the present invention, the target object positioning device further includes:

[0106] A center point determination module is used to detect the center point of the target object from the environment image before calculating the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value.

[0107] In some embodiments of the present invention, the center point determination module includes:

[0108] The center point detection submodule is used to process the environment image using the CenterNet target detection model to obtain the center point of the target object.

[0109] In some embodiments of the present invention, the center point determination module includes:

[0110] A deviation output model submodule, used to pre-train the deviation output model based on the center point of the rectangular box and the center point of the known target object;

[0111] The deviation calculation submodule is used to input the pixel coordinates of the center point of the rectangular box into the trained deviation output model for processing, so as to obtain the deviation between the center point of the rectangular box and the center point of the target object;

[0112] The center point correction submodule is used to correct the center point of the rectangular frame based on the deviation to obtain the center point of the target object.

[0113] In some embodiments of the present invention, the coordinate determination module 303 includes:

[0114] An X-axis coordinate calculation submodule, used to calculate the X-axis coordinate of the center point of the target object in the camera coordinate system based on the X-axis coordinate of the pixel coordinate of the center point of the target object, the camera parameters and the second depth prediction value;

[0115] The Y-axis coordinate calculation submodule is used to calculate the Y-axis coordinate of the center point of the target object in the camera coordinate system based on the Y-axis coordinate of the pixel coordinate of the center point of the target object, the camera parameters and the second depth prediction value.

[0116] In some embodiments of the present invention, the X-axis coordinate calculation submodule includes:

[0117] a sixth numerical value calculation unit, configured to calculate a difference between an X-axis coordinate of a pixel coordinate of a center point of the target object and an offset of an optical axis of the camera relative to a coordinate center of a projection plane on an X-axis to obtain a sixth numerical value;

[0118] a seventh value calculation unit, configured to calculate the product of the sixth value and the focal length of the camera on the X-axis to obtain a seventh value;

[0119] An X-axis coordinate calculation unit is used to calculate the quotient of the seventh value and the second depth prediction value to obtain the X-axis coordinate of the center point of the target object in the camera coordinate system.

[0120] In some embodiments of the present invention, the Y-axis coordinate calculation submodule includes:

[0121] a ninth numerical value calculation unit, configured to calculate a difference between the Y-axis coordinate of the pixel coordinate of the center point of the target object and an offset of the optical axis of the camera relative to the coordinate center of the projection plane on the Y-axis to obtain a ninth numerical value;

[0122] a tenth value calculation unit, configured to calculate the product of the ninth value and the focal length of the camera on the Y axis to obtain a tenth value;

[0123] The Y-axis coordinate calculation unit is used to calculate the quotient of the tenth value and the second depth prediction value to obtain the Y-axis coordinate of the center point of the target object in the camera coordinate system.

[0124] The above-mentioned target object positioning device can execute the target object positioning method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the target object positioning method.

[0125] Embodiment 4

[0126] Embodiment 4 of the present invention provides a computer device, Figure 4 A schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention is shown in FIG. Figure 4 As shown, the computer device includes:

[0127] Processor 401, memory 402, communication module 403, input device 404 and output device 405; the number of processors 401 in the mobile terminal may be one or more. Figure 4 A processor 401 is taken as an example; the processor 401, the memory 402, the communication module 403, the input device 404 and the output device 405 in the mobile terminal can be connected by a bus or other means, Figure 4 The above processor 401, memory 402, communication module 403, input device 404 and output device 405 may be integrated in a computer device.

[0128] The memory 402 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as the module corresponding to the target object positioning method in the above embodiment. The processor 401 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 402, that is, realizing the above-mentioned target object positioning method.

[0129] The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the microcomputer, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 402 may further include a memory remotely arranged relative to the processor 401, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0130] The communication module 403 is used to establish a connection with an external device (such as a smart terminal) and realize data interaction with the external device. The input device 404 can be used to receive input digital or character information and generate key signal input related to user settings and function control of the computer device.

[0131] A computer device provided in this embodiment can execute the target object positioning method provided in any of the above embodiments of the present invention, and has corresponding functions and beneficial effects.

[0132] Embodiment 5

[0133] Embodiment 5 of the present invention provides a storage medium containing computer executable instructions, on which a computer program is stored. When the program is executed by a processor, a target object positioning method provided in any of the above embodiments of the present invention is implemented. The method includes:

[0134] Calculating a first depth prediction value of the target object in a camera coordinate system based on the target object detected from the environment image;

[0135] Correcting the first depth prediction value to obtain a second depth prediction value;

[0136] The coordinates of the target object in the camera coordinate system are calculated based on the center point of the target object and the second depth prediction value.

[0137] It should be noted that, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0138] Through the above description of the implementation mode, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation mode. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a robot, a personal computer, a server, or a network device, etc.) to perform the target object positioning method described in any embodiment of the present invention.

[0139] It is worth noting that in the above-mentioned device, the various modules, sub-modules and units included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0140] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0141] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0142] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for locating a target object, characterized in that: include: Calculating a first depth prediction value of the target object in the camera coordinate system based on the target object detected from the environment image, wherein the depth of the target object in the camera coordinate system is a horizontal distance between the target object and the origin of the camera coordinate system; Correcting the first depth prediction value to obtain a second depth prediction value; Calculate the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value; Correcting the first depth prediction value to obtain a second depth prediction value includes: Pre-training a correction parameter output model based on a first depth prediction value of the target object in the camera coordinate system and a known actual depth value; The first depth prediction value of the target object in the camera coordinate system is input into the trained training correction parameter output model for processing to obtain the correction parameter; wherein the correction parameter is defined as: δ = D gt / D ref , where δ represents the correction parameter, D gt Indicates the actual depth value of the target object, D ref Represents the first depth prediction value of the target object in the camera coordinate system; The first depth prediction value is corrected by using the correction parameter to obtain a second depth prediction value; wherein the correction formula is expressed as: D pre =δ*D ref , where D pre represents the second depth prediction value.

2. The target object positioning method according to claim 1, characterized in that: Calculating a first depth prediction value of the target object in a camera coordinate system based on the target object detected from the environment image, comprising: Obtain environmental images collected by the camera of the unmanned equipment; Detecting a target object from the environment image, wherein the target object is shown in the environment image with a rectangular frame; A first depth prediction value of the target object in the camera coordinate system is calculated based on the pixel coordinates of the rectangular frame and the camera parameters.

3. The target object positioning method according to claim 2, characterized in that: Calculating a first depth prediction value of the target object in a camera coordinate system based on the pixel coordinates of the rectangular frame and camera parameters includes: Subtract the offset of the optical axis of the camera relative to the coordinate center of the projection plane on the Y axis from the maximum value of the Y axis coordinate in the pixel coordinates of the rectangular frame to obtain a first value; Calculate the quotient of the focal length of the camera on the Y axis and the first value to obtain a second value; Calculate the product of the second value and the height of the camera from the ground to obtain a third value; Calculate the product of the second value and the tangent value of the inclination angle of the camera relative to the ground to obtain a fourth value; Calculating a quotient of the third value and the fourth value to obtain a fifth value; A sum of the third value and the fifth value is calculated to obtain a first depth prediction value of the target object in the camera coordinate system.

4. The target object positioning method according to any one of claims 1 to 3, characterized in that: Before calculating the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value, the method further includes: A center point of a target object is detected from the environment image.

5. The target object positioning method according to claim 4, characterized in that: Detecting a center point of a target object from the environment image includes: The CenterNet target detection model is used to process the environment image to obtain the center point of the target object.

6. The target object positioning method according to claim 4, characterized in that: Detecting a center point of a target object from the environment image includes: Pre-train the deviation output model based on the center point of the rectangular box and the center point of the known target object; Inputting the pixel coordinates of the center point of the rectangular frame into the trained deviation output model for processing, and obtaining the deviation between the center point of the rectangular frame and the center point of the target object; The center point of the rectangular frame is corrected based on the deviation to obtain the center point of the target object.

7. The target object positioning method according to any one of claims 1 to 3, characterized in that: Calculating the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value includes: Calculate the X-axis coordinate of the center point of the target object in the camera coordinate system based on the X-axis coordinate of the pixel coordinate of the center point of the target object, the camera parameter and the second depth prediction value; The Y-axis coordinate of the center point of the target object in the camera coordinate system is calculated based on the Y-axis coordinate of the pixel coordinate of the center point of the target object, the camera parameter and the second depth prediction value.

8. The target object positioning method according to claim 7, characterized in that: Calculating the X-axis coordinate of the center point of the target object in the camera coordinate system based on the X-axis coordinate of the pixel coordinate of the center point of the target object, the camera parameter, and the second depth prediction value includes: Calculate the difference between the X-axis coordinate of the pixel coordinate of the center point of the target object and the offset of the optical axis of the camera relative to the coordinate center of the projection plane on the X-axis to obtain a sixth value; Calculate the product of the sixth value and the focal length of the camera on the X-axis to obtain a seventh value; The quotient of the seventh value and the second depth prediction value is calculated to obtain the X-axis coordinate of the center point of the target object in the camera coordinate system.

9. The target object positioning method according to claim 7, characterized in that: Calculating the Y-axis coordinate of the center point of the target object in the camera coordinate system based on the Y-axis coordinate of the pixel coordinate of the center point of the target object, the camera parameter, and the second depth prediction value includes: Calculate the difference between the Y-axis coordinate of the pixel coordinate of the center point of the target object and the Y-axis offset of the optical axis of the camera relative to the coordinate center of the projection plane to obtain a ninth value; Calculate the product of the ninth value and the focal length of the camera on the Y axis to obtain a tenth value; The quotient of the tenth value and the second depth prediction value is calculated to obtain the Y-axis coordinate of the center point of the target object in the camera coordinate system.

10. A target object positioning device, characterized in that: include: A first depth prediction value calculation module, configured to calculate a first depth prediction value of the target object in a camera coordinate system based on the target object detected from the environment image, wherein the depth of the target object in the camera coordinate system is a horizontal distance between the target object and the origin of the camera coordinate system; A correction module, used for correcting the first depth prediction value to obtain a second depth prediction value; A coordinate determination module, configured to calculate the coordinates of the target object in the camera coordinate system based on the center point of the target object and the second depth prediction value; The correction module comprises: A correction parameter output model training submodule, used to pre-train a correction parameter output model based on a first depth prediction value of a target object in a camera coordinate system and a known actual depth value; The correction parameter calculation submodule is used to input the first depth prediction value of the target object in the camera coordinate system into the trained training correction parameter output model for processing to obtain the correction parameter; wherein the definition of the correction parameter is expressed as: δ = D gt / D ref , where δ represents the correction parameter, D gt Indicates the actual depth value of the target object, D ref Represents the first depth prediction value of the target object in the camera coordinate system; A correction submodule is used to correct the first depth prediction value using the correction parameter to obtain a second depth prediction value; wherein the correction formula is expressed as: D pre =δ*D ref , where D pre represents the second depth prediction value.

11. A computer device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the target object positioning method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the target object positioning method as described in any one of claims 1 to 9 is implemented.

Images