Roadway roof tray three-dimensional space auxiliary positioning method based on two-dimensional target detection result

Abstract

The invention provides a roadway roof tray three-dimensional space auxiliary positioning method based on a two-dimensional target detection result, and the method comprises the steps: obtaining an image of a roof tray, detecting a pixel bounding box of the tray in an original image, and calculating a three-dimensional ray vector corresponding to the center point of each detection target based on the internal reference of a camera; generating a plurality of three-dimensional candidate frames according to the actual size of the top plate tray and a preset candidate depth set, and calculating four angular point coordinates of the corresponding three-dimensional candidate frames in a camera coordinate system; reversely projecting the four angular point coordinates into a pixel coordinate system of the image through internal reference of the camera to form a pixel candidate frame; and calculating the intersection-to-union ratio between each pixel candidate frame and the original detection frame, and selecting the central point of the three-dimensional candidate frame corresponding to the pixel candidate frame with the maximum intersection-to-union ratio as a three-dimensional space positioning result. According to the method, centimeter-level three-dimensional space positioning precision of the roadway roof tray can be realized under the condition that only a low-cost camera and a two-dimensional target detection result are used.

Description

Technical Field

[0001] This invention relates to the fields of intelligent mining and computer vision technology, and in particular to a three-dimensional spatial auxiliary positioning method for roadway roof trays based on two-dimensional target detection results. Background Technology

[0002] This research developed a 3D-space assisted positioning method for a top-mounted pallet based on 2D target detection results. The method features high positioning accuracy, fast positioning speed, and low equipment cost. A robust target detection method is used to detect target objects in the image and obtain their pixel coordinates. By utilizing camera intrinsics, pallet dimensions, and perspective algorithms, the target's position in 3D space is accurately located, achieving real-time assisted positioning of the target object.

[0003] Current methods for monitoring surface displacement in tunnels have the following two shortcomings: 1. LiDAR-based ranging directly captures the coordinates of a target object in three-dimensional space. However, this method often fails to capture sufficient visual information, leading to misjudgments of the target object and affecting its positioning accuracy. 2. A method based on integrated radar-camera positioning, where information is captured jointly by lidar and camera, and directly combined with 3D target detection, is used to achieve positioning. However, this method suffers from problems such as difficulty in aligning the radar and camera coordinate systems, high computational resources, and high equipment costs, resulting in substantial cost investment. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose a three-dimensional spatial assisted positioning method for a roadway roof tray based on two-dimensional target detection results.

[0006] The second objective of this invention is to propose a three-dimensional spatial auxiliary positioning device for a roadway roof tray based on two-dimensional target detection results.

[0007] The third objective of this invention is to provide an electronic device.

[0008] The fourth objective of this invention is to provide a computer-readable storage medium.

[0009] The fifth objective of this invention is to provide a computer program product.

[0010] To achieve the above objectives, a first aspect of the present invention proposes a three-dimensional spatial assisted positioning method for a roadway roof tray based on two-dimensional target detection results, comprising: Images of the roof tray are acquired by a low-cost camera installed in the tunnel. The pixel bounding box of the tray in the original image is detected by a trained robust two-dimensional target detector. The three-dimensional ray vector corresponding to the center point of each detected target in the image is calculated based on the camera intrinsic parameters. Based on the actual size of the top plate tray and the preset candidate depth set, multiple 3D candidate boxes are generated, and the coordinates of the four corner points of the 3D candidate box corresponding to each candidate depth in the camera coordinate system are calculated. The coordinates of the four corner points of each 3D candidate box are back-projected into the pixel coordinate system of the image through the camera intrinsic parameters to form the corresponding pixel candidate box; Calculate the cross-union ratio (CUP) between each candidate pixel and the original detection box, and select the center point of the 3D candidate box corresponding to the candidate pixel with the largest CUP as the 3D spatial positioning result of the top plate tray.

[0011] Optional, the camera intrinsic parameters of a low-cost camera are denoted as:

[0012] The original image captured is denoted as .

[0013] Optional, trained robust 2D object detector Used for detection The pixel position of the target tray in the image is denoted as:

[0014] in, N represents The number of targets detected in the middle, This represents the top-left corner of the calibration frame for the nth object. This represents the bottom right corner of the nth object.

[0015] Optionally, the three-dimensional ray vector corresponding to the center point of each detected target in the image is calculated based on camera intrinsic parameters, including: right Target pallet label box , obtain center point ,Right now and ; Based on camera participation center point The ray vector for calculating the coordinates of the center point is expressed as:

[0016] in, Indicates the depth of the center point in three-dimensional space. and express The unit vector.

[0017] Optionally, based on the actual dimensions of the top tray and a preset set of candidate depths, multiple 3D candidate bounding boxes are generated, and the coordinates of the four corner points of the 3D candidate bounding box corresponding to each candidate depth in the camera coordinate system are calculated, including: Given a specific set of candidate depth values, For each depth The coordinates of the corresponding center point are represented as follows:

[0018] Assuming all vertices of the top tray have the same vertical coordinate, the coordinates of the four corner points of the top tray are represented as follows:

[0019]

[0020]

[0021]

[0022] in, These represent the coordinates of the top left, bottom left, top right, and bottom right corners of the tray's 3D frame, respectively. Indicates the actual radius of the pallet; by For each depth, candidate locations in the three-dimensional space corresponding to the target pallet are obtained. .

[0023] Optionally, the coordinates of the four corner points of each 3D candidate box are back-projected into the pixel coordinate system of the image using camera intrinsics to form the corresponding pixel candidate box, including: Based on camera intrinsic parameters, Corner points of each candidate box in Reproject to In the pixel coordinate system, it is denoted as:

[0024] in, , They represent The horizontal and vertical coordinates on the horizontal plane.

[0025] Optionally, the intersection-union ratio (CUI) between each candidate pixel and the original detection box is calculated, and the center point of the 3D candidate box corresponding to the candidate pixel with the largest CUI is selected as the 3D spatial localization result of the top plate tray, including: By calculating the projection frame Intersection over Union (IOU) with the ground truth bounding boxes; select the center point of the candidate bounding box with the highest IOU. This refers to the three-dimensional positioning of the tray, and the spatial distance between the tray and the camera is calculated. .

[0026] To achieve the above objectives, a second aspect of the present invention provides a three-dimensional spatial auxiliary positioning device for a roadway roof tray based on two-dimensional target detection results, comprising: The first execution module is used to acquire images of the top plate tray through a low-cost camera installed in the tunnel, detect the pixel bounding box of the tray in the original image through a trained robust two-dimensional target detector, and calculate the three-dimensional ray vector corresponding to the center point of each detected target in the image based on the camera intrinsic parameters. The second execution module is used to generate multiple 3D candidate boxes based on the actual size of the top plate tray and the preset candidate depth set, and to calculate the coordinates of the four corner points of the 3D candidate box corresponding to each candidate depth in the camera coordinate system. The third execution module is used to back-project the coordinates of the four corner points of each 3D candidate box to the pixel coordinate system of the image through the camera intrinsic parameters, so as to form the corresponding pixel candidate box. The fourth execution module is used to calculate the cross-union ratio (CUP) between each pixel candidate box and the original detection box, and select the center point of the 3D candidate box corresponding to the pixel candidate box with the largest CUP as the 3D spatial positioning result of the top plate tray.

[0027] To achieve the above objectives, a third aspect of the present invention provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of the first aspects.

[0028] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of the first aspects.

[0029] To achieve the above objectives, a fifth aspect of the present invention provides a computer program product that, when executed by a processor, implements the method described in any one of the first aspects.

[0030] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects: it can effectively locate the position of the target tray in three-dimensional space with only planar image information and available information for two-dimensional target detection, and control the positioning error at the centimeter level, thus meeting the spatial positioning requirements of the target tray.

[0031] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0032] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating a three-dimensional spatial assisted positioning method for a roadway roof tray based on two-dimensional target detection results, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a three-dimensional spatial auxiliary positioning device for a roadway roof tray based on two-dimensional target detection results, provided in an embodiment of the present invention. Detailed Implementation

[0033] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0034] Figure 1 This is a schematic diagram illustrating a three-dimensional spatial assisted positioning process for a roadway roof tray based on two-dimensional target detection results, provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps: S1. Images of the roof tray are acquired using a low-cost camera installed in the tunnel. The pixel bounding boxes of the tray in the original image are detected by a trained robust two-dimensional target detector. The three-dimensional ray vector corresponding to the center point of each detected target in the image is calculated based on the camera intrinsic parameters.

[0035] In one embodiment of the invention, images of the roof tray are acquired using a low-cost camera installed within the tunnel. The camera is fixedly mounted on the internal structure of the tunnel and is used for continuous monitoring and image acquisition of the tunnel roof area, ensuring relatively clear image information even in complex tunnel environments. The acquired images can be color or grayscale, both of which can meet the needs of subsequent identification.

[0036] In this embodiment of the application, the camera intrinsic parameters of the low-cost camera are denoted as:

[0037] in, and These represent the focal length of the camera in the x and y directions, respectively. and These represent the principal point coordinates of the camera's optical axis on the imaging plane. This camera intrinsic parameter matrix allows us to map the image coordinates to the camera coordinate system, providing a precise geometric relationship for subsequent 3D ray calculations.

[0038] The original image captured is denoted as Utilizing a trained, robust two-dimensional target detector For images The image is processed to detect the pixel bounding box of the tray. This detection process can be implemented based on a deep learning object detection network, such as YOLO, Faster R-CNN, or other lightweight network models, to adapt to the complex lighting and dust interference environment of the alleyway. The detection result can be represented as...

[0039] in, N represents The number of targets detected in the middle, This represents the top-left corner of the calibration frame for the nth object. This represents the bottom right corner of the nth object.

[0040] In this embodiment, the three-dimensional ray vector corresponding to the center point of each detected target in the image is calculated based on camera intrinsic parameters. Specifically, for Target pallet label box First obtain center point ,Right now and The center point represents the approximate position of the tray in the image plane.

[0041] Subsequently, based on the camera's internal participation center point The ray vector for calculating the coordinates of the center point is expressed as:

[0042] in, Indicates the depth of the center point in three-dimensional space. and express The unit vector.

[0043] Through the above steps, the tray detection results in the two-dimensional image can be converted into three-dimensional ray information, thereby providing basic data support for subsequent multi-camera spatial intersection calculations or depth estimation. This application's embodiment uses a combination of a low-cost camera and a robust detection algorithm, which can significantly reduce system hardware costs while ensuring recognition accuracy, making it suitable for automated monitoring applications in coal mines, tunnels, and other confined spaces.

[0044] S2, based on the actual size of the top plate tray and the preset candidate depth set, generates multiple 3D candidate boxes, and calculates the coordinates of the four corner points of the 3D candidate box corresponding to each candidate depth in the camera coordinate system.

[0045] In this embodiment of the application, after completing the two-dimensional detection of the target pallet in the image and the three-dimensional ray direction calculation, multiple possible three-dimensional candidate boxes are generated based on the actual physical size of the top plate pallet and the preset candidate depth set, so as to achieve a preliminary estimate of the spatial position of the pallet.

[0046] First, a specific set of candidate depth values ​​is given. For each depth Using the central ray direction vector calculated in step S1 Calculate the spatial coordinates of the center point of the tray in the camera coordinate system at this depth, using the following expression:

[0047] In this embodiment, it is assumed that the top tray is an approximately planar structure, with all its vertices at the same vertical height, meaning that its four corner points have the same z-coordinate. Based on this assumption, the coordinates of the four corner points of the tray in the camera coordinate system can be derived from the actual radius r of the tray and the coordinates of its center point, as follows:

[0048]

[0049]

[0050]

[0051] in, These represent the coordinates of the top left, bottom left, top right, and bottom right corners of the tray's 3D frame, respectively. This indicates the actual radius of the pallet.

[0052] Therefore, by For each depth, a candidate position in the three-dimensional space corresponding to the target pallet can be obtained. .

[0053] Through the above calculations, this embodiment of the application realizes the generation of multiple three-dimensional spatial candidate boxes by expanding a single two-dimensional detection result, thereby providing a foundation for subsequent spatial geometric constraint calculations, viewpoint matching, or multi-view fusion. This step not only considers the correspondence between the actual geometric features of the tray and the camera imaging model, but also effectively introduces a depth candidate set to reduce the depth ambiguity problem under monocular imaging, providing a feasible search space and computational basis for subsequent precise positioning.

[0054] S3, the coordinates of the four corner points of each 3D candidate box are back-projected into the pixel coordinate system of the image through the camera intrinsic parameters to form the corresponding pixel candidate box.

[0055] In this embodiment, after generating the 3D candidate boxes, the coordinates of the four corner points of each 3D candidate box are further back-projected to the image pixel coordinate system through camera intrinsic parameters to obtain the corresponding pixel-level candidate boxes, which are then used to match and score the 2D detection results. The specific implementation is as follows: For any 3D candidate bounding box obtained in step S2, based on the camera intrinsic parameters, Corner points of each candidate box in Reproject to In the pixel coordinate system, it is denoted as:

[0056] in, , They represent The horizontal and vertical coordinates on the horizontal plane.

[0057] Through the aforementioned back projection steps, the mapping from the three-dimensional candidate space to the two-dimensional image space is completed, laying the foundation for subsequent candidate depth selection based on pixel overlap and geometric consistency, determination of the optimal three-dimensional bounding box, and multi-view fusion localization.

[0058] S4. Calculate the cross-union ratio (CUP) between each pixel candidate box and the original detection box, and select the center point of the 3D candidate box corresponding to the pixel candidate box with the largest CUP as the 3D spatial positioning result of the top plate tray.

[0059] In this embodiment of the application, the pixel candidate box set obtained in step S3 is used to evaluate the overlap with the original two-dimensional detection box in step S1 in order to determine the three-dimensional candidate depth that best matches the image observation and its corresponding three-dimensional center coordinates, thereby completing the three-dimensional spatial positioning of the top plate tray.

[0060] Specifically, by calculating the projection frame Intersection over Union (IOU) with the ground truth bounding boxes; select the center point of the candidate bounding box with the highest IOU. This refers to the three-dimensional positioning of the tray, and the spatial distance between the tray and the camera is calculated. .

[0061] Therefore, this embodiment of the application achieves precise 3D positioning of the roof tray under monocular camera conditions through a chain of "2D detection—3D candidate generation—pixel back projection—cross-union selection". This method does not require expensive sensors, but relies only on low-cost cameras and robust detection networks to obtain stable 3D positioning results in complex tunnel environments, and has good real-time performance and engineering deployment value.

[0062] To achieve the above embodiments, the present invention also proposes a three-dimensional spatial auxiliary positioning device for a roadway roof tray based on two-dimensional target detection results. Figure 2 This is a schematic diagram of a three-dimensional spatial auxiliary positioning device for a roadway roof tray based on two-dimensional target detection results, provided as an embodiment of the present invention. Figure 2 As shown, the device includes: The first execution module 100 is used to acquire images of the top plate tray through a low-cost camera installed in the tunnel, detect the pixel bounding box of the tray in the original image through a trained robust two-dimensional target detector, and calculate the three-dimensional ray vector corresponding to the center point of each detected target in the image based on the camera intrinsic parameters. The second execution module 200 is used to generate multiple three-dimensional candidate boxes based on the actual size of the top plate tray and the preset candidate depth set, and to calculate the coordinates of the four corner points of the three-dimensional candidate box corresponding to each candidate depth in the camera coordinate system. The third execution module 300 is used to back-project the coordinates of the four corner points of each 3D candidate box to the pixel coordinate system of the image through the camera intrinsic parameters, so as to form the corresponding pixel candidate box. The fourth execution module 400 is used to calculate the intersection-union ratio (IUR) between each pixel candidate box and the original detection box, and select the center point of the three-dimensional candidate box corresponding to the pixel candidate box with the largest IUR as the three-dimensional spatial positioning result of the top plate tray.

[0063] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0064] To implement the above embodiments, the present invention also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0065] To implement the above embodiments, the present invention also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.

[0066] To implement the above embodiments, the present invention also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.

[0067] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0068] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0069] This invention is intended to provide implementation schemes for users to selectively prevent the use or access to personal information data. That is, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information can be de-identified to protect user privacy.

[0070] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0071] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0072] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0073] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0074] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0075] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0076] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0077] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0078] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0079] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A three-dimensional spatial assisted positioning method for a roadway roof tray based on two-dimensional target detection results, characterized in that, include: Images of the roof tray are acquired by a low-cost camera installed in the tunnel. The pixel bounding box of the tray in the original image is detected by a trained robust two-dimensional target detector. The three-dimensional ray vector corresponding to the center point of each detected target in the image is calculated based on the camera intrinsic parameters. Based on the actual size of the top plate tray and the preset candidate depth set, multiple 3D candidate boxes are generated, and the coordinates of the four corner points of the 3D candidate box corresponding to each candidate depth in the camera coordinate system are calculated. The coordinates of the four corner points of each 3D candidate box are back-projected into the pixel coordinate system of the image through the camera intrinsic parameters to form the corresponding pixel candidate box; Calculate the cross-union ratio (CUP) between each candidate pixel and the original detection box, and select the center point of the 3D candidate box corresponding to the candidate pixel with the largest CUP as the 3D spatial positioning result of the top plate tray.

2. The method according to claim 1, characterized in that, The camera internal parameters of a low-cost camera are denoted as: The original image captured is denoted as .

3. The method according to claim 2, characterized in that, Robust 2D Object Detector Used for detection The pixel position of the target tray in the image is denoted as: in, N represents The number of targets detected in the middle, This represents the top-left corner of the calibration frame for the nth object. This represents the bottom right corner of the nth object.

4. The method according to claim 3, characterized in that, The three-dimensional ray vector corresponding to the center point of each detected target in the image is calculated based on camera intrinsic parameters, including: right Target pallet label box , obtain center point ,Right now and ; Based on camera participation center point The ray vector for calculating the coordinates of the center point is expressed as: in, Indicates the depth of the center point in three-dimensional space. and express The unit vector.

5. The method according to claim 4, characterized in that, Based on the actual dimensions of the top tray and a preset set of candidate depths, multiple 3D candidate bounding boxes are generated, and the coordinates of the four corner points of the 3D candidate bounding box corresponding to each candidate depth in the camera coordinate system are calculated, including: Given a specific set of candidate depth values, For each depth The coordinates of the corresponding center point are represented as follows: Assuming all vertices of the top tray have the same vertical coordinate, the coordinates of the four corner points of the top tray are represented as follows: in, These represent the coordinates of the top left, bottom left, top right, and bottom right corners of the tray's 3D frame, respectively. Indicates the actual radius of the pallet; by For each depth, candidate locations in the three-dimensional space corresponding to the target pallet are obtained. .

6. The method according to claim 5, characterized in that, The coordinates of the four corner points of each 3D candidate box are back-projected into the pixel coordinate system of the image using camera intrinsic parameters to form the corresponding pixel candidate box, including: Based on camera intrinsic parameters, Corner points of each candidate box in Reproject to In the pixel coordinate system, it is denoted as: in, , They represent The horizontal and vertical coordinates on the horizontal plane.

7. The method according to claim 6, characterized in that, Calculate the intersection-union ratio (IUGR) between each candidate pixel and the original detection box, and select the center point of the 3D candidate box corresponding to the candidate pixel with the largest IUGR as the 3D spatial localization result of the top plate tray, including: By calculating the projection frame Intersection over Union (IOU) with the ground truth bounding boxes; select the center point of the candidate bounding box with the highest IOU. This refers to the three-dimensional positioning of the tray, and the spatial distance between the tray and the camera is calculated. .

8. A three-dimensional spatial auxiliary positioning device for a roadway roof tray based on two-dimensional target detection results, characterized in that, include: The first execution module is used to acquire images of the top plate tray through a low-cost camera installed in the tunnel, detect the pixel bounding box of the tray in the original image through a trained robust two-dimensional target detector, and calculate the three-dimensional ray vector corresponding to the center point of each detected target in the image based on the camera intrinsic parameters. The second execution module is used to generate multiple 3D candidate boxes based on the actual size of the top plate tray and the preset candidate depth set, and to calculate the coordinates of the four corner points of the 3D candidate box corresponding to each candidate depth in the camera coordinate system. The third execution module is used to back-project the coordinates of the four corner points of each 3D candidate box to the pixel coordinate system of the image through the camera intrinsic parameters, so as to form the corresponding pixel candidate box. The fourth execution module is used to calculate the intersection-union ratio (IUR) between each pixel candidate box and the original detection box, and select the center point of the 3D candidate box corresponding to the pixel candidate box with the largest IUR as the 3D spatial positioning result of the top plate tray.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

Images