Mobile equipment pose control method and device, equipment and storage medium

Point cloud images and real scene images are obtained through radar and cameras, and combined with image library to screen matching reference images, the precise posture control of the automatic guided car during material docking is realized, and the problem of insufficient docking accuracy in the existing technology is solved. .

CN119006596APending Publication Date: 2024-11-22GUANGDONG LEAD INTELLIGENT LOGISTICS TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411047803.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing automatic guided trolleys are difficult to achieve accurate positioning during material docking, mainly because it is difficult to achieve accurate position determination based on the planar images obtained by relying on a single camera.

Method used

The image acquisition device of the radar and camera is configured to obtain point cloud images and real scene images, and the point cloud images are used to determine the relative position information of the device to be connected to the mobile device, and the matching reference images are filtered from the preset image library to control the mobile device. pose for accurate docking.

Benefits of technology

It improves the docking accuracy between mobile devices and devices to be connected, reduces the use of computing resources, and improves the docking accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006596A_ABST
    Figure CN119006596A_ABST
Patent Text Reader

Abstract

The invention relates to a mobile equipment pose control method and device, equipment and a storage medium, and belongs to the technical field of automatic guide trolleys, and the method comprises the steps: obtaining a point cloud image and a live-action image of a marker on to-be-docked equipment through an image collection device configured by the mobile equipment; according to the point cloud image, determining first position information of the to-be-docked device relative to the mobile device; determining a matched first reference image from reference images in a preset image library according to the first position information and the real scene image; and controlling the pose of the mobile device according to the second position information, reflected by the first reference image, of the to-be-docked device relative to the mobile device.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of automatic guided vehicles, and more specifically, to a method, apparatus, device and storage medium for controlling the position and posture of a mobile device. Background Art

[0002] At present, as one of the important tools for material handling and transportation, AGVs are not only required to transport materials from the starting point to each node in the end point, but also to accurately pick up and place materials from the docking points of each node. Existing AGVs often rely on a single camera to obtain real-life images, and use the real-life images to determine the relative position of the docking points of each node and the AGV. Since the real-life images are flat images, it is difficult to achieve accurate positioning. Summary of the invention

[0003] One purpose of the embodiments of the present disclosure is to provide a new technical solution for controlling the posture of a mobile device.

[0004] According to a first aspect of the present disclosure, a method for controlling a posture of a mobile device is provided, the method comprising:

[0005] The image acquisition device configured on the mobile device is used to obtain a point cloud image and a real scene image of the marker on the device to be connected;

[0006] Determine first position information of the device to be docked relative to the mobile device according to the point cloud image;

[0007] Determine a matching first reference image from reference images in a preset image library according to the first position information and the real scene image;

[0008] The posture of the mobile device is controlled according to the second position information of the device to be docked and the mobile device relative to each other reflected by the first reference image.

[0009] Optionally, the marker on the device to be docked is a reflective marker, and a geometric center of the reflective marker is associated with a docking position of the device to be docked.

[0010] Optionally, the image acquisition device includes a radar and a camera; the image acquisition device configured by the mobile device acquires a point cloud image and a real scene image where a marker on the device to be docked is located, including:

[0011] Acquire a point cloud image of a marker on the device to be docked by the radar, and determine the acquisition time of the point cloud image;

[0012] A real scene image is screened out from each image captured by the camera; wherein the difference between the capture time of the real scene image and the acquisition time is less than or equal to the difference between the capture time and the acquisition time in each image captured by the camera.

[0013] Optionally, determining first position information of the device to be docked relative to the mobile device according to the point cloud image includes:

[0014] Determine, according to the point cloud image, third position information of the device to be docked relative to the mobile device in the radar coordinate system;

[0015] According to the set first conversion relationship and the third position information, the first position information of the device to be docked and the mobile device relative to each other in the camera coordinate system is determined; wherein the first conversion relationship reflects that the coordinate positions of different points in the radar coordinate system correspond to the coordinate positions of different points in the camera coordinate system.

[0016] Optionally, determining a matching first reference image from reference images in a preset image library according to the first position information and the real scene image includes:

[0017] According to the first position information, a target area meeting a set condition is screened out in the real scene image;

[0018] A first reference image matching the target area is determined from reference images in a preset image library.

[0019] Optionally, controlling the posture of the mobile device according to the second position information of the device to be docked relative to the mobile device reflected by the first reference image includes:

[0020] Determine fourth position information of the device to be docked and the mobile device relative to each other in the vehicle coordinate system according to the second position information of the device to be docked and the mobile device reflected by the first reference image and the second conversion relationship corresponding to the first reference image; wherein the second conversion relationship reflects that different points of the camera coordinate system correspond to different points of the vehicle coordinate system;

[0021] The posture of the mobile device is controlled according to the position coordinates and the deflection angle of the device to be docked reflected by the fourth position information.

[0022] According to a second aspect of the present disclosure, there is also provided a device for controlling the posture of a mobile device, comprising:

[0023] An image acquisition module is used to acquire a point cloud image and a real scene image of a marker on the device to be docked through an image acquisition device configured on the mobile device;

[0024] An information determination module, configured to determine first position information of the device to be docked relative to the mobile device according to the point cloud image;

[0025] An image determination module, configured to determine a matching first reference image from reference images in a preset image library according to the first position information and the real scene image;

[0026] The device control module is used to control the posture of the mobile device according to the second position information of the device to be docked and the mobile device relative to each other reflected by the first reference image.

[0027] According to a third aspect of the present disclosure, an electronic device is also provided, including a processor, wherein the processor is connected to a memory, and the memory is used to store a computer program; the processor is used to execute the computer program to implement the method described in the first aspect of the present disclosure.

[0028] According to a fourth aspect of the present disclosure, a mobile device is further provided, comprising the mobile device as described in the second aspect or the electronic device as described in the third aspect.

[0029] According to a fifth aspect of the present disclosure, a computer storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0030] A beneficial effect of the embodiments of the present disclosure is that the method for controlling the posture of a mobile device provided by the present invention can obtain a point cloud image and a real scene image through an image acquisition device, use the point cloud image to determine the first relative position information of the device to be docked and the mobile device, and then use the first position information and the real scene image to screen a matching first reference image in an image library, use the first reference image to control the posture of the mobile device, and use the point cloud image and the real scene image to merge, and use the image library to screen the matching first reference image to control the posture of the mobile device. This can reduce the computing resources occupied by the mobile device for determining the docking posture with the device to be docked, while improving the docking accuracy between the mobile device and the device to be docked.

[0031] Other features and advantages of the embodiments of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure.

[0033] Figure 1is a schematic diagram of the composition structure of a control system for the position and posture of a mobile device to which a method for controlling the position and posture of a mobile device according to an embodiment can be applied;

[0034] Figure 2 is a flow chart of a method for controlling a posture of a mobile device according to an embodiment;

[0035] Figure 3 is a schematic diagram of the hardware structure of a device for controlling the posture of a mobile device according to an embodiment;

[0036] Figure 4 is a block diagram of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0037] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0038] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0039] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0040] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0041] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0042] <System Embodiment>

[0043] Figure 1 1 is a schematic diagram of the composition structure of a method for controlling the position and posture of a mobile device to which a control system for the position and posture of a mobile device according to an embodiment can be applied. Figure 1 As shown, the docking system for mobile devices includes a mobile device 10 and a device to be docked 20 .

[0044] The above mobile device 10 can be an automated guided vehicle (AGV), which is equipped with a radar and a camera. The radar is, for example, a laser radar, and the camera is, for example, a camera. The laser radar and the camera can be set on the body of the automated guided vehicle, and the laser radar and the camera can be set towards the forward direction of the automated guided vehicle.

[0045] The above-mentioned device 20 to be docked is, for example, a production line or a cache rack. The device 20 to be docked is provided with a corresponding docking position for the above-mentioned mobile device 10 to dock. The docking position can be a point on the output port or the input port. The point can be a center point or a point at a specific distance from the center point, which is not limited here. An identification mark 21 is attached to each of the two specific legs of the device 20 to be docked. The midpoint of the line connecting the two identification marks 21 corresponds to the docking position of the device 20 to be docked. That is to say, after determining the midpoint of the line connecting the two identification marks 21, the mobile device 10 can accurately dock with the device 20 to be docked.

[0046] As used in the embodiments of the present disclosure, the memory of the mobile device 10 is used to store a computer program, which is used to control the processor of the mobile device 10 to operate according to the control method of the mobile device posture of any embodiment. A technician can design a computer program according to the scheme of the embodiments of the present disclosure. How the computer program controls the processor to operate is well known in the art, so it will not be described in detail here.

[0047] <Method Example>

[0048] Figure 2 FIG. 1 is a flow chart of a method for controlling the posture of a mobile device according to an embodiment. Figure 1 The mobile device 10.

[0049] like Figure 2 As shown, the method for controlling the posture of a mobile device in this embodiment may include the following steps S210 to S240:

[0050] Step S210, obtaining a point cloud image and a real scene image of the marker on the device to be docked by using an image acquisition device configured in the mobile device.

[0051] In some examples, such as Figure 1 As shown, the markers are markers 21 attached to the two legs of the device 20 to be docked. The mobile device can recognize the two markers 21 through the configured image acquisition device and obtain a point cloud image and a real scene image containing the two markers 21.

[0052] In some embodiments, the marker on the device to be docked is a reflective marker, and the geometric center of the reflective marker is associated with the docking device of the device to be docked.

[0053] In the present application, one or more reflective markers may be provided. When there is one reflective marker, the geometric center of the reflective marker corresponds to the docking position of the device to be docked. When there are two reflective markers, the midpoint of the line connecting the geometric centers of the two reflective markers corresponds to the docking position of the device to be docked. When there are multiple reflective markers, the center point of the graphic formed by the geometric centers of the multiple reflective markers corresponds to the docking position of the device to be docked. In other words, by providing reflective markers on the legs of the device to be docked, the image acquisition device of the mobile device can quickly identify them with higher accuracy, so as to achieve precise docking between the mobile device and the device to be docked.

[0054] In some examples, such as Figure 1 As shown, the mobile device 10 can capture two reflective markers 21 located on the legs of the device to be docked 20 through the configured image acquisition device, and determine the midpoint of the line connecting the geometric centers of the two reflective markers 21 to determine the docking position of the device to be docked 20.

[0055] In some embodiments, the image acquisition device includes a radar and a camera, and step S210 may include the following steps S310 and S320:

[0056] Step S310, obtaining a point cloud image of the marker on the device to be docked through radar, and determining the acquisition time of the point cloud image.

[0057] In some examples, the radar can send some point cloud images to the mobile device, and the radar can also send the acquisition time of these point cloud images to the mobile device. The mobile device can store these point cloud images fed back by the radar, and the mobile device associates these point cloud images with the corresponding acquisition time and stores them.

[0058] Step S320, screening out real scene images from among the images captured by the camera; wherein the difference between the capture time and the acquisition time of the real scene images is less than or equal to the difference between the capture time and the acquisition time of the images captured by the camera.

[0059] In some examples, the camera can send images to a mobile device, and the camera can also send the acquisition time of each image to the mobile device. The mobile device can store the images fed back by the camera, and the mobile device can store these images in association with the corresponding acquisition time. In addition, the mobile device can determine the difference between the acquisition time and the acquisition time corresponding to each image, and obtain the image with the smallest difference among these images as the real scene image.

[0060] In some examples, the sampling frequency of the radar and the camera can be set to 15 Hz, and the radar and the camera can work synchronously to feed back the point cloud image and the real scene image to the mobile device.

[0061] In other words, by determining the minimum value of the difference between the acquisition time and the acquisition time in each image captured by the camera, that is, the difference between the acquisition time and the acquisition time of the real scene image is less than or equal to the difference between the acquisition time and the acquisition time in each image captured by the camera, a real scene image with the minimum time difference with the point cloud image can be obtained, so as to improve the temporal synchronization between the point cloud image and the real scene image.

[0062] Step S220: determining first position information of the device to be docked relative to the mobile device according to the point cloud image.

[0063] In some examples, the first position information may be filtering out the high-reflective point cloud in the point cloud image based on the strong reflective characteristics of the reflective marker, and filtering out the location area where the marker is located through the high-reflective point cloud and the point cloud boundary of a specific shape. The marker may be a leg to which the reflective marker is attached, the point cloud boundary may be a point cloud surrounding the high-reflective point cloud, and the specific shape may be a bar, a rectangle, or a square.

[0064] In some embodiments, step S220 may include the following steps S410 and S420:

[0065] Step S410: Determine third position information of the device to be docked relative to the mobile device in the radar coordinate system according to the point cloud image.

[0066] In some examples, the third location information may include a first relative distance between the mobile device and the marker and a second relative distance between the markers. Figure 1 As shown, the first relative distance may be the straight-line distance between the midpoint of the line connecting the two marks 21 and the mobile device 10, and the second relative distance may be the straight-line distance between the two marks 21. The mark 21 may be represented by the center point of the high-reflection point cloud cluster formed by the mark 21.

[0067] Step S420, determining the first position information of the device to be docked and the mobile device relative to each other in the camera coordinate system according to the set first conversion relationship and the third position information; wherein the first conversion relationship reflects that the coordinate positions of different points in the radar coordinate system correspond to the coordinate positions of different points in the camera coordinate system.

[0068] In some examples, the coordinates of the mobile device may be the origin, that is, in the radar coordinate system, the coordinate position of the device to be docked can represent the third position information, and in the camera coordinate system, the coordinate position of the device to be docked can represent the first position information. By setting the external parameter calibration, the first conversion relationship T between the radar coordinate system and the camera coordinate system can be obtained. LC Accordingly, the first position information, namely P C =T LC *P l , where P l It is the third position information.

[0069] In other words, by setting the first conversion relationship, the mobile device, radar and camera can be spatially synchronized to further improve the docking accuracy between the mobile device and the device to be docked.

[0070] In some examples, the mobile device may determine fifth position information of the device to be docked relative to the mobile device in the vehicle coordinate system according to the set first conversion relationship and third conversion relationship.

[0071] In this application, by setting the radar calibration of the AGV navigation system, the external parameter calibration of the radar and the vehicle body motion center can be obtained, that is, the third transformation relationship T between the radar coordinate system and the vehicle body coordinate system LB Accordingly, the mobile device can determine the fifth position information of the device to be docked and the mobile device relative to each other in the vehicle body coordinate system according to the point cloud image, that is, the fifth position information

[0072] Step S230: Determine a matching first reference image from reference images in a preset image library according to the first position information and the real scene image.

[0073] In the present application, the mobile device may pre-store an image library, and the image library may store different reference images, wherein the reference images reflect the legs of the device to be docked with a mark attached thereto, photographed at different positions.

[0074] In some embodiments, step S230 may include the following steps S510 and S520:

[0075] Step S510: Filter out a target area that meets a set condition in the real scene image according to the first position information.

[0076] In the present application, the ROI area containing the marker, i.e., the target area, can be screened out from the real-scene image through the first position information of the device to be docked and the mobile device relative to each other in the camera coordinate system. The set condition is, for example, the area of ​​the legs containing two reflective markers in a real-scene image.

[0077] Step S520: Determine a first reference image that matches the target area from reference images in a preset image library.

[0078] In the present application, a first reference image with the smallest error and a second conversion relationship corresponding to the first reference image are screened out from a preset image library by using an iterative closest point (ICP) algorithm.

[0079] In other words, by comparing the target area with the reference image in the image library, the computing resources occupied by the mobile device for determining the docking posture with the device to be docked can be reduced.

[0080] Step S240: Control the posture of the mobile device according to the second position information of the device to be docked and the mobile device relative to each other reflected by the first reference image.

[0081] In the present application, the posture of a mobile device refers to the position and posture of the mobile device, wherein the position of the mobile device may be the actual position of the mobile device, and the posture of the mobile device may be the deviation angle of the mobile device relative to the device to be docked.

[0082] In some embodiments, step S240 may include the following steps S610 and S620:

[0083] Step S610: Determine fourth position information of the device to be docked and the mobile device relative to each other in the vehicle coordinate system according to the second position information of the device to be docked and the mobile device relative to each other reflected by the first reference image and the second conversion relationship corresponding to the first reference image.

[0084] In some examples, when the coordinates of the mobile device are the origin, the fourth position information may be the coordinates P of the docking position. R , the coordinates of the docking position Among them, P T is the second position information, T TR is a second transformation relationship corresponding to the first reference image, and the second transformation relationship reflects that different points in different camera coordinate systems correspond to different points in different vehicle body coordinate systems.

[0085] Step S620: Control the posture of the mobile device according to the position coordinates and the deflection angle of the device to be docked reflected by the fourth position information.

[0086] In the present application, when the fourth position information indicates that the position of the mobile device is offset, the mobile device can control the movement and / or steering of the mobile device so that the mobile device is realigned with the device to be docked. Calibrating the movement path of the mobile device based on the fourth position information is a prior art and will not be elaborated herein.

[0087] In other words, the fourth position information can represent the position coordinates and deflection angle of the device to be docked in the vehicle coordinate system, and can control the mobile device to move a certain distance or rotate a certain angle, so that the mobile device can accurately dock with the device to be docked.

[0088] <Equipment Example 1>

[0089] Figure 3 is a principle block diagram of a device for controlling the posture of a mobile device according to an embodiment. Figure 3 As shown, the control device 300 for the mobile device posture may include:

[0090] The image acquisition module 310 is used to acquire a point cloud image and a real scene image of the marker on the device to be docked through an image acquisition device configured on the mobile device;

[0091] An information determination module 320, configured to determine first position information of the device to be docked relative to the mobile device according to the point cloud image;

[0092] An image determination module 330 is used to determine a matching first reference image from reference images in a preset image library according to the first position information and the real scene image;

[0093] The device control module 340 is used to control the posture of the mobile device according to the second position information of the device to be docked and the mobile device relative to each other reflected by the first reference image.

[0094] Optionally, the image acquisition module 310 is also used to acquire, through radar, a point cloud image of the marker on the device to be docked, and to determine the acquisition time of the point cloud image; and to filter out real-scene images from among the images captured by the camera; wherein the difference between the acquisition time and the acquisition time of the real-scene image is less than or equal to the difference between the acquisition time and the acquisition time in each image captured by the camera.

[0095] Optionally, the information determination module 320 is also used to determine the third position information of the device to be docked and the mobile device relative to each other in the radar coordinate system based on the point cloud image; determine the first position information of the device to be docked and the mobile device relative to each other in the vehicle coordinate system based on the set first conversion relationship and the third position information; wherein the first conversion relationship reflects that different points of the radar coordinate system correspond to different points of the vehicle coordinate system.

[0096] Optionally, the image determination module 330 is further configured to screen out a target area meeting set conditions in the real scene image according to the first position information; and determine a first reference image matching the target area from reference images in a preset image library.

[0097] Optionally, the device control module 340 is also used to determine the fourth position information of the device to be docked and the mobile device relative to each other in the vehicle coordinate system based on the second position information of the device to be docked and the mobile device relative to each other reflected by the first reference image and the second transformation relationship corresponding to the first reference image; and control the posture of the mobile device based on the position coordinates and deflection angle of the device to be docked reflected by the fourth position information.

[0098] The control device 300 of the mobile device posture can be Figure 1 A mobile device 10 is provided.

[0099] <Equipment Example 2>

[0100] Figure 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment.

[0101] like Figure 4 As shown, the electronic device 400 includes a processor 410 and a memory 420, wherein the memory 420 is used to store an executable computer program, and the processor 410 is used to execute a method such as any of the above method embodiments under the control of the computer program.

[0102] The electronic device 400 may be Figure 1 The controller 50 in.

[0103] Each module of the above-mentioned mobile device posture control device 300 can be implemented by the processor 410 in this embodiment executing a computer program stored in the memory 420, or can be implemented by other structures, which is not limited here.

[0104] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0105] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0106] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0107] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0108] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0109] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0110] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0111] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0112] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A method for controlling the posture of a mobile device, characterized in that: The method comprises: The image acquisition device configured on the mobile device is used to obtain a point cloud image and a real scene image of the marker on the device to be connected; Determine first position information of the device to be docked relative to the mobile device according to the point cloud image; Determine a matching first reference image from reference images in a preset image library according to the first position information and the real scene image; The posture of the mobile device is controlled according to the second position information of the device to be docked and the mobile device relative to each other reflected by the first reference image.

2. The method according to claim 1, characterized in that The marker on the device to be docked is a reflective marker, and the geometric center of the reflective marker is associated with the docking position of the device to be docked.

3. The method according to claim 1, characterized in that The image acquisition device includes a radar and a camera; the image acquisition device configured by the mobile device obtains a point cloud image and a real scene image where a marker on the device to be docked is located, including: Acquire a point cloud image of a marker on the device to be docked by the radar, and determine the acquisition time of the point cloud image; A real scene image is screened out from each image captured by the camera; wherein the difference between the capture time of the real scene image and the acquisition time is less than or equal to the difference between the capture time and the acquisition time in each image captured by the camera.

4. The method according to claim 3, characterized in that The determining, according to the point cloud image, first position information of the device to be docked relative to the mobile device includes: Determine, according to the point cloud image, third position information of the device to be docked relative to the mobile device in the radar coordinate system; According to the set first conversion relationship and the third position information, the first position information of the device to be docked and the mobile device relative to each other in the camera coordinate system is determined; wherein the first conversion relationship reflects that the coordinate positions of different points in the radar coordinate system correspond to the coordinate positions of different points in the camera coordinate system.

5. The method according to any one of claims 1 to 4, characterized in that: The determining, according to the first position information and the real scene image, a matching first reference image from each reference image in a preset image library includes: According to the first position information, a target area meeting a set condition is screened out in the real scene image; A first reference image matching the target area is determined from reference images in a preset image library.

6. The method according to claim 5, characterized in that The controlling the posture of the mobile device according to the second position information of the device to be docked relative to the mobile device reflected by the first reference image includes: Determine fourth position information of the device to be docked and the mobile device relative to each other in the vehicle coordinate system according to the second position information of the device to be docked and the mobile device reflected by the first reference image and the second conversion relationship corresponding to the first reference image; wherein the second conversion relationship reflects that different points of the camera coordinate system correspond to different points of the vehicle coordinate system; The posture of the mobile device is controlled according to the position coordinates and the deflection angle of the device to be docked reflected by the fourth position information.

7. A device for controlling the posture of a mobile device, characterized in that: include: An image acquisition module is used to acquire a point cloud image and a real scene image of a marker on the device to be docked through an image acquisition device configured on the mobile device; An information determination module, configured to determine first position information of the device to be docked relative to the mobile device according to the point cloud image; An image determination module, configured to determine a matching first reference image from reference images in a preset image library according to the first position information and the real scene image; The device control module is used to control the posture of the mobile device according to the second position information of the device to be docked and the mobile device relative to each other reflected by the first reference image.

8. An electronic device, characterized in that: The method comprises a processor connected to a memory, wherein the memory is used to store a computer program; the processor is used to execute the computer program to implement the method according to any one of claims 1 to 6.

9. A mobile device, characterized in that: A docking device for a mobile device as claimed in claim 7 or an electronic device as claimed in claim 8.

10. A computer storage medium, characterized in that: The computer storage medium stores a computer-readable program, and the computer-readable program is executed by a processor to perform the steps of the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • AR display method and device, electronic equipment and storage medium

    CN114067087A

  • Point cloud registration method and device, equipment and storage medium

    CN115311337A

  • Detection method and device based on robot inspection and storage medium

    CN117103262A

  • Method and apparatus for displaying images, and electronic device

    WO2023151558A1