A collision detection method, apparatus and electronic device

Abstract

Embodiments of the present application provide a collision detection method and device and electronic equipment, and relate to the technical field of robots. Current position information and a carrying object height of a target robot are obtained, and a target image collected by an image collection device is obtained. When there is a suspended obstacle in the target image, it is determined that the distance between the target robot and the suspended obstacle is less than a preset distance based on the current position information, and the suspended obstacle is located above the moving path of the target robot, target size feature data of the image region where the suspended obstacle is located is determined. The target corresponding relationship between the size feature data of the suspended obstacle and the height is determined, and the current height of the suspended obstacle is determined based on the target size feature data and the target corresponding relationship. If the current height is not greater than the carrying object height, a detection result about whether there is a collision risk between the target robot and the suspended obstacle is determined. Compared with related technologies, the application of the embodiments of the present application can reduce the risk of collision.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a collision detection method, apparatus, and electronic device. Background Technology

[0002] As robotics technology matures, robots are being widely used in various work and life scenarios. For example, in freight centers, AGVs (Automated Guided Vehicles) are used to move goods; in shopping malls, robots are used to perform cleaning tasks in designated areas.

[0003] As the robot moves along the designated path, there may be suspended obstacles in its path. If the height of the suspended obstacle from the ground is less than the height of the robot, and the staff cannot clear the suspended obstacle in time, the robot may collide with the suspended obstacle, thereby causing damage to the robot.

[0004] Therefore, there is an urgent need for a collision detection method to reduce the risk of collisions during robot movement. Summary of the Invention

[0005] The purpose of this application is to provide a collision detection method, device, and electronic device to reduce the risk of robots colliding with suspended obstacles during movement. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a collision detection method, the method comprising:

[0007] Obtain the current position information of the target robot and the height of the payload, and acquire the target image captured by the image acquisition device installed at the specified location;

[0008] When there is a suspended obstacle in the target image, and the distance between the target robot and the suspended obstacle is determined to be less than a preset distance based on the current position information, and the suspended obstacle is located above the pre-acquired movement path of the target robot, the target size feature data of the image area where the suspended obstacle is located in the target image is determined.

[0009] Determine a preset target correspondence between the size feature data and height of the suspended obstacle, and determine the current height of the suspended obstacle based on the target size feature data and the target correspondence;

[0010] If the current height is not greater than the height of the transport object, then based on the movement parameters of the target robot and the processing parameters for the suspended obstacle, a detection result is determined regarding whether there is a collision risk between the target robot and the suspended obstacle; wherein, the movement parameters include: the movement speed of the target robot, and the processing parameters include: the removal progress of the suspended obstacle.

[0011] Optionally, in one specific implementation, the method further includes:

[0012] When the detection result indicates a collision risk, the movement path of the target robot is changed, and the target robot is controlled to move along the changed movement path.

[0013] Optionally, in one specific implementation, determining the preset target correspondence between the size feature data and height of the suspended obstacle includes:

[0014] The suspended obstacle in the target image is subjected to obstacle type detection to determine the obstacle type of the suspended obstacle;

[0015] Among a number of preset correspondences between the size feature data and height of different types of calibration obstacles, a correspondence between the size feature data and height of a target calibration obstacle of the same type as the obstacle is determined as the target correspondence between the size feature data and height of the suspended obstacle; wherein, the correspondence between the size feature data and height of each calibration obstacle is determined based on the size feature data of the image region where the calibration obstacle is located in each image of the calibration obstacle acquired by the image acquisition device when the calibration obstacle is at different heights.

[0016] Optionally, in one specific implementation, the installation height of the image acquisition device is higher than the maximum height of the suspended obstacle. The target correspondence between the size feature data of the suspended obstacle and the height is: the correspondence between the size feature data of the suspended obstacle and the height difference, where the height difference corresponding to each feature data is: the height difference between the installation height and the current height of the suspended obstacle when the size feature data of the image area where the suspended obstacle is located is that size feature data;

[0017] Determining the current height of the suspended obstacle based on target size feature data and the target correspondence includes:

[0018] In the target correspondence, the target height difference corresponding to the target size feature data is determined, and the difference between the installation height and the target height difference is calculated as the current height of the suspended obstacle.

[0019] Optionally, in one specific implementation, the method for establishing the correspondence between the size feature data of the suspended obstacle and the height difference includes:

[0020] Determine the installation height of the image acquisition device;

[0021] The image acquisition device acquires sample images of the target calibration obstacle at various sample heights, and determines the sample size feature data of the image region where the target calibration obstacle is located in each sample image; wherein, the sample size feature data includes: the number of pixels;

[0022] For each sample height, the height difference between the installation height and the sample height is calculated, and based on each height difference and the sample size feature data under each height difference, a correspondence between the size feature data of the suspended obstacle and the height difference is established;

[0023] Among them, the sample size feature data under each height difference is: when the height difference between the sample height where the target calibration obstacle is located and the installation height is the height difference, the sample size feature data of the image area where the target calibration obstacle is located in the sample image of the target calibration obstacle acquired by the image acquisition device.

[0024] Optionally, in one specific implementation, establishing the correspondence between the size feature data of the suspended obstacle and the height differences based on each height difference and the sample size feature data under each height difference includes:

[0025] The initial model is trained based on the height differences and the sample size feature data under each height difference.

[0026] When the initial model meets the preset conditions, training stops, and an obstacle height detection model is obtained, which serves as the correspondence between the size feature data and the height difference of the suspended obstacle.

[0027] In the target correspondence, determining the target height difference corresponding to the target size feature data includes:

[0028] The target size feature data is input into the obstacle height detection model, and the height detection result output by the obstacle height detection model is obtained as the target height difference corresponding to the target size feature data.

[0029] Optionally, in one specific implementation, the method for detecting whether there is a suspended obstacle in the target image includes:

[0030] Feature extraction is performed on the target image to obtain the target image features;

[0031] If the target image features match the reference image features of the reference image, then it is determined that the suspended obstacle exists in the target image;

[0032] The reference image is an image of the suspended obstacle captured by the image acquisition device when the suspended obstacle exists within the acquisition area of ​​the image acquisition device.

[0033] Secondly, embodiments of this application provide a collision detection device, the device comprising:

[0034] The location information acquisition module is used to acquire the current location information of the target robot and the height of the transported object, and to acquire the target image captured by the image acquisition device installed at the specified location;

[0035] The feature data determination module is used to determine the target size feature data of the image area where the suspended obstacle is located in the target image when there is a suspended obstacle in the target image, the distance between the target robot and the suspended obstacle is less than a preset distance based on the current position information, and the suspended obstacle is located above the pre-acquired movement path of the target robot.

[0036] The height determination module is used to determine a preset target correspondence between the size feature data of the suspended obstacle and its height, and to determine the current height of the suspended obstacle based on the target size feature data and the target correspondence.

[0037] The result determination module is used to determine, based on the target robot's movement parameters and the processing parameters for the suspended obstacle, whether there is a collision risk between the target robot and the suspended obstacle if the current height is not greater than the height of the carrier; wherein the movement parameters include the target robot's movement speed, and the processing parameters include the removal progress of the suspended obstacle.

[0038] Optionally, in one specific implementation, the apparatus further includes:

[0039] The path changing module is used to change the movement path of the target robot when the detection result indicates that there is a collision risk, and to control the target robot to move along the changed movement path.

[0040] Optionally, in one specific implementation, the height determination module is specifically used for:

[0041] The suspended obstacle in the target image is subjected to obstacle type detection to determine the obstacle type of the suspended obstacle;

[0042] Among a number of preset correspondences between the size feature data and height of different types of calibration obstacles, a correspondence between the size feature data and height of a target calibration obstacle of the same type as the obstacle is determined as the target correspondence between the size feature data and height of the suspended obstacle; wherein, the correspondence between the size feature data and height of each calibration obstacle is determined based on the size feature data of the image region where the calibration obstacle is located in each image of the calibration obstacle acquired by the image acquisition device when the calibration obstacle is at different heights.

[0043] Optionally, in one specific implementation, the installation height of the image acquisition device is higher than the maximum height of the suspended obstacle. The target correspondence between the size feature data of the suspended obstacle and the height is: the correspondence between the size feature data of the suspended obstacle and the height difference, where the height difference corresponding to each feature data is: the height difference between the installation height and the current height of the suspended obstacle when the size feature data of the image area where the suspended obstacle is located is that size feature data;

[0044] The height determination module includes:

[0045] The target height difference determination submodule is used to determine the target height difference corresponding to the target size feature data in the target correspondence relationship, and calculate the difference between the installation height and the target height difference as the current height of the suspended obstacle.

[0046] Optionally, in one specific implementation, the device further includes a correspondence establishment module, which includes:

[0047] The installation height determination submodule is used to determine the installation height of the image acquisition device;

[0048] The sample image acquisition submodule is used to acquire various sample images of the target calibration obstacle collected by the image acquisition device when the target calibration obstacle is located at various sample heights, and to determine the sample size feature data of the image region where the target calibration obstacle is located in each sample image; wherein, the sample size feature data includes: the number of pixels;

[0049] The relationship establishment submodule is used to calculate the height difference between the installation height and the sample height for each sample height, and establish a correspondence between the size feature data of the suspended obstacle and the height difference based on each height difference and the sample size feature data under each height difference;

[0050] Among them, the sample size feature data under each height difference is: when the height difference between the sample height where the target calibration obstacle is located and the installation height is the height difference, the sample size feature data of the image area where the target calibration obstacle is located in the sample image of the target calibration obstacle acquired by the image acquisition device.

[0051] Optionally, in one specific implementation, the relationship-establishing submodule is specifically used for:

[0052] The initial model is trained based on the height differences and the sample size feature data under each height difference.

[0053] When the initial model meets the preset conditions, training stops, and an obstacle height detection model is obtained, which serves as the correspondence between the size feature data and the height difference of the suspended obstacle.

[0054] The target height difference determination submodule is specifically used for:

[0055] The target size feature data is input into the obstacle height detection model, and the height detection result output by the obstacle height detection model is obtained as the target height difference corresponding to the target size feature data.

[0056] Optionally, in one specific implementation, the device further includes an obstacle detection module; the obstacle detection module is specifically used for:

[0057] Feature extraction is performed on the target image to obtain the target image features;

[0058] If the target image features match the reference image features of the reference image, then it is determined that the suspended obstacle exists in the target image;

[0059] The reference image is an image of the suspended obstacle captured by the image acquisition device when the suspended obstacle exists within the acquisition area of ​​the image acquisition device.

[0060] Thirdly, embodiments of this application provide an electronic device, including:

[0061] Memory, used to store computer programs;

[0062] When a processor executes a program stored in memory, it implements the steps of any of the above method embodiments.

[0063] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above method embodiments.

[0064] Fifthly, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of any of the above method embodiments.

[0065] Beneficial effects of the embodiments in this application:

[0066] As can be seen from the above, by applying the method provided in this application embodiment, a target correspondence between the size feature data and height of suspended obstacles can be established in advance. Thus, during the movement of the target robot, the current position information of the target robot and the height of the transported object can be acquired, along with the target image captured by an image acquisition device installed at a designated location. Then, if a suspended obstacle exists in the target image, the distance between the target robot and the suspended obstacle is determined to be less than a preset distance based on the current position information, and the suspended obstacle is located above the pre-acquired movement path of the target robot, the target size feature data of the image area where the suspended obstacle is located in the target image can be determined. Then, based on the aforementioned target correspondence and the target size feature data, the current height of the suspended obstacle can be determined. Next, if the current height is not greater than the height of the transported object, the detection result regarding the potential collision risk between the target robot and the suspended obstacle can be determined based on the target robot's movement speed and the progress of removing the suspended obstacle.

[0067] Based on this, applying the solution provided in this application, when it is determined that there is a suspended obstacle above the target robot's movement path, and the distance between the suspended obstacle and the target robot is less than a preset distance, the current height of the suspended obstacle can be determined based on a pre-established target correspondence between the size feature data and height of the suspended obstacle, and the target size feature data of the suspended obstacle in the target image acquired by the image acquisition device. Thus, when the current height of the suspended obstacle is not greater than the height of the transport object, the risk of a collision between the target robot and the suspended obstacle can be determined based on the target robot's movement speed and the progress of removing the suspended obstacle. Furthermore, when a collision risk exists, the target robot's movement route can be changed in a timely manner to avoid the collision risk. In this way, by performing collision detection, the risk of the robot colliding with a suspended obstacle during movement can be reduced. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0069] Figure 1A schematic flowchart of a collision detection method provided for a real-time example of this application;

[0070] Figure 2(a)-Figure 2(b) These are schematic diagrams illustrating specific examples of suspended obstacles provided in the embodiments of this application;

[0071] Figure 3 A schematic diagram illustrating the current height of a suspended obstacle provided in an embodiment of this application;

[0072] Figure 4 Another schematic flowchart of the collision detection method provided in the embodiments of this application;

[0073] Figure 5 This is another schematic flowchart of the collision detection method provided in the embodiments of this application;

[0074] Figure 6(a) is a schematic diagram of the installation height of the smart camera provided in the embodiment of this application;

[0075] Figures 6(b)-6(d) These are schematic diagrams illustrating the height differences provided in the embodiments of this application;

[0076] Figures 6(e)-6(g) These are schematic diagrams of obstacle images captured by the smart camera provided in the embodiments of this application;

[0077] Figure 7 A schematic diagram of the installation height determination method provided in the embodiments of this application;

[0078] Figure 8 A schematic diagram illustrating a specific example of the collision detection method provided in the embodiments of this application;

[0079] Figure 9 This is a schematic diagram of the collision detection device provided in an embodiment of this application;

[0080] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0081] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0082] As a robot moves along a designated path, there may be suspended obstacles in its path. If the height of the suspended obstacle above the ground is less than the height of the robot, and the staff cannot remove the obstacle in time, the robot may collide with it, resulting in damage. Therefore, a collision detection method is urgently needed to reduce the risk of collisions during robot movement.

[0083] To address the aforementioned technical problems, this application provides a collision detection method.

[0084] This method can be applied to various application scenarios where collision detection is required, such as collision detection for inbound robots in freight centers and collision detection for cargo robots in factories.

[0085] Furthermore, this method can be applied to various image acquisition devices with data processing capabilities, such as smart cameras and smart cameras; it can also be applied to various electronic devices that communicate with image acquisition devices, such as mobile phones and computers; and when the subject of this method is an electronic device, the electronic device can be an independent electronic device or a cluster of devices composed of multiple electronic devices, hereinafter referred to as electronic devices.

[0086] Therefore, the embodiments of this application do not specifically limit the application scenarios and execution entities of the method.

[0087] This application provides a collision detection method that may include the following steps:

[0088] Obtain the current position information of the target robot and the height of the payload, and acquire the target image captured by the image acquisition device installed at the specified location;

[0089] When there is a suspended obstacle in the target image, and the distance between the target robot and the suspended obstacle is determined to be less than a preset distance based on the current position information, and the suspended obstacle is located above the pre-acquired movement path of the target robot, the target size feature data of the image area where the suspended obstacle is located in the target image is determined.

[0090] Determine a preset target correspondence between the size feature data and height of the suspended obstacle, and determine the current height of the suspended obstacle based on the target size feature data and the target correspondence;

[0091] If the current height is not greater than the height of the transport object, then based on the movement parameters of the target robot and the processing parameters for the suspended obstacle, a detection result is determined regarding whether there is a collision risk between the target robot and the suspended obstacle; wherein, the movement parameters include: the movement speed of the target robot, and the processing parameters include: the removal progress of the suspended obstacle.

[0092] As can be seen from the above, by applying the method provided in the embodiments of this application, a target correspondence between the size feature data and height of suspended obstacles can be established in advance. Thus, during the movement of the target robot, the current position information of the target robot and the height of the transported object can be acquired, and a target image captured by an image acquisition device installed at a designated location can be obtained. Then, if a suspended obstacle exists in the target image, the distance between the target robot and the suspended obstacle is determined to be less than a preset distance based on the current position information, and the suspended obstacle is located above the pre-acquired movement path of the target robot, the target size feature data of the image area where the suspended obstacle is located in the target image can be determined. Then, the preset target correspondence between the size feature data and height of the suspended obstacle can be determined, and the current height of the suspended obstacle can be determined based on the target size feature data and the target correspondence. Next, if the current height is not greater than the height of the transported object, a detection result regarding whether there is a collision risk between the target robot and the suspended obstacle is determined based on the target robot's movement speed and the progress of removing the suspended obstacle.

[0093] Based on this, applying the solution provided in this application, when it is determined that there is a suspended obstacle above the target robot's movement path, and the distance between the suspended obstacle and the target robot is less than a preset distance, the current height of the suspended obstacle can be determined based on a pre-established target correspondence between the size feature data and height of the suspended obstacle, and the target size feature data of the suspended obstacle in the target image acquired by the image acquisition device. Thus, when the current height of the suspended obstacle is not greater than the height of the transport object, the risk of a collision between the target robot and the suspended obstacle can be determined based on the target robot's movement speed and the progress of removing the suspended obstacle. Furthermore, when a collision risk exists, the target robot's movement route can be changed in a timely manner to avoid the collision risk. In this way, by performing collision detection, the risk of the robot colliding with a suspended obstacle during movement can be reduced.

[0094] The collision detection method provided in this application will now be described in detail with reference to the accompanying drawings.

[0095] Figure 1 This is a flowchart illustrating a collision detection method provided in an embodiment of this application, as shown below. Figure 1As shown, the method may include the following steps S101-S104.

[0096] S101: Obtain the current position information of the target robot and the height of the transported object, and obtain the target image captured by the image acquisition device installed at the specified location;

[0097] Taking a target robot used to perform cargo transportation tasks as an example, when using the target robot to perform cargo transportation tasks, a movement path can be planned in advance with the location of the cargo to be transported as the starting point and the destination of the task as the ending point. In this way, after loading the cargo, the target robot can perform the cargo transportation task along the planned movement path.

[0098] Typically, to avoid collisions with the target robot, there are no obstacles on the planned movement path; furthermore, since both the target robot itself and the cargo it carries can be at a certain height, there are also no obstacles above the planned movement path to prevent collisions between the target robot and the cargo it carries and any suspended obstacles.

[0099] However, there are some suspended objects in the environment where the target robot is located. For example, there may be hooks of cranes used for loading and unloading goods in a factory, or suspended raw material barrels. These suspended objects may appear in the movement path of the target robot during the movement process.

[0100] When the aforementioned suspended object appears on the target robot's movement path, the suspended object can be regarded as a suspended obstacle, that is, a suspended obstacle appears on the target robot's movement path.

[0101] Specifically, the highest point of the entire assembly consisting of the target robot and its cargo can be taken as the cargo height of the target robot; and the current distance between the lowest point of the suspended obstacle and the ground can be taken as the current height of the suspended obstacle. Thus, if the current height of the suspended obstacle is not greater than the cargo height of the target robot, the entire assembly consisting of the target robot and its cargo may collide with the suspended obstacle.

[0102] For example, as shown in Figure 2(a), if there is a suspended object above the target robot's movement path, the distance between the lowest point of the suspended object and the ground is the current height of the suspended object, and the height of the target robot's payload is the same as the payload height. Since the current height of the suspended object is less than the payload height, the payload carried by the target robot will collide with the suspended object as the target robot moves along the movement path.

[0103] Therefore, to avoid collisions between the target robot and suspended obstacles appearing above its movement path, at least one image acquisition device can be installed at a designated location. The acquisition range of this device should cover the area where the robot and obstacles might appear. Thus, for the target robot, the image acquisition device can acquire images of its acquisition range at preset intervals. Using the acquired images, the pre-acquired movement path of the target robot, and its current position, it can be determined whether any suspended obstacles exist above the target robot's movement path. Furthermore, if a suspended obstacle is detected above the target robot's movement path, the current height of the obstacle can be used to determine whether there is a risk of collision between the obstacle and the target robot.

[0104] The specified location can be set according to actual needs, such as the center point of the factory roof, which is reasonable and not specifically limited in this embodiment.

[0105] The above-mentioned collection interval can be set according to actual needs, such as 5 seconds, 20 seconds, etc., which are all reasonable and are not specifically limited in this application embodiment.

[0106] Optionally, considering that the acquisition range of each image acquisition device is limited, the number and location of image acquisition devices can be determined based on the size and shape of the scene area, the device parameters of the image acquisition devices, and the tasks performed by the robots in the scene area. This ensures that the acquisition range of the installed image acquisition devices can at least cover the area in the scene area where robots and obstacles may appear, or it can cover the entire scene area.

[0107] In this way, the electronic device can obtain the target robot's movement path in advance before controlling the target robot to perform the cargo loading task. For example, the electronic device can plan the movement path for the target robot based on the location of the cargo and its destination; or, for example, the electronic device can receive the target robot's movement path sent by the worker through a client communicating with the electronic device.

[0108] Of course, the specific implementation methods of various electronic devices for obtaining the movement path of the target robot are merely illustrative examples of movement path acquisition methods, and not limitations. Any method capable of acquiring the movement path falls within the protection scope of this application.

[0109] Furthermore, during the process of the target robot performing cargo transportation tasks, the electronic equipment can obtain the target robot's current location information and the height of the cargo, as well as the target image captured by the image acquisition device installed at the designated location.

[0110] For example, an electronic device can receive the carrying height of a target robot sent by a worker through a client communicating with the electronic device; or, for example, an electronic device can determine the carrying height of a target robot using a specified image of the target robot and the cargo it carries, captured by a specified image acquisition device.

[0111] Of course, the specific implementation methods of obtaining the height of the target robot's cargo using the various electronic devices mentioned above are merely illustrative examples of methods for obtaining the height of the cargo, and not limitations. Any method capable of obtaining the height of the cargo falls within the protection scope of this application.

[0112] Furthermore, after acquiring the target image, the target image can be detected to determine whether there are any suspended obstacles in the target image.

[0113] Optionally, in one specific implementation, after acquiring the target image from the image acquisition device installed at the designated location in step S101, the target image can be used to detect any suspended obstacles. Furthermore, the method for detecting whether there are suspended obstacles in the target image can include the following steps 11-12:

[0114] Step 11: Extract features from the target image to obtain the target image features;

[0115] Step 12: If the features of the target image match the features of the reference image, then it is determined that there is a suspended obstacle in the target image;

[0116] The reference image is the image of the suspended obstacle captured by the image acquisition device when there is a suspended obstacle in the acquisition area of ​​the image acquisition device.

[0117] In this specific implementation, in order to detect suspended obstacles in the images acquired by the image acquisition device, suspended obstacles can be set in advance in the acquisition area of ​​the image acquisition device. Then, when there are suspended obstacles in the acquisition area, a reference image of the suspended obstacles acquired by the image acquisition device can be obtained, and the reference image features of the reference image can be extracted.

[0118] Optionally, for each type of suspended obstacle that may appear, a reference image of the suspended obstacle appearing within the acquisition area of ​​the image acquisition device can be acquired to obtain a reference image for each type of suspended obstacle. Furthermore, feature extraction can be performed on each reference image to obtain the reference image features for each type of suspended obstacle.

[0119] In this way, after acquiring the target image through the target image acquisition device, feature extraction can be performed on the target image to obtain the target image features. Then, the target feature image is matched with the reference image features of a reference image. If the target feature image matches the reference image features, it can be determined that a suspended obstacle exists in the target image.

[0120] Optionally, the target image features of the target image can be matched with the reference image features of the multiple reference images. Then, when the target image features match any of the reference image features, it can be determined that there is a suspended obstacle in the target image.

[0121] Optionally, a standard image of the acquisition area, where no suspended obstacles exist, can be pre-acquired by the image acquisition device, and the standard image features of this standard image can be extracted. Then, after extracting the target image features of the target image, these target image features are matched with the standard image features. Furthermore, if the target image features do not match the standard image features, it can be determined that a suspended obstacle exists in the target image.

[0122] S102: When there is a suspended obstacle in the target image, the distance between the target robot and the suspended obstacle is determined to be less than a preset distance based on the current position information, and the suspended obstacle is located above the pre-acquired movement path of the target robot, determine the target size feature data of the image area where the suspended obstacle is located in the target image.

[0123] Since the image acquisition device is installed in a designated location, its acquisition range can be fixed, and therefore, the location information of the acquisition range is definite. Thus, when it is determined that a suspended obstacle exists in a target image, the target image can be analyzed. Based on the location of the area containing the suspended obstacle in the target image and the location information of the acquisition range of the image acquisition device, the location information of the suspended obstacle can be determined.

[0124] In this case, since the distance between the target robot and the suspended obstacle is relatively far, it can be assumed that there is no risk of collision between the target robot and the suspended obstacle in a short period of time. However, when the distance between the target robot and the suspended obstacle is relatively close, it can be assumed that there is a risk of collision between the target robot and the suspended obstacle in a short period of time. Therefore, it is necessary to further determine whether to adjust the movement path of the target robot to avoid the above-mentioned collision risks.

[0125] Based on this, it is possible to determine whether the distance between the target robot and the suspended obstacle is less than a preset distance, based on the location information of the suspended obstacle and the current location information of the target robot.

[0126] The aforementioned preset distance can be set according to actual needs, such as 5 meters, 20 meters, etc., which are all reasonable and are not specifically limited in this embodiment.

[0127] Furthermore, since the target robot moves along its path, it will not collide with the suspended obstacle if the obstacle is not located above its path. Therefore, the electronic device can determine whether the suspended obstacle exists above the target robot's path based on the determined location information of the obstacle and the pre-acquired path of the target robot.

[0128] Furthermore, if it is determined that there is a suspended obstacle in the target image, the distance between the target robot and the suspended obstacle is less than a preset distance, and the suspended obstacle is located above the target robot's movement path, it can be assumed that the target robot may collide with the suspended obstacle in a short period of time. Thus, the existence of the collision risk between the target robot and the suspended obstacle can be further determined based on the height of the target robot's payload and the current height of the suspended obstacle.

[0129] Furthermore, since the size of the image area where the object is located in the image acquired by the image acquisition device can represent information such as the height of the object and the distance from the image acquisition device under the fixed shooting angle of the image acquisition device, the target size feature data of the image area where the suspended obstacle is located in the above target image can be determined first, and the current height of the suspended obstacle can be determined by using the target size feature data.

[0130] The target size feature data of the image region where the suspended obstacle is located can be the area of ​​the image region where the suspended obstacle is located; it can be the area ratio of the image region where the suspended obstacle is located in the target image; or it can be the number of pixels in the image region where the suspended obstacle is located. All of these are reasonable and are not specifically limited in this embodiment.

[0131] S103: Determine the preset target correspondence between the size feature data and height of the suspended obstacle, and determine the current height of the suspended obstacle based on the target size feature data and the target correspondence;

[0132] To determine the current height of a detected suspended obstacle, various types of suspended obstacles that may appear on the target robot's movement path can be pre-defined. Furthermore, for each type of suspended obstacle, a calibration obstacle can be established, and for each type of calibration obstacle, a correspondence between its dimensional characteristics and height can be created.

[0133] For each type of calibration obstacle, the calibration obstacle can be placed at different heights. Then, when the calibration obstacle is at each height, an image acquisition device can be used to acquire images of the calibration obstacle. The size feature data of the image area where the calibration obstacle is located can be determined using each image, thereby establishing the correspondence between the size feature data of the calibration obstacle and its height.

[0134] The above-mentioned placement of the calibration obstacle at different heights refers to placing the calibration obstacle in different positions so that the distance between the lowest point of the calibration obstacle and the ground is different. In other words, the calibration obstacle being at any height means that the height of the lowest point of the calibration obstacle is taken as the height at which the calibration obstacle is located.

[0135] Furthermore, suspended obstacles belonging to the same category but of different sizes can be classified as different types of suspended obstacles.

[0136] Based on this, when a suspended obstacle is detected in the target image, a preset target correspondence between the size feature data and height of the suspended obstacle can be determined. Then, using the target size feature data of the image area where the suspended obstacle is located and the above target correspondence, the current height of the suspended obstacle can be determined.

[0137] It should be emphasized that, for the target correspondence determined in step S103 above, when constructing the target correspondence, the image acquisition device used to acquire images of the calibration obstacles located at different heights and the image acquisition device used for the target image in step S101 above are the same image acquisition device installed at the same specified position and with the same acquisition angle, or the same model of image acquisition device installed at the same specified position and with the same acquisition angle.

[0138] In other words, in this embodiment of the application, when determining whether the target robot has a collision risk with a suspended obstacle using a target image acquired by an image acquisition device, the image acquisition device used to construct the target correspondence in the above determination process and the image acquisition device that acquired the target image have the same installation position, the same shooting angle, the same shooting range, and the same shooting parameters. Therefore, the two image acquisition devices are the same image acquisition device installed in the same specified position and with the same shooting angle, or the same model of image acquisition device installed in the same specified position and with the same shooting angle.

[0139] Optionally, in one specific implementation, step S103 above may include the following steps 21-22:

[0140] Step 21: Detect the type of suspended obstacles in the target image to determine their type;

[0141] Step 22: Among the preset correspondences between the size feature data and height of multiple types of calibration obstacles, determine the correspondence between the size feature data and height of the target calibration obstacle of the same type as the obstacle, and use it as the target correspondence between the size feature data and height of the suspended obstacle;

[0142] The correspondence between the size feature data and height of each calibration obstacle is determined based on the size feature data of the image region where the calibration obstacle is located in each image acquired by the image acquisition device when the calibration obstacle is at different heights.

[0143] In this specific implementation, obstacle type detection can be performed on suspended obstacles in the target image to determine the obstacle type of the suspended obstacle.

[0144] Optionally, when it is determined that there is a suspended obstacle in the target image, the feature information of the suspended obstacle can be identified, such as the symbol, size, shape, color, etc. of the suspended obstacle. Thus, the obstacle type of the suspended obstacle can be determined based on the feature information of the suspended obstacle.

[0145] Then, from the preset correspondence between the size feature data and height of multiple types of calibration obstacles, the correspondence between the size feature data and height of the target calibration obstacle of the same type as the above obstacle can be determined, and the determined correspondence can be used as the target correspondence between the size feature data and height of the suspended obstacle.

[0146] As mentioned above, the correspondence between the size feature data and height of the calibration obstacle is determined based on the size feature data of the image region where the calibration obstacle is located in each image acquired by the image acquisition device when the calibration obstacle is at different heights.

[0147] After determining the target correspondence between the size feature data and the height of the suspended obstacle, the height corresponding to the target size feature data can be determined from the above target correspondence. Then, the current height of the suspended obstacle can be determined using the above height.

[0148] Optionally, the target correspondence between the size feature data and height of the suspended obstacle is as follows: the correspondence between the size feature data and current height of the suspended obstacle, where the current height corresponding to each feature data is: the current height of the suspended obstacle when the size feature data of the image area where the suspended obstacle is located is that size feature data.

[0149] Optionally, in one specific implementation, the installation height of the image acquisition device is higher than the maximum height of the suspended obstacle. The target correspondence between the size feature data of the suspended obstacle and the height is as follows: the correspondence between the size feature data of the suspended obstacle and the height difference, where the height difference corresponding to each feature data is: the height difference between the installation height and the current height of the suspended obstacle when the size feature data of the image area where the suspended obstacle is located is that size feature data.

[0150] Accordingly, step S103 above may include the following step 31:

[0151] Step 31: In the target correspondence relationship, determine the target height difference corresponding to the target size feature data, and calculate the difference between the installation height and the target height difference as the current height of the suspended obstacle.

[0152] In this specific implementation, the installation height of the image acquisition device is higher than the maximum height of the aforementioned suspended obstacle. Furthermore, the preset target correspondence between the size feature data of the suspended obstacle and its height can be a correspondence between the size feature data of the suspended obstacle and its height difference. The height difference corresponding to each feature data refers to the height difference between the installation height of the image acquisition device and the current height of the suspended obstacle when the size feature data of the image area where the suspended obstacle is located is that size feature data. In other words, the height difference is the distance between the image acquisition device and the suspended obstacle in the vertical direction.

[0153] After determining the target size feature data of the image area where the suspended obstacle is located and the target correspondence, the target height difference corresponding to the target size feature data can be determined according to the target correspondence. Then, based on the determined target height difference, the difference between the installation height of the image acquisition device and the target height difference is calculated, and the difference is taken as the current height of the suspended obstacle.

[0154] Optionally, in the target correspondence relationship, the target height difference corresponding to the target size feature data is determined, and a first difference between the installation height and the target height difference is calculated. Then, a second difference between the first difference and the height of the suspended obstacle is calculated. After that, the second difference is used as the current height of the suspended obstacle.

[0155] For example, such as Figure 3As shown, the height difference corresponding to the size feature data of the suspended object can be determined by using the correspondence between the size feature data of the suspended object and the height difference, that is, the target height difference between the smart camera and the suspended object can be determined. Then, the difference between the installation height H of the smart camera and the target height difference of the suspended object can be calculated, and the current height X of the suspended object can be determined by using the target height difference and the height of the suspended object itself.

[0156] To ensure a smooth writing style, the method for establishing the correspondence between the dimensional characteristics of suspended obstacles and their height differences will be described in detail below.

[0157] S104: If the current height is not greater than the height of the transported object, then based on the target robot's movement parameters and the processing parameters for the suspended obstacle, determine the detection result regarding whether there is a collision risk between the target robot and the suspended obstacle;

[0158] The movement parameters include the target robot's movement speed, and the processing parameters include the progress of removing suspended obstacles.

[0159] Typically, as shown in Figure 2(a), if the current height of the suspended object is not greater than the height of the carrier, the carrier will collide with the suspended object during the robot's movement; while as shown in Figure 2(b), if the current height of the suspended object is greater than the height of the carrier, the carrier will not collide with the suspended object during the robot's movement.

[0160] Based on this, by determining the numerical relationship between the current height of the suspended obstacle and the height of the target robot's payload, it can be determined whether the target robot will collide with the suspended obstacle.

[0161] Furthermore, since the presence of a suspended obstacle in the target image can be determined by the operator, the suspended obstacle can be removed. If the suspended obstacle has been removed when the target robot moves to the location of the suspended obstacle, the target robot will not collide with the suspended obstacle.

[0162] Based on this, it is possible to determine whether the target robot will collide with the suspended obstacle by considering the numerical relationship between the current height of the suspended obstacle and the height of the target robot's payload, the target robot's movement parameters, and the processing parameters for the suspended obstacle.

[0163] If the current height of the suspended obstacle is not greater than the height of the target robot's payload, the target robot's movement parameters and processing parameters for the suspended obstacle can be determined. Then, based on these parameters, a detection result regarding the potential collision risk between the target robot and the suspended obstacle is determined.

[0164] The aforementioned movement parameters may include the target robot's movement speed; while the aforementioned processing parameters may include the progress of removing suspended obstacles.

[0165] Optionally, when it is determined that there is a suspended obstacle in the target image, the distance between the target robot and the suspended obstacle is less than a preset distance based on the current position information, and the suspended obstacle is located above the pre-acquired movement path of the target robot, the acquisition area of ​​the image acquisition device is determined as a collision risk area. Then, a first notification message can be output to indicate the existence of a collision risk area, so that relevant personnel can receive the first notification message and take corresponding measures to remove the suspended obstacle.

[0166] The output of the first notification message can be: sending an email to a preset email address to indicate the presence of a collision risk area; or displaying an alarm icon on the display screen of the device on which the electronic device is located; or emitting an alarm sound. All of these are reasonable and are not specifically limited in this application embodiment.

[0167] Optionally, when the target robot can move along the aforementioned movement path at a preset speed, the movement time for the target robot to reach the location of the suspended obstacle can be determined based on the target robot's movement speed and the distance between the target robot and the suspended obstacle. Furthermore, the removal time for removing the suspended obstacle is calculated based on the removal speed of the suspended obstacle. If the target robot's movement time is greater than the removal time of the suspended obstacle, the suspended obstacle will have been removed by the time the target robot reaches its location, and the target robot will not collide with the suspended obstacle. Conversely, if the target robot's movement time is not greater than the removal time of the suspended obstacle, the suspended obstacle will not have been removed by the time the target robot reaches its location, and the target robot may collide with the suspended obstacle.

[0168] Optionally, after determining the above test results, the above test results can be output.

[0169] The output of the above detection results can be: sending an email containing the above detection results to a preset email address; or displaying the above detection results on the display screen of the device where the electronic device is located; both are reasonable and this application embodiment does not impose specific limitations.

[0170] Alternatively, in one specific implementation, such as Figure 4 As shown in the embodiment of this application, a collision detection method may further include the following step S105:

[0171] S105: When the detection results indicate a collision risk, change the target robot's movement path and control the target robot to move along the changed movement path.

[0172] In this specific implementation, when the detection results indicate that there is a risk of collision between the target robot and the suspended obstacle, the movement path of the target robot can be changed, and then the target robot can be controlled to move along the changed movement path.

[0173] Optionally, when the above detection results indicate a collision risk, the target robot can be controlled to stop moving, and a new movement path can be planned for the target robot, starting from the stopped position and ending at the task target location, and the target robot can be controlled to move along the new movement path.

[0174] Optionally, if the above detection results indicate a collision risk, a local movement path can be planned and constructed for the robot, starting from its current position and ending at a designated point on its movement path that bypasses the suspended obstacle. The robot can then be controlled to move along this local movement path around the suspended obstacle. Subsequently, the robot can be controlled to continue moving along the previous path.

[0175] As can be seen from the above, by applying the solution provided in this application, when it is determined that there is a suspended obstacle above the target robot's movement path, and the distance between the suspended obstacle and the target robot is less than a preset distance, the current height of the suspended obstacle can be determined based on the pre-established target correspondence between the size feature data and height of the suspended obstacle, and the target size feature data of the suspended obstacle in the target image acquired by the image acquisition device. Thus, when the current height of the suspended obstacle is not greater than the height of the transport object, the risk of a collision between the target robot and the suspended obstacle can be determined based on the target robot's movement speed and the progress of removing the suspended obstacle. Furthermore, when a collision risk exists, the target robot's movement route can be changed in a timely manner to avoid the collision risk. In this way, by performing collision detection, the risk of the robot colliding with a suspended obstacle during movement can be reduced.

[0176] The following section provides a detailed explanation of how the correspondence between the dimensional characteristics of suspended obstacles and their height differences is established.

[0177] Alternatively, in one specific implementation, such as Figure 5 As shown, the method for establishing the correspondence between the dimensional feature data of suspended obstacles and the height difference may include the following steps S501-S503:

[0178] S501: Determine the installation height of the image acquisition device;

[0179] S502: Acquire sample images of the target calibration obstacle collected by the image acquisition device when the target calibration obstacle is located at each sample height, and determine the sample size feature data of the image area where the target calibration obstacle is located in each sample image;

[0180] The sample size feature data includes: the number of pixels;

[0181] S503: For each sample height, calculate the height difference between the installation height and the sample height, and based on each height difference and the sample size feature data under each height difference, establish the correspondence between the size feature data of the suspended obstacle and the height difference;

[0182] Among them, the sample size feature data under each height difference is: when the height difference between the sample height where the target calibration obstacle is located and the installation height is the same, the sample size feature data of the image area where the target calibration obstacle is located in the sample image of the target calibration obstacle acquired by the image acquisition device.

[0183] In this specific implementation, for the same image acquisition device, the installation height and acquisition range of the device are fixed. However, for the same calibration obstacle, when the height difference between the calibration obstacle and the image acquisition device is different, the size characteristic data of the image area where the calibration obstacle is located will be different in the image acquired by the image acquisition device.

[0184] For example, the mounting height H of the smart camera is shown in Figure 6(a); Figures 6(b)-6(d) These represent the height difference between obstacle A and the smart camera when obstacle A is in different positions; Figures 6(e)-6(g) Obstacle A is in the following position: Figures 6(b)-6(d) Images of obstacle A captured by the smart camera at different locations are shown.

[0185] As shown in Figure 6(b), when the height difference between obstacle A and the smart camera is X1, the image of obstacle A captured by the smart camera is shown in Figure 6(e); as shown in Figure 6(c), when the height difference between obstacle A and the smart camera is X2, the image of obstacle A captured by the smart camera is shown in Figure 6(f); and as shown in Figure 6(d), when the height difference between obstacle A and the smart camera is X3, the image of obstacle A captured by the smart camera is shown in Figure 6(g), where X1 is greater than X2, and X2 is greater than X3.

[0186] It is possible Figures 6(e)-6(g) In the process, the number of pixels in the image region where obstacle A is located is determined.

[0187] As shown in Figure 6(e), when the height difference between obstacle A and the smart camera is X1, the number of pixels in the image area where obstacle A is located is determined to be the first number.

[0188] As shown in Figure 6(f), when the height difference between obstacle A and the smart camera is X2, the number of pixels in the image area where obstacle A is located is determined as the second number; wherein, the second number is less than the first number mentioned above;

[0189] As shown in Figure 6(g), when the height difference between obstacle A and the smart camera is X3, the number of pixels in the image area where obstacle A is located is determined to be the third number; and the third number is less than the second number and less than the first number.

[0190] As can be seen above, as the height difference between obstacle A and the smart camera gradually increases, the number of pixels in the image area where obstacle A is located gradually decreases.

[0191] Based on this, the installation height of the image acquisition device can be determined first. Then, target calibration obstacles of the same type as the suspended obstacle are placed at various sample heights, and images of these target calibration obstacles are acquired using the image acquisition device. Next, sample images of the target calibration obstacle are obtained when it is located at each sample height, thus yielding multiple sample images of the target calibration obstacle.

[0192] For each sample image, the sample size feature data of the image region where the target calibration obstacle is located can be determined.

[0193] The sample size feature data mentioned above may include the number of pixels, the area of ​​the region, or the area ratio of the region in the sample image. These are all reasonable and are not specifically limited in this embodiment.

[0194] Next, for each sample height, the height difference between the installation height and the sample height can be calculated. Then, when the height difference between the sample height where the target calibration obstacle is located and the installation height is determined to be the height difference, the sample size feature data of the image region where the target calibration obstacle is located in the sample image of the target calibration obstacle acquired by the image acquisition device is obtained, thereby obtaining the sample size feature data under the height difference.

[0195] In this way, after determining the height difference between the installation height and the sample height at each sample height, as well as the sample size characteristic data at that height difference, a correspondence between the size characteristic data of the suspended obstacle and the height difference is established.

[0196] Optionally, the initial model can be trained based on each height difference and the sample size feature data under each height difference to obtain an obstacle height detection model, and the obstacle height detection model can be used as the correspondence between the size feature data of the suspended obstacle and the height difference.

[0197] Optionally, based on the above height differences and the sample size feature data under each height difference, a lookup table of the relationship between the size feature data of the target calibration obstacle and the height difference can be established, and the above lookup table can be used as the correspondence between the size feature data of the suspended obstacle and the height difference.

[0198] Optionally, in one specific implementation, step S603 above, establishing the correspondence between the size feature data of the suspended obstacle and the height difference based on each height difference and the sample size feature data under each height difference, may include the following steps 41-42:

[0199] 41: The initial model is trained based on the height differences and the sample size feature data under each height difference;

[0200] 42: When the initial model meets the preset conditions, stop training and obtain the obstacle height detection model, which serves as the correspondence between the size feature data and the height difference of the suspended obstacle;

[0201] Accordingly, step S1031 above, which determines the target height difference corresponding to the target size feature data in the target correspondence relationship, may include the following step 43:

[0202] Step 43: Input the target size feature data into the obstacle height detection model and obtain the height detection result output by the obstacle height detection model as the target height difference corresponding to the target size feature data.

[0203] In this specific implementation, after obtaining the sample size feature data of each height difference and each height difference, the initial model can be trained using the aforementioned sample size feature data of each height difference and each height difference.

[0204] The sample size feature data under each height difference can be used as training samples, and each height difference can be used as the sample label for the sample size feature data under that height difference. Then, the preset initial model is trained using the training samples, and training stops when the initial model meets preset conditions, thereby obtaining the obstacle height detection model. Furthermore, the obstacle height detection model can be used as a correspondence between the size feature data of the calibrated obstacle and the specified height.

[0205] Training can be stopped once the initial model meets the preset conditions, resulting in a state detection model.

[0206] Optionally, the above preset condition can be that the number of iterations for the training samples reaches a preset number;

[0207] Optionally, the training samples can be divided into training set samples and test set samples. Furthermore, the preset condition can be that the error between the true value and the predicted value of the obstacle height detection result of each test set sample is less than a preset error. The true value of the obstacle height detection result of the test set sample can be the height difference between the sample labels of the test set sample, and the predicted value of the sample label of the test set sample can be the obstacle height detection result obtained by inputting the test set sample into the obstacle height detection model.

[0208] Furthermore, the electronic device used for model training and the electronic device that executes the collision detection method provided in the embodiments of this application may be the same electronic device or different electronic devices.

[0209] In this way, after training the obstacle height detection model, the target size feature data of the suspended obstacle can be input into the obstacle height detection model, and the detection results output by the obstacle height detection model can be obtained, thereby obtaining the target height difference corresponding to the target size feature data.

[0210] Then, the difference between the target height difference and the installation height can be calculated as the target obstacle height corresponding to the target size feature data.

[0211] Typically, errors may occur during the engineering installation process, causing the ideal installation height of the image acquisition equipment to differ from the actual installation height. For example, the ideal installation height is 8 meters, while the actual installation height is 8.2 meters.

[0212] Furthermore, after the image acquisition device is installed, its installation height can be detected to determine its actual installation height.

[0213] To detect the installation height of the aforementioned image acquisition equipment, a correspondence between the installation height and the dimensional feature data of the calibrated obstacle can be established in advance.

[0214] In establishing the correspondence between the installation height and the dimensional characteristic data of the calibration obstacle, the calibration obstacle can be pre-selected and placed in a designated location. Then, the image acquisition device can be installed at different reference heights. Next, for each reference height, a reference image including the calibration obstacle can be acquired using the image acquisition device, and the dimensional characteristic data of the calibration obstacle can be determined from the acquired reference images. In this way, a correspondence between the installation height and the dimensional characteristic data of the calibration obstacle can be established based on each reference height and the dimensional characteristic data of the calibration obstacle at each reference height.

[0215] Generally, for the same calibration obstacle, the greater the height difference between the calibration obstacle and the image acquisition device, the smaller the size feature data of the calibration obstacle in the image acquired by the image acquisition device. In other words, when the position of the calibration obstacle is determined, the higher the installation height of the acquisition device, the smaller the size feature data of the calibration obstacle in the image it acquires.

[0216] Furthermore, the aforementioned size feature data can be the area of ​​the image region where the obstacle is located, the area ratio of the image region where the obstacle is located, or the number of pixels in the image region where the obstacle is located.

[0217] For example, such as Figure 7 As shown, a calibration object is placed within the image acquisition area of ​​the installed smart camera, and an image including the calibration object is acquired by the smart camera. Based on this image, the dimensional characteristics of the calibration object can be determined. Then, based on the correspondence between the installation height and the dimensional characteristics of the calibration object, the installation height of the smart camera can be determined, thus obtaining the actual installation height of the smart camera.

[0218] To facilitate understanding of the collision detection method provided in this application, the following is combined with... Figure 8 Please provide an explanation. Figure 8 The diagram shows a specific example of applying the collision detection method described above.

[0219] To perform collision detection on the robot, multiple detection zones can be set along the robot's movement path, and a smart camera can be installed at a designated location within each detection zone to capture images of that zone. These smart cameras can then capture images of the detection zone at preset intervals and send the captured images to a management platform in the server room.

[0220] The management platform can acquire the robot's current position information and the height of the transported object, and can receive images captured by the smart camera. Then, it can perform suspended object detection on the received images. Furthermore, when a suspended object is detected in the image, the distance between the robot and the suspended object is no greater than a preset distance, and the suspended object is located above the robot's movement path, the platform can determine the target size feature data of the image area containing the suspended object in the image captured by the smart camera.

[0221] Next, the correspondence between the size characteristic data and height of the suspended object can be determined. Using the above correspondence and the target size characteristic data, the current height of the suspended object corresponding to the target size characteristic data can be determined. If the current height of the suspended object is not greater than the height of the robot's carrying object, the area where the suspended object is located in the movement path can be identified as a collision risk area. A notification message about the existence of a collision risk area is sent to the management platform. In this way, the management platform can receive the above notification message and send the notification message to the PDA (Personal Digital Assistant, handheld terminal) so that relevant personnel can receive the above notification message, take abnormal measures in the above collision risk area, and upload the removal progress of the suspended object.

[0222] Upon receiving the progress report on the removal of the suspended object from the PDA, the management platform can determine the detection result regarding the potential collision risk between the robot and the suspended object based on the removal progress and the robot's moving speed. If the detection result indicates a collision risk between the robot and the suspended object, an alarm message can be output, and the robot's movement path can be changed. Subsequently, the robot can be controlled to move along the changed path.

[0223] Based on the same application concept, and corresponding to the embodiments provided in the above application, Figure 1 The present application provides a collision detection device in addition to the collision detection method shown in the embodiment.

[0224] Figure 9 This is a schematic diagram of the structure of a collision detection device provided in an embodiment of this application, as shown below. Figure 9 As shown, the device may include the following modules:

[0225] The location information acquisition module 910 is used to acquire the current location information of the target robot and the height of the transported object, and to acquire the target image acquired by the image acquisition device installed at the specified location;

[0226] The feature data determination module 920 is used to determine the target size feature data of the image area where the suspended obstacle is located in the target image when there is a suspended obstacle in the target image, the distance between the target robot and the suspended obstacle is less than a preset distance based on the current position information, and the suspended obstacle is located above the pre-acquired movement path of the target robot.

[0227] The height determination module 930 is used to determine a preset target correspondence between the size feature data of the suspended obstacle and its height, and to determine the current height of the suspended obstacle based on the target size feature data and the target correspondence.

[0228] The result determination module 940 is used to determine, based on the target robot's movement parameters and the processing parameters for the suspended obstacle, whether there is a collision risk between the target robot and the suspended obstacle if the current height is not greater than the height of the carrier; wherein the movement parameters include the target robot's movement speed, and the processing parameters include the removal progress of the suspended obstacle.

[0229] As can be seen from the above, by applying the solution provided in this application, when it is determined that there is a suspended obstacle above the target robot's movement path, and the distance between the suspended obstacle and the target robot is less than a preset distance, the current height of the suspended obstacle can be determined based on the pre-established target correspondence between the size feature data and height of the suspended obstacle, and the target size feature data of the suspended obstacle in the target image acquired by the image acquisition device. Thus, when the current height of the suspended obstacle is not greater than the height of the transport object, the risk of a collision between the target robot and the suspended obstacle can be determined based on the target robot's movement speed and the progress of removing the suspended obstacle. Furthermore, when a collision risk exists, the target robot's movement route can be changed in a timely manner to avoid the collision risk. In this way, by performing collision detection, the risk of the robot colliding with a suspended obstacle during movement can be reduced.

[0230] Optionally, in one specific implementation, the apparatus further includes:

[0231] The path changing module is used to change the movement path of the target robot when the detection result indicates that there is a collision risk, and to control the target robot to move along the changed movement path.

[0232] Optionally, in one specific implementation, the height determination module 930 is specifically used for:

[0233] The suspended obstacle in the target image is subjected to obstacle type detection to determine the obstacle type of the suspended obstacle;

[0234] Among a number of preset correspondences between the size feature data and height of different types of calibration obstacles, a correspondence between the size feature data and height of a target calibration obstacle of the same type as the obstacle is determined as the target correspondence between the size feature data and height of the suspended obstacle; wherein, the correspondence between the size feature data and height of each calibration obstacle is determined based on the size feature data of the image region where the calibration obstacle is located in each image of the calibration obstacle acquired by the image acquisition device when the calibration obstacle is at different heights.

[0235] Optionally, in one specific implementation, the installation height of the image acquisition device is higher than the maximum height of the suspended obstacle. The target correspondence between the size feature data of the suspended obstacle and the height is: the correspondence between the size feature data of the suspended obstacle and the height difference, where the height difference corresponding to each feature data is: the height difference between the installation height and the current height of the suspended obstacle when the size feature data of the image area where the suspended obstacle is located is that size feature data;

[0236] The height determination module 930 includes:

[0237] The target height difference determination submodule is used to determine the target height difference corresponding to the target size feature data in the target correspondence relationship, and calculate the difference between the installation height and the target height difference as the current height of the suspended obstacle.

[0238] Optionally, in one specific implementation, the device further includes a correspondence establishment module, which includes:

[0239] The installation height determination submodule is used to determine the installation height of the image acquisition device;

[0240] The sample image acquisition submodule is used to acquire various sample images of the target calibration obstacle collected by the image acquisition device when the target calibration obstacle is located at various sample heights, and to determine the sample size feature data of the image region where the target calibration obstacle is located in each sample image; wherein, the sample size feature data includes: the number of pixels;

[0241] The relationship establishment submodule is used to calculate the height difference between the installation height and the sample height for each sample height, and establish a correspondence between the size feature data of the suspended obstacle and the height difference based on each height difference and the sample size feature data under each height difference;

[0242] Among them, the sample size feature data under each height difference is: when the height difference between the sample height where the target calibration obstacle is located and the installation height is the height difference, the sample size feature data of the image area where the target calibration obstacle is located in the sample image of the target calibration obstacle acquired by the image acquisition device.

[0243] Optionally, in one specific implementation, the relationship-establishing submodule is specifically used for:

[0244] The initial model is trained based on the height differences and the sample size feature data under each height difference.

[0245] When the initial model meets the preset conditions, training stops, and an obstacle height detection model is obtained, which serves as the correspondence between the size feature data and the height difference of the suspended obstacle.

[0246] The target height difference determination submodule is specifically used for:

[0247] The target size feature data is input into the obstacle height detection model, and the height detection result output by the obstacle height detection model is obtained as the target height difference corresponding to the target size feature data.

[0248] Optionally, in one specific implementation, the device further includes an obstacle detection module; the obstacle detection module is specifically used for:

[0249] Feature extraction is performed on the target image to obtain the target image features;

[0250] If the target image features match the reference image features of the reference image, then it is determined that the suspended obstacle exists in the target image;

[0251] The reference image is an image of the suspended obstacle captured by the image acquisition device when the suspended obstacle exists within the acquisition area of ​​the image acquisition device.

[0252] This application also provides an electronic device, such as... Figure 10 As shown, it includes:

[0253] Memory 1001 is used to store computer programs;

[0254] The processor 1002, when executing the program stored in the memory 1001, implements any of the collision detection methods described above.

[0255] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 1002, the communication interface, and the memory 1001 communicating with each other via the communication bus.

[0256] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0257] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0258] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0259] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0260] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the collision detection methods described above.

[0261] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the collision detection methods described above.

[0262] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), etc.

[0263] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0264] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, and computer-readable storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0265] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A collision detection method characterized by, The method includes: The system acquires the current position information and carrying height of the target robot moving in the scene area, and acquires the target image captured by the image acquisition device installed at a specified position in the scene area. When there is a suspended obstacle in the target image, and the distance between the target robot and the suspended obstacle is determined to be less than a preset distance based on the current position information, and the suspended obstacle is located above the pre-acquired movement path of the target robot, the target size feature data of the image area where the suspended obstacle is located in the target image is determined. A preset target correspondence between the size feature data and height of the suspended obstacle is determined, and the current height of the suspended obstacle is determined based on the target size feature data and the target correspondence. The installation height of the image acquisition device is higher than the maximum height of the suspended obstacle. The target correspondence between the size feature data and height of the suspended obstacle is: a correspondence between the size feature data and height difference of the suspended obstacle. The height difference corresponding to each feature data is: the height difference between the installation height and the current height of the suspended obstacle when the size feature data of the image area where the suspended obstacle is located is that size feature data. If the current height is not greater than the height of the transport object, then based on the movement parameters of the target robot and the processing parameters for the suspended obstacle, a detection result is determined regarding whether there is a collision risk between the target robot and the suspended obstacle; wherein, the movement parameters include: the movement speed of the target robot, and the processing parameters include: the removal progress of the suspended obstacle.

2. The method according to claim 1, characterized in that, The method further includes: When the detection result indicates a collision risk, the movement path of the target robot is changed, and the target robot is controlled to move along the changed movement path.

3. The method according to claim 1, characterized in that, The determination of the preset target correspondence between the size feature data and height of the suspended obstacle includes: The suspended obstacle in the target image is subjected to obstacle type detection to determine the obstacle type of the suspended obstacle; Among a number of preset correspondences between the size feature data and height of different types of calibration obstacles, a correspondence between the size feature data and height of a target calibration obstacle of the same type as the obstacle is determined as the target correspondence between the size feature data and height of the suspended obstacle; wherein, the correspondence between the size feature data and height of each calibration obstacle is determined based on the size feature data of the image region where the calibration obstacle is located in each image of the calibration obstacle acquired by the image acquisition device when the calibration obstacle is at different heights.

4. The method according to any one of claims 1-3, characterized in that, Determining the current height of the suspended obstacle based on target size feature data and the target correspondence includes: In the target correspondence, the target height difference corresponding to the target size feature data is determined, and the difference between the installation height and the target height difference is calculated as the current height of the suspended obstacle.

5. The method according to claim 4, characterized in that, The method for establishing the correspondence between the dimensional feature data and the height difference of the suspended obstacle includes: Determine the installation height of the image acquisition device; The image acquisition device acquires sample images of the target calibration obstacle at various sample heights, and determines the sample size feature data of the image region where the target calibration obstacle is located in each sample image; wherein, the sample size feature data includes: the number of pixels; For each sample height, the height difference between the installation height and the sample height is calculated, and based on each height difference and the sample size feature data under each height difference, a correspondence between the size feature data of the suspended obstacle and the height difference is established; Among them, the sample size feature data under each height difference is: when the height difference between the sample height where the target calibration obstacle is located and the installation height is the height difference, the sample size feature data of the image area where the target calibration obstacle is located in the sample image of the target calibration obstacle acquired by the image acquisition device.

6. The method according to claim 5, characterized in that, The process of establishing a correspondence between the size feature data of the suspended obstacle and the height differences based on each height difference and the sample size feature data under each height difference includes: The initial model is trained based on the height differences and the sample size feature data under each height difference. When the initial model meets the preset conditions, training stops, and an obstacle height detection model is obtained, which serves as the correspondence between the size feature data and the height difference of the suspended obstacle. In the target correspondence, determining the target height difference corresponding to the target size feature data includes: The target size feature data is input into the obstacle height detection model, and the height detection result output by the obstacle height detection model is obtained as the target height difference corresponding to the target size feature data.

7. The method according to claim 1, characterized in that, Methods for detecting whether there are suspended obstacles in the target image include: Feature extraction is performed on the target image to obtain the target image features; If the target image features match the reference image features of the reference image, then it is determined that the suspended obstacle exists in the target image; The reference image is an image of the suspended obstacle captured by the image acquisition device when the suspended obstacle exists within the acquisition area of ​​the image acquisition device.

8. A collision detection device, characterized in that, The device includes: The location information acquisition module is used to acquire the current location information and the height of the carrying object of the target robot moving in the scene area, and to acquire the target image acquired by the image acquisition device installed at a specified position in the scene area; The feature data determination module is used to determine the target size feature data of the image area where the suspended obstacle is located in the target image when there is a suspended obstacle in the target image, the distance between the target robot and the suspended obstacle is less than a preset distance based on the current position information, and the suspended obstacle is located above the pre-acquired movement path of the target robot. A height determination module is used to determine a preset target correspondence between the size feature data of the suspended obstacle and its height, and to determine the current height of the suspended obstacle based on the target size feature data and the target correspondence. The installation height of the image acquisition device is higher than the maximum height of the suspended obstacle. The target correspondence between the size feature data of the suspended obstacle and its height is a correspondence between the size feature data of the suspended obstacle and its height difference. The height difference corresponding to each feature data is the height difference between the installation height and the current height of the suspended obstacle when the size feature data of the image area where the suspended obstacle is located is that size feature data. The result determination module is used to determine, based on the target robot's movement parameters and the processing parameters for the suspended obstacle, whether there is a collision risk between the target robot and the suspended obstacle if the current height is not greater than the height of the carrier; wherein the movement parameters include the target robot's movement speed, and the processing parameters include the removal progress of the suspended obstacle.

9. The apparatus according to claim 8, characterized in that, The device further includes: The path changing module is used to change the movement path of the target robot when the detection result indicates that there is a collision risk, and to control the target robot to move along the changed movement path.

10. The apparatus according to claim 8, characterized in that, The height determination module is specifically used for: The suspended obstacle in the target image is subjected to obstacle type detection to determine the obstacle type of the suspended obstacle; Among a number of preset correspondences between the size feature data and height of different types of calibration obstacles, a correspondence between the size feature data and height of a target calibration obstacle of the same type as the obstacle is determined as the target correspondence between the size feature data and height of the suspended obstacle; wherein, the correspondence between the size feature data and height of each calibration obstacle is determined based on the size feature data of the image region where the calibration obstacle is located in each image of the calibration obstacle acquired by the image acquisition device when the calibration obstacle is at different heights.

11. The apparatus according to any one of claims 8-10, characterized in that, The height determination module includes: The target height difference determination submodule is used to determine the target height difference corresponding to the target size feature data in the target correspondence relationship, and calculate the difference between the installation height and the target height difference as the current height of the suspended obstacle.

12. The apparatus according to claim 11, characterized in that, The device further includes a correspondence establishment module, which includes: The installation height determination submodule is used to determine the installation height of the image acquisition device; The sample image acquisition submodule is used to acquire various sample images of the target calibration obstacle collected by the image acquisition device when the target calibration obstacle is located at various sample heights, and to determine the sample size feature data of the image region where the target calibration obstacle is located in each sample image; wherein, the sample size feature data includes: the number of pixels; The relationship establishment submodule is used to calculate the height difference between the installation height and the sample height for each sample height, and establish a correspondence between the size feature data of the suspended obstacle and the height difference based on each height difference and the sample size feature data under each height difference; Among them, the sample size feature data under each height difference is: when the height difference between the sample height where the target calibration obstacle is located and the installation height is the height difference, the sample size feature data of the image area where the target calibration obstacle is located in the sample image of the target calibration obstacle acquired by the image acquisition device.

13. The apparatus according to claim 12, characterized in that, The relationship establishment submodule is specifically used for: The initial model is trained based on the height differences and the sample size feature data under each height difference. When the initial model meets the preset conditions, training stops, and an obstacle height detection model is obtained, which serves as the correspondence between the size feature data and the height difference of the suspended obstacle. The target height difference determination submodule is specifically used for: The target size feature data is input into the obstacle height detection model, and the height detection result output by the obstacle height detection model is obtained as the target height difference corresponding to the target size feature data.

14. The apparatus according to claim 8, characterized in that, The device further includes an obstacle detection module; the obstacle detection module is specifically used for: Feature extraction is performed on the target image to obtain the target image features; If the target image features match the reference image features of the reference image, then it is determined that the suspended obstacle exists in the target image; The reference image is an image of the suspended obstacle captured by the image acquisition device when the suspended obstacle exists within the acquisition area of ​​the image acquisition device.

15. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images