Obstacle avoidance method and device for self-propelled equipment based on line laser, equipment and medium

Abstract

The present disclosure relates to a method and apparatus, equipment and medium for obstacle avoidance of a self-propelled equipment based on a line laser, the method including: obtaining a line laser recognition image of the self-propelled equipment; determining whether a wire-like obstacle exists based on the line laser recognition image to obtain a line laser recognition result; and performing a corresponding obstacle avoidance operation based on the line laser recognition result.

Description

[Technical Field]

[0001] This disclosure claims priority to a patent application for an invention filed on December 30, 2022, application number CN202211732848.4, and titled "Method and apparatus for obstacle avoidance for self-propelled equipment based on line laser, equipment and medium."

[0002] The present disclosure relates to the field of cleaning robot control technology, and in particular to a method and apparatus for obstacle avoidance of a self-propelled device based on a line laser, a device, and a medium. [Background technology]

[0003] Common obstacles in a home environment, such as shoes, socks, scales, fans, etc., are all obstacles with fixed shapes and fixed sizes. For these obstacles, a method that combines general-purpose recognition with a line laser method can meet most obstacle avoidance needs. That is, image recognition is used to obtain the type and location information of the obstacle, and this is combined with line laser ranging to obtain the location information of the obstacle. By combining the two, the type and location information of the obstacle can be obtained more accurately.

[0004] On the other hand, the deployed scattered rays cannot be completely observed from the robot's viewpoint because, first, the posture of the wire is not fixed and it occupies a relatively large space even after semi-deployment / deployment. Wires that cannot be completely observed, especially those with an unfixed posture, are more likely to have blind spots, and if the machine runs near a blind spot, there is a higher risk of the machine's work being obstructed. Secondly, since the height of relatively thin wires is relatively low, the accuracy of conventional line laser ranging solutions cannot meet the needs of realizing ranging based on changes in the height of obstacles.

[0005] It should be noted that the information disclosed in the above Background section is intended solely to enhance understanding of the background of the present disclosure and may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The present disclosure aims to provide an obstacle avoidance method and device for a self-propelled device, a self-propelled device, and a computer-readable storage medium that are arranged to achieve accurate recognition of wire-type obstacles.

[0007] According to a first aspect of the present disclosure, Obtaining a line laser recognition image of the self-propelled device; Determine whether there is a wire-like obstacle based on the line laser recognition image to obtain a line laser recognition result; Based on the line laser recognition result, performing a corresponding obstacle avoidance operation; The present invention provides a line laser-based obstacle avoidance method for a self-propelled device, including:

[0008] In an exemplary embodiment of the present disclosure, determining whether a wire obstacle exists based on the line laser recognition image to obtain a line laser recognition result includes: analyzing the line laser recognition image to determine whether a laser breakpoint exists for each line laser in the line laser recognition image; If a laser breakpoint exists, determining that a wire obstruction exists; If there is no laser breakpoint, determine that there is no wire obstruction; Includes:

[0009] In one exemplary embodiment of the present disclosure, the method comprises: Obtaining an image recognition result; In response to the presence of a wire-like obstacle being indicated in both the image recognition result and the line laser recognition result, performing a corresponding obstacle avoidance operation based on the image recognition result and the line laser recognition result; Further includes:

[0010] In an exemplary embodiment of the present disclosure, performing a corresponding obstacle avoidance operation based on the image recognition result and the line laser recognition result includes: Obtaining a first accuracy rate of the historical image recognition result and a second accuracy rate of the historical line laser recognition result respectively; Determine a first weight corresponding to the image recognition result based on the first accuracy rate, and determine a second weight corresponding to the line laser recognition result based on the second accuracy rate; determining a comprehensive recognition result based on the image recognition result, the first weight, the line laser recognition result, and the second weight; Based on the comprehensive recognition result, perform a corresponding obstacle avoidance operation for the wire-like obstacle; Includes:

[0011] In one exemplary embodiment of the present disclosure, the method comprises: acquiring image recognition results and line laser recognition results multiple times in response to the image recognition result indicating the presence of a wire-like obstacle and the line laser recognition result indicating the absence of a wire-like obstacle, or in response to the image recognition result indicating the absence of a wire-like obstacle and the line laser recognition result indicating the presence of a wire-like obstacle; Re-determining whether or not a wire-like obstacle exists based on the image recognition results and the line laser recognition results obtained multiple times; Further includes:

[0012] In an exemplary embodiment of the present disclosure, the image recognition result includes a location box corresponding to an obstacle, and the method includes: determining first position coordinate information of the wire obstacle based on the coordinates of each laser breakpoint in the line laser recognition image; Performing semantic segmentation on the image recognition result to determine whether each pixel point in the location box corresponding to the wire-like obstacle belongs to the wire-like obstacle; If the pixel point belongs to the wire-like obstacle, mark the pixel point to determine second position coordinate information of the wire-like obstacle; determining final position coordinate information based on the first position coordinate information and the second position coordinate information; Executing a corresponding obstacle avoidance operation based on the final position coordinate information; Further includes:

[0013] In one exemplary embodiment of the present disclosure, the method comprises: Determining the start and end points of the wire obstacle; A first obstacle avoidance operation is performed in a first area where the start point and end point of the wire-like obstacle are present, and a second obstacle avoidance operation is performed in a second area other than the first area, and the avoidance distance of the first obstacle avoidance operation is greater than the obstacle avoidance distance of the second obstacle avoidance operation; Further includes:

[0014] In an exemplary embodiment of the present disclosure, determining the start point and the end point of the wire obstacle includes: The method further comprises recognizing the image recognition result through a target image recognition algorithm to determine the start and end points of the wire obstacle.

[0015] In an exemplary embodiment of the present disclosure, determining the start point and the end point of the wire obstacle includes: determining an end point position of the wire-like obstacle based on the first position coordinate information of the wire-like obstacle; determining the end point positions as the start and end points of the wire obstacle; Includes:

[0016] In an exemplary embodiment of the present disclosure, determining the start point and the end point of the wire obstacle includes: Obtaining multiple line laser recognition results from the self-propelled device, and determining all corresponding pixel points in the wire-like obstacle range based on the multiple line laser recognition results; Performing clustering processing on all the pixel points to determine pixel point breakpoints within the wire obstacle range; determining the pixel point breakpoints as the start and end points of the wire obstacle; Includes:

[0017] According to a second aspect of the present disclosure, an acquisition unit arranged to acquire a line laser recognition image of the self-propelled device; a result determining unit configured to determine whether a wire-like obstacle exists based on the line laser recognition image to obtain a line laser recognition result; a processing unit configured to perform a corresponding obstacle avoidance action based on the line laser recognition result; The present invention provides an obstacle avoidance device for a self-propelled device based on a line laser, comprising:

[0018] According to a third aspect of the present disclosure, there is provided a computer-readable medium having stored thereon a computer program that, when executed by a processor, realizes the obstacle avoidance method for a self-propelled device based on a line laser described in the first aspect of the above examples.

[0019] According to a fourth aspect of the present disclosure, there is provided an electronic device including one or more processors and a memory device that stores one or more programs and is configured such that, when the one or more programs are executed by the one or more processors, the one or more processors implement the obstacle avoidance method for a self-propelled device based on a line laser described in the first aspect of the above embodiment.

[0020] The present disclosure provides a method and device for obstacle avoidance for self-propelled equipment based on a line laser, which obtains a line laser recognition image of the self-propelled equipment, determines whether a wire-like obstacle exists based on the line laser recognition image, obtains a line laser recognition result, and performs a corresponding obstacle avoidance operation based on the line laser recognition result. According to the technical solution of the present disclosure, the existence of a wire-like obstacle is recognized through image recognition and line laser recognition, and based on the image recognition, the coordinates of the wire-like obstacle are recognized by pixel marks to obtain a more accurate wire range, and at the same time, the coordinates of the wire-like obstacle are determined by the laser breakpoint of the line laser, and an avoidance operation is performed based on the image recognition result and the line laser recognition result, thereby improving the accuracy of determining the wire range and the avoidance operation.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. [Brief explanation of the drawings]

[0022] The drawings herein are incorporated into the specification and constitute a part of this specification, illustrate embodiments pertaining to the present disclosure, and are arranged to explain the principles of the present disclosure together with the specification. Of course, the drawings described below are merely some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without any creative effort. [Figure 1] 1 is a structural schematic diagram of a self-propelled device according to an exemplary method; [Figure 2] 1 is a structural schematic diagram of a line laser module in a self-propelled device according to an exemplary method; [Figure 3] 1 is a flowchart of an obstacle avoidance method for a line laser-based self-propelled device in accordance with an exemplary embodiment. [Figure 4] 1 is a schematic diagram of an acquired line laser image of a self-propelled device in accordance with an exemplary embodiment; [Figure 5] FIG. 10 is a schematic diagram of a line laser image captured by another self-propelled device in accordance with an exemplary embodiment. [Figure 6]10 is a schematic diagram of an acquired line laser image of a further self-propelled device according to an exemplary embodiment; [Figure 7] FIG. 1 is a schematic diagram of an image recognition structure according to an exemplary embodiment. [Figure 8] FIG. 10 is another image recognition structure schematic diagram according to an exemplary embodiment. [Figure 9] FIG. 1 is a block diagram of an obstacle avoidance system for a line laser-based self-propelled machine according to an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0023] Exemplary embodiments will now be described more fully with reference to the drawings. However, the exemplary embodiments may be implemented in a variety of forms and should not be understood as being limited to the examples set forth herein. Rather, providing these embodiments will make the present disclosure more thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to fully understand the embodiments of the present disclosure. However, those skilled in the art will recognize that the technical solutions of the present disclosure may be implemented without one or more of the specific details, or may employ other methods, components, devices, steps, etc. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0024] Furthermore, the drawings are merely schematic diagrams of the present disclosure and are not necessarily drawn to scale. In the drawings, the same reference numerals indicate the same or similar parts, and therefore, redundant descriptions thereof will be omitted. Some block diagrams shown in the drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented by software, or may be implemented in one or more hardware modules or integrated circuits, or may be implemented in different networks and / or processor and / or microcontroller devices.

[0025] The embodiments of the present disclosure provide possible application scenarios including a self-propelled device 100, such as a robotic vacuum cleaner, a mop robot, a vacuum cleaner, or a weeder. In one embodiment, as shown in FIG. 1 , a household robotic vacuum cleaner is used as an example. During the operation of the robotic vacuum cleaner, it can clean based on a preset path or an automatically planned path. However, it is inevitable that the robotic vacuum cleaner will get stuck at a location such as a chair or a table. In this case, the robotic vacuum cleaner can recognize the obstacle in a cloud server, a local server, or its own storage system and mark the location as an obstacle location. The next time the robotic vacuum cleaner travels to that location, it will automatically avoid the obstacle. In this embodiment, the robot may be equipped with a touch-sensitive display to receive operation commands input by a user, or may be controlled via a mobile device. The self-propelled equipment may be equipped with various sensors, such as sensor devices such as buffers, cliff sensors, ultrasonic sensors, infrared sensors, magnetometers, accelerometers, gyroscopes, and odometers (the specific structure of each sensor will not be described in detail, but any one of the above sensors can be adopted and should be arranged in this self-propelled equipment), and the robot may be equipped with a wireless communication module such as a WIFI module or a Bluetooth module to connect to a smart terminal or a server and receive operation commands sent from the smart terminal or server through the wireless communication module.

[0026] An optional embodiment of the present disclosure provides a line laser module, and another optional embodiment of the present disclosure provides an autonomous mobile device, in which the line laser module should be arranged as the autonomous mobile device, and specifically, the autonomous mobile device is an intelligent cleaning device, such as a robot vacuum cleaner, a mop robot, a floor polishing robot, or a weeding robot. For convenience of explanation, this embodiment takes a robot vacuum cleaner as an example to describe the technical solution of the present disclosure.

[0027] As shown in Fig. 2, in one alternative embodiment of the present disclosure, an autonomous mobile device may include a device body, a sensing system, a control system, a drive system, a cleaning system, an energy system, and a human-machine interaction system. The systems cooperate with each other to enable the autonomous mobile device to move autonomously and achieve a cleaning function. The functional elements constituting each of the above systems in the autonomous mobile device are integrated and installed within the device body.

[0028] The device body has a generally circular shape (both front and back are circular), but may have other shapes, including but not limited to a generally D-shape with a square front and a circular rear. The sensing system includes a line laser module located above or to the side of the device body, and a main control unit of the control system is connected to the line laser module and controls the functions of the autonomous mobile device according to the sensing results of the line laser module.

[0029] In the embodiments provided herein, the specific location of the line laser module on the device body is not limited, and may be, for example, but not limited to, the front, rear, left, right, top, middle, or bottom of the device body. Furthermore, the line laser module may be installed at the middle, top, or bottom position in the height direction of the device body.

[0030] In some possible embodiments provided in the present disclosure, the autonomous mobile device moves forward to perform work tasks, and in order to better detect environmental information ahead, the line laser module is installed on the front side of the device body, the front side being the side that the device body faces during the forward movement process of the autonomous mobile device.

[0031] 2 , the line laser module includes a main body 140 and a first image acquisition assembly, the first image acquisition assembly including a first camera 120 mounted on the main body 140, at least one pair of line laser emitting devices 110, and a first image processing module, the pair of line laser emitting devices 110 being located on both sides of the first camera 120 and configured to emit line lasers projected in a linear fashion, the first camera 120 operating in cooperation with the line laser emitting device 110 and configured to collect a first environmental image, and the first image processing module being capable of obtaining obstacle distance information based on the first environmental image. The line laser emitting device 110 is movably connected to the main body 140 and / or the main body 140 has a movable structure, so that the azimuth angle and rotation angle of the line laser emitting device 110 are adjustable.

[0032] In the line laser module provided in the embodiment of the present disclosure, the first camera 120 cooperates with the line laser emitting device 110 to recognize obstacles or terrain in front of the device body and perform corresponding obstacle avoidance or cleaning operations. Because the line laser emitting device 110 is movably connected to the body 140 and / or the body 140 has a movable structure, the azimuth angle and rotation angle of the line laser emitting device 110 can be adjusted by adjusting the position of the line laser emitting device 110 relative to the body 140 and / or adjusting the relative positions of each component of the body 140 itself. This makes it easy to adjust the irradiation angle and irradiation range of the line laser emitted from the line laser emitting device 110 when assembling the line laser emitting device 110 to the body 140, and allows the self-calibration of the line laser emitting device 110 and the cooperative calibration with the first camera 120 to be quickly and easily realized, resulting in simple and convenient operation and improved assembly efficiency.

[0033] Furthermore, by appropriately adjusting the azimuth angle of the line laser emitting device 110, the line laser can be positioned within the viewing angle range of the first camera 120, thereby ensuring that the first camera 120 accurately and comprehensively captures the light emitted from the line laser emitting device 110 reflected by obstacles, thereby improving the accuracy and comprehensiveness of the environmental image captured by the first camera 120. As can be seen, after adjusting the azimuth angles of the line laser emitting devices 110 on both sides of the first camera 120, the line lasers on both sides are fixed with adhesive. By appropriately adjusting the rotation angle of the line laser emitting device 110, the line laser can be made perpendicular to the horizontal plane, which is advantageous for expanding the distance measurement range.

[0034] Specifically, on the one hand, since the line laser emitting device 110 is movably connected to the main body 140, the azimuth angle and rotation angle of the line laser emitting device 110 can be adjusted by adjusting the position of the line laser emitting device 110 relative to the main body 140; on the other hand, since the main body 140 has a movable structure, the azimuth angle and rotation angle of the line laser emitting device 110 can be adjusted by adjusting the relative positions of each component of the main body 140 itself; and on the other hand, since the line laser emitting device 110 is movably connected to the main body 140 and the main body 140 has a movable structure, the azimuth angle and rotation angle of the line laser emitting device 110 can be adjusted by adjusting the position of the line laser emitting device 110 relative to the main body 140 and adjusting the relative positions of each component of the main body 140 itself. The embodiments provided in the present disclosure can adjust the azimuth angle and rotation angle of the line laser emitting device 110 by different methods, thereby meeting the needs of different structures of the main body 140 and different connection methods between the line laser emitting device 110 and the main body 140.

[0035] Furthermore, the line laser emitting device 110 is arranged to emit a line laser that is projected in a line shape, and when the line laser emitting device 110 emits a laser surface outward, after the laser surface reaches an obstacle, a line laser is formed on the surface of the obstacle, and an environmental image is detected through the line laser. As can be understood, a wave mirror may be installed in the light emitting direction (e.g., forward) of the line laser emitting device 110. Specifically, the wave mirror is a concave lens, and when the concave lens is installed in front of the laser tube, light of a specific wavelength (e.g., infrared) is emitted from the laser tube and becomes divergent light after passing through the concave lens, so that a straight line is formed on a plane perpendicular to the optical path.

[0036] The line laser emitting device 110 is distributed on both sides of the first camera 120, and the first camera 120 and the line laser emitting device 110 work in cooperation with each other; that is, the first environmental image collected by the first camera 120 is the reflected light of the line laser emitted from the line laser emitting device 110 reflected by an obstacle, and obstacle distance information can be obtained based on the first environmental image, and the distance between the obstacle and the device body or the line laser module can be measured, and corresponding obstacle avoidance operations can be performed.

[0037] The control system is installed on a circuit board within the robot's body and includes a computing processor (e.g., a central processing unit and an application processor) that communicates with non-transitory memory such as a hard disk, flash memory, or random access memory. The application processor uses localization algorithms such as simultaneous localization and mapping (SLAM) to create a real-time map of the robot's environment based on obstacle information fed back from the laser ranging device. It also uses distance and speed information fed back from sensors such as buffer sensors, cliff sensors, magnetometers, accelerometers, gyroscopes, and odometers to comprehensively determine the current operating state and location of the vacuum cleaner, as well as its current state (e.g., whether it has crossed a threshold, stepped on a carpet, reached a cliff, is clogged above or below, or the dustbin is full or has been picked up). It then proposes specific next action strategies according to different situations, allowing the robot's operation to better meet the owner's needs and provide a better user experience.

[0038] The drive system can operate the robot to move across a ground surface based on drive commands having distance and angle information (e.g., x, y, and θ components). The drive system includes a drive wheel module capable of simultaneously controlling the left and right wheels. Preferably, the drive wheel modules include a left drive wheel module and a right drive wheel module, respectively, for more precise control of the machine's movement. The left and right drive wheel modules are installed opposite each other along a horizontal axis defined by the body. To enable the robot to move more stably or have stronger mobility on the ground surface, the robot may include one or more driven wheels, including, but not limited to, swivel wheels. The drive wheel module includes a running wheel, a drive motor, and a control circuit for controlling the drive motor. The drive wheel module may be connected to a circuit for measuring drive current or an odometer. The drive wheel module may be detachably connected to the body for easy attachment, detachment, and maintenance. The drive wheel may be movably fastened to the robot body, e.g., may have a bias drop suspension system that is rotatably attached and subjected to a spring bias that is biased downward and away from the robot body. The spring bias allows the drive wheels to maintain contact and traction with the ground with a constant landing force, while the cleaning element of the self-propelled device 100 also contacts the ground with a constant pressure.

[0039] The cleaning system may be a dry cleaning system and / or a wet cleaning system. As a dry cleaning system, the main cleaning function comes from the cleaning system, which is composed of a roller brush, a dust box, a fan, an exhaust port, and the connecting members between the four. The roller brush, which has a certain interference with the ground, sweeps up debris on the ground and draws it forward through the dust removal port between the roller brush and the dust box. The dust box is then sucked into the dust box by suction gas generated by the fan. The dry cleaning system may further include a side brush 152 with a rotating shaft, which is positioned at a certain angle to the ground to move debris to the roller brush area of ​​the cleaning system.

[0040] The energy system includes a rechargeable battery, such as a nickel-metal hydride battery or a lithium battery. A charging control circuit, a battery pack charging temperature detection circuit, and a battery low-voltage monitoring circuit are connected to the rechargeable battery, and the charging control circuit, battery pack charging temperature detection circuit, and battery low-voltage monitoring circuit may be further connected to a single-chip microcomputer control circuit. The host is charged by connecting it to a charging stand via charging electrodes installed on the side or bottom of the main body. If dust adheres to the exposed charging electrodes, the accumulated charge during charging can melt and deform the plastic body around the electrodes, and even deform the electrodes themselves, preventing normal charging.

[0041] The human-machine interaction system includes buttons on a host panel for the user to select functions, and may further include a display and / or indicator lights and / or a horn, which indicate the current machine status or function selection items to the user, and may further include a mobile phone client program. In the case of a route navigation type self-propelled device, the mobile phone client can show the user a map of the environment in which the device is located and the location of the machine, and can provide the user with a richer and more user-friendly function items.

[0042] FIG. 3 is a flowchart of an obstacle avoidance method for a line laser-based self-propelled device according to one exemplary embodiment.

[0043] As shown in FIG. 3, the obstacle avoidance method for self-propelled equipment based on line laser includes:

[0044] Step S301 is to acquire a line laser recognition image of the self-propelled device.

[0045] By irradiating the ground with a line laser, a line laser recognition image can be obtained.

[0046] Step S302 is to determine whether there is a wire obstacle based on the line laser recognition image, and obtain a line laser recognition result; In one embodiment, step S302 preferably includes: analyzing the line laser recognition image to determine whether a laser breakpoint exists for each line laser in the line laser recognition image; If a laser breakpoint exists, determining that a wire obstruction exists; If there is no laser breakpoint, determine that there is no wire obstruction; Includes:

[0047] If there is no obstacle within the obstacle avoidance distance range that the self-propelled device is interested in, when the line laser is irradiated onto the ground, a set of continuous horizontal plane height (i.e., height 0) distance measurement data is acquired. As shown in Figure 4, the ground light is irradiated onto the wall in the distance and rises, but it is outside the obstacle avoidance distance range that the self-propelled device is interested in.

[0048] When a wire (wire of a non-absorbent color or material) appears within the operating range of the line laser, part of the light that should be irradiated to the ground is blocked by the wire itself, and the ranging data at this time shows a small number of off-axis ranging points with a certain height, as shown in Figure 5. As the arrows indicate discrete ranging points with a certain height, they are caused by hitting the wire. At the same time, the light is blocked by the wire itself, forming an optical breakpoint.

[0049] As shown in Figure 6, in the case of wires made of light-absorbing materials such as black, the light irradiated onto the wire itself cannot be observed, and the distance measurement data in this case is characterized by the fact that the continuous horizontal plane height distance measurement data is cut off.

[0050] In this way, by using the break-off distance measurement point or the feature that the line laser encounters the wire along the ground and cuts it, the distance to the wire can be determined and its position can be obtained, so that the wire can be identified and its position information can be obtained at the same time. Preferably, the feature of being cut is used for the determination, i.e., the laser breakpoint, which is more effective.

[0051] In some embodiments, in order to improve the accuracy of judgment, a comprehensive judgment can be made based on the multiple laser breakpoints present in the recognition results of multiple / multiple line lasers, and if the multiple laser breakpoints present in the multiple / multiple initial line laser recognition results form continuous laser breakpoints longer than a predetermined length, it is determined that a wire-like obstacle exists in the line laser recognition results.

[0052] Step S303 is to perform a corresponding obstacle avoidance operation based on the line laser recognition result.

[0053] In this embodiment, the presence or absence of a wire-like obstacle is determined by determining whether a laser breakpoint exists in the line laser recognition image, and if a wire-like obstacle exists, a corresponding obstacle avoidance operation is performed; if no wire-like obstacle exists, the self-propelled equipment runs normally without performing any avoidance operation.

[0054] In one embodiment, the method preferably comprises: Obtaining an image recognition result; In response to the presence of a wire-like obstacle being indicated in both the image recognition result and the line laser recognition result, performing a corresponding obstacle avoidance operation based on the image recognition result and the line laser recognition result; Further includes:

[0055] In this embodiment, the image recognition result may be obtained, and whether or not there is a wire-like obstacle may be determined based on the image recognition result, and the wire-like obstacle may be individually recognized based on the image recognition result.

[0056] However, there is a possibility of misjudgment in image recognition, and overfitting to the characteristics of line laser ranging (for example, if part of the ground material absorbs / reflects light, when making a judgment by searching for breakpoints, such a scene may be mistakenly judged to be a wire), so when used as a condition basis for triggering obstacle avoidance behavior, in order to ensure the accuracy of the results, it is possible to recognize wire-like obstacles by applying the image recognition results and line laser recognition results simultaneously.

[0057] In one embodiment, preferably, performing a corresponding obstacle avoidance operation based on the image recognition result and the line laser recognition result includes: Obtaining a first accuracy rate of the historical image recognition result and a second accuracy rate of the historical line laser recognition result respectively; Determine a first weight corresponding to the image recognition result based on the first accuracy rate, and determine a second weight corresponding to the line laser recognition result based on the second accuracy rate; determining a comprehensive recognition result based on the image recognition result, the first weight, the line laser recognition result, and the second weight; Based on the comprehensive recognition result, perform a corresponding obstacle avoidance operation for the wire-like obstacle; Includes:

[0058] In this embodiment, the weights of the image recognition results and the line laser recognition results can be determined based on the accuracy rates of the results. For example, for a recognition method that often makes misjudgments and has low accuracy, the weight value can be set low, while for a recognition method that has high accuracy, the corresponding weight value can be set high, thereby ensuring higher accuracy of the recognition results.

[0059] Furthermore, if the presence of a wire-like obstacle cannot be determined from the image recognition results and line laser recognition results collected once, multiple recognition verifications are required for each recognition method.

[0060] Among these, a recognition method with a high recognition accuracy rate can conclude that a wire is present even with fewer recognition attempts, whereas a recognition method with a low recognition accuracy rate requires more recognition attempts to conclude that a wire is present, after which both methods are combined for a determination. Alternatively, if one method's recognition accuracy rate is sufficiently high, that method can be relied upon primarily, with the other method used as a simple supplement or ignored. For example, if the oversight rate and false positive rate of the image recognition results are low, the presence of a wire can be determined with only a small number of ranging features, whereas if the false positive rate is high, more ranging features are required to determine the presence of a wire.

[0061] Preferably, in response to the image recognition result indicating the presence of a wire-like obstacle and the line laser recognition result indicating the absence of a wire-like obstacle, or in response to the image recognition result indicating the absence of a wire-like obstacle and the line laser recognition result indicating the presence of a wire-like obstacle, the image recognition result and the line laser recognition result are obtained multiple times, and whether or not a wire-like obstacle exists is re-determined based on the image recognition result and the line laser recognition result obtained multiple times.

[0062] When only the image recognition result or the line laser recognition result indicates the presence of a wire-like obstacle within a certain area, multiple image recognition results or multiple line laser recognition results are required. The multiple recognition basically adopts a probability graph method, that is, instead of making a direct judgment based on a single recognition result, the probability of the existence of an obstacle at a certain location is continuously adjusted through multiple recognitions, so that if a recognition is made once, the probability of the obstacle existing in the corresponding area is increased, and if there is no recognition result of the target object in a certain frame, the probability of the obstacle existing in the robot's field of view angle is reduced. In this way, after multiple recognitions, an obstacle existence probability distribution for each area is formed, and then a final judgment is made based on the requirements of a preset probability threshold, and the self-propelled equipment performs corresponding actions such as obstacle avoidance or ignorance based on this.

[0063] In this way, by detecting the above features in the ranging data of each frame, it is possible to determine which parts of the entire known map range may contain ranging features due to scattered rays, and to determine the approximate distribution location of scattered rays.

[0064] It should be noted that the line laser recognition result of a single position can be used to determine that a wire obstacle exists at a certain position, and the process can be repeated by constantly steering or driving to finally obtain the distribution positions of all wire obstacles in the environment; or, first, one recognition result can be repeatedly used to obtain the distribution positions of all current obstacles in the environment, then the other recognition result can be repeatedly used to obtain the distribution positions of all current obstacles in the environment, and finally, both can be taken into consideration comprehensively to obtain the final distribution positions.

[0065] In one embodiment, preferably, the image recognition result includes a location box corresponding to the obstacle, and the method further comprises: determining first position coordinate information of the wire obstacle based on the coordinates of each laser breakpoint in the line laser recognition image; Performing semantic segmentation on the image recognition result to determine whether each pixel point in the location box corresponding to the wire-like obstacle belongs to the wire-like obstacle; Further comprising: As shown in Figure 7, in the image recognition results, the target object is given a minimum rectangle in the horizontal and vertical directions to determine the object's range, but obstacles such as wires, which are long and thin and may have very irregular shapes, clearly have a small effective portion in the image, and most of them are relatively safe areas that do not require obstacle avoidance.

[0066] On the other hand, the present disclosure uses the bounding box of an already recognized object to mark each pixel within the recognition box as belonging to the target object, as shown in Figure 8. By processing in this way, the recognition result does not only provide a standard rectangular image as in the past, but also provides an obstacle area with a more specific shape, which contributes to obtaining the specific position coordinates of each part of the wire. This improves the accuracy of wire position information in image recognition.

[0067] If the pixel point belongs to the wire-like obstacle, mark the pixel point to determine second position coordinate information of the wire-like obstacle; determining final position coordinate information based on the first position coordinate information and the second position coordinate information; In this embodiment, the first position coordinate information and the second position coordinate information are unioned, and the union between them can be determined as the final position coordinate information.

[0068] Based on the final position coordinate information, a corresponding obstacle avoidance operation is performed.

[0069] Self-propelled equipment can easily get the wire head at the end of the wire caught in the side brush, roller brush rotating part, etc., affecting subsequent work, but the middle part of the wire is relatively safe. By distinguishing between these two parts, the robot's response actions can be better adjusted.

[0070] In one embodiment, the method preferably comprises: Determining the start and end points of the wire obstacle; A first obstacle avoidance operation is performed in a first area where the start point and end point of the wire-like obstacle are present, and a second obstacle avoidance operation is performed in a second area other than the first area, and the avoidance distance of the first obstacle avoidance operation is greater than the obstacle avoidance distance of the second obstacle avoidance operation; Further includes:

[0071] In this embodiment, after determining the start and end points of the wire head of the wire-like obstacle, obstacle avoidance can be triggered at a longer range for the wire head. For example, for the non-wire head, a method is adopted to minimize sweeping errors as much as possible, and the wire is allowed to approach a position within range A at most, as it is relatively safe. For the wire head, a method is adopted to prevent the wire head from being entangled as much as possible, and the autonomous mobile device is allowed to approach a position within range B at most from the wire head, and the self-propelled device B>A.

[0072] Specifically, by analyzing the direction of the wire in the image, it can be determined whether the observed local area is a wire head. Specifically, the wire head determination may be performed by the following several methods.

[0073] In one embodiment, preferably, determining the start point and the end point of the wire obstacle includes: The method further comprises recognizing the image recognition result through a target image recognition algorithm to determine the start and end points of the wire obstacle.

[0074] In this embodiment, the line head is directly used as an object type for recognition training, so that a specific range of line heads can be recognized.

[0075] In one embodiment, preferably, determining the start point and the end point of the wire obstacle includes: determining an end point position of the wire-like obstacle based on the first position coordinate information of the wire-like obstacle; determining the end point positions as the start and end points of the wire obstacle; Includes:

[0076] In this embodiment, after recognizing the entire wire from the image, a specific wire range, i.e., the first position coordinate information of the wire-like obstacle, is obtained through semantic segmentation, and all pixel points belonging to the wire-like obstacle are tracked to determine the locations of its continuous endpoints. For example, as shown in Figure 8, if it is determined that the endpoints of the wire are located in the upper left corner and the lower right corner within the box range, these endpoints can be considered to be the locations where the wire head is located. If it is determined that the upper left corner is the wire head, the obstacle avoidance distance for that wire head range can be increased. Although the lower right corner is not a true wire head, the part outside the field of view belongs to the current blind spot and there is a risk of it being mistakenly blocked, so the obstacle avoidance distance for that range can also be increased. In addition, a dedicated strategy may be set up for false line heads, that is, it is possible that the authenticity of a line head located at the edge of the entire image is unknown, and when traveling near such a line head, the current action of the self-propelled device can be temporarily stopped, the field of view can be turned toward the unknown line head, and the device can be made to observe, recognize and judge again, and then resume the previous action based on the updated recognition result.

[0077] In one embodiment, preferably, determining the start point and the end point of the wire obstacle includes: Obtaining multiple line laser recognition results from the self-propelled device, and determining all corresponding pixel points in the wire-like obstacle range based on the multiple line laser recognition results; Performing clustering processing on all the pixel points to determine pixel point breakpoints within the wire obstacle range; determining the pixel point breakpoints as the start and end points of the wire obstacle; Includes:

[0078] In this embodiment, the results of multiple line laser scans are used to obtain all range points that are determined to be wire, and clustering is used to determine which ranges belong to "one line." Next, by searching along the wire range, it is possible to determine the specific position at which the range is broken, and the position of the break can then be considered as the wire head position.

[0079] In this way, by constantly performing image recognition while the self-propelled equipment is moving and by performing steps such as segmenting and processing the obtained images, combined with the room map information created by the self-propelled equipment, it is possible to determine the locations of wire rods within the entire known map range, and further to know which locations within those locations may belong to high-risk wire head areas, making it possible to create various behavioral strategies for different types of wire heads.

[0080] It should be noted that based on the acquired final image recognition result and the above-mentioned final line laser recognition result, it is not limited to being configured to perform obstacle avoidance operations, but only needs to be configured to recognize the type and position of obstacles on the wire, and the information can be fully configured for other purposes according to actual needs.

[0081] FIG. 9 is a block diagram of an obstacle avoidance system for a line laser-based self-propelled machine according to one exemplary embodiment.

[0082] As shown in FIG. 9 , according to a second aspect of the present disclosure, there is provided an obstacle avoidance device for a self-propelled device based on a line laser, the device comprising: an acquisition unit 91 arranged to acquire a line laser recognition image of the self-propelled device; a result determining unit 92, which is configured to determine whether there is a wire obstacle based on the line laser recognition image, and obtain a line laser recognition result; a processing unit 93 configured to perform corresponding obstacle avoidance actions based on the line laser recognition result; Includes:

[0083] In one embodiment, the result determination unit preferably comprises: Analyzing the line laser recognition image and determining whether or not a laser breakpoint exists for each line laser in the line laser recognition image; If a laser breakpoint exists, it is determined that a wire obstacle exists; If no laser breakpoint is present, it is determined that no wire obstruction is present.

[0084] In one embodiment, preferably, the device comprises: a result obtaining unit arranged to obtain an image recognition result; The processing unit is configured to, in response to both the image recognition result and the line laser recognition result indicating the presence of a wire-like obstacle, perform a corresponding obstacle avoidance action based on the image recognition result and the line laser recognition result.

[0085] In one embodiment, the processing unit preferably comprises: Obtain a first accuracy rate of the historical image recognition result and a second accuracy rate of the historical line laser recognition result respectively; Determine a first weight corresponding to the image recognition result based on the first accuracy rate, and determine a second weight corresponding to the line laser recognition result based on the second accuracy rate; Determine a comprehensive recognition result based on the image recognition result, the first weight, the line laser recognition result, and the second weight; The robot is configured to perform a corresponding obstacle avoidance action on the wire-like obstacle based on the comprehensive recognition result.

[0086] In one embodiment, preferably the processing unit further comprises: acquiring image recognition results and line laser recognition results multiple times in response to the image recognition result indicating the presence of a wire-like obstacle and the line laser recognition result indicating the absence of a wire-like obstacle, or in response to the image recognition result indicating the absence of a wire-like obstacle and the line laser recognition result indicating the presence of a wire-like obstacle; The apparatus is configured to re-determine whether or not a wire-like obstacle exists based on the image recognition results and the line laser recognition results obtained multiple times.

[0087] In one embodiment, preferably, the image recognition result includes a location box corresponding to the obstacle, and the device: a first coordinate determination unit arranged to determine first position coordinate information of the wire obstacle based on the coordinates of each laser breakpoint in the line laser recognition image; a pixel point determining unit, configured to perform semantic segmentation on the image recognition result, and determine whether each image pixel point in the position box corresponding to the wire-like obstacle belongs to the wire-like obstacle; a marking unit, which is arranged to mark the pixel point if the pixel point belongs to the wire-like obstacle, to determine second position coordinate information of the wire-like obstacle; a second coordinate determination unit arranged to determine final position coordinate information based on the first position coordinate information and the second position coordinate information; Further comprising: The processing unit is further configured to perform a corresponding obstacle avoidance action based on the final position coordinate information.

[0088] In one embodiment, preferably, the device comprises: The wire obstacle detection device further includes a start point and end point determination unit configured to determine a start point and an end point of the wire obstacle, The processing unit is arranged to perform a first obstacle avoidance operation in a first area where the start point and end point of the wire-like obstacle are located, and to perform a second obstacle avoidance operation in a second area other than the first area, wherein the avoidance distance of the first obstacle avoidance operation is greater than the obstacle avoidance distance of the second obstacle avoidance operation.

[0089] In one embodiment, preferably, the start point / end point determination unit comprises: The target image recognition algorithm is configured to recognize the image recognition result and determine the start and end points of the wire obstacle.

[0090] In one embodiment, preferably, the start point / end point determination unit comprises: Determine the end point position of the wire-like obstacle based on the first position coordinate information of the wire-like obstacle; The end points are positioned to define the start and end points of the wire obstacle.

[0091] In one embodiment, preferably, the start point / end point determination unit comprises: Obtaining multiple line laser recognition results from the self-propelled device, and determining all corresponding pixel points in the wire obstacle range based on the multiple line laser recognition results; Perform clustering processing on all the pixel points to determine pixel point breakpoints within the wire obstacle range; The pixel point breakpoints are positioned to define the start and end points of the wire obstacle.

[0092] In particular, according to embodiments of the present disclosure, the processes described with reference to the flowcharts above may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program embodied on a computer-readable medium, the computer program including program code configured to perform the methods illustrated in the flowcharts.

[0093] For illustrative purposes, the computer-readable medium referred to in this disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or has stored thereon a program usable by or in connection with a command execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a propagated data signal, in baseband or as part of a carrier, carrying computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which is capable of transmitting, propagating, or transmitting a program arranged for use by or in connection with a command execution system, apparatus, or device. Program code contained in a computer-readable medium may be transmitted over any suitable medium, including, but not limited to, wireless, wire, cable, RF, etc., or any suitable combination of the above.

[0094] The flowcharts and block diagrams in the drawings illustrate possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which includes one or more executable commands arranged to implement a certain logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from the order marked in the drawings. For example, two successively shown blocks may in fact be executed substantially in parallel, or in some cases, may be executed in the reverse order, as determined by the functionality involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented by a dedicated hardware-based system that performs a predetermined function or operation, or by a combination of dedicated hardware and computer commands.

[0095] The units according to the embodiments of the present disclosure may be implemented by software or hardware, and the units described may be installed in a processor, and the names of these units may not necessarily be limiting of the units themselves.

[0096] In another aspect, the present application further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment, or may exist independently without being assembled to the electronic device. The computer-readable medium stores one or more programs, which, when executed by one of the electronic devices, cause the electronic device to realize the obstacle avoidance method for a self-propelled device based on a line laser described in the above embodiment.

[0097] For example, the electronic device can realize the steps of: step S301 of acquiring a line laser recognition image of the self-propelled device, step S302 of determining whether or not a wire-like obstacle exists based on the line laser recognition image and obtaining a line laser recognition result, as shown in FIG. 3; and step S303 of performing a corresponding obstacle avoidance operation based on the line laser recognition result.

[0098] It should be noted that although the above detailed description refers to several modules or units of an apparatus configured to perform operations, such division is not mandatory. In fact, according to an embodiment of the present disclosure, the features and functions of two or more of the modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one of the modules or units described above may be further divided and embodied in multiple modules or units.

[0099] Furthermore, although the figures depict steps of the methods in this disclosure in a particular order, this does not require or imply that the steps must be performed in that particular order, or that all of the steps shown must be performed to achieve a desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined and performed as a single step, and / or a single step may be broken down into multiple steps and performed.

[0100] From the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be realized by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be realized in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or a network, and includes some commands to make a computing device (which can be a personal computer, a server, a mobile terminal, a network device, etc.) execute the method according to the embodiments of the present disclosure.

[0101] Other embodiments of the present disclosure will be readily apparent to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses, and adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means known in the art but not disclosed in the present disclosure. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.

Claims

1. 1. A line laser-based obstacle avoidance method for a self-propelled device, comprising: acquiring a line laser recognition image of the self-propelled device; determining whether a wire obstacle exists based on the line laser recognition image to obtain a line laser recognition result; and performing a corresponding obstacle avoidance operation based on the line laser recognition result. A method for obstacle avoidance in a self-propelled device.

2. determining whether a wire-like obstacle exists based on the line laser recognition image to obtain a line laser recognition result; analyzing the line laser recognition image to determine whether a laser breakpoint exists for each line laser in the line laser recognition image; If a laser breakpoint exists, determining that a wire obstruction exists; If no laser breakpoint exists, determining that no wire obstruction exists. The obstacle avoidance method for a self-propelled device according to claim 1.

3. The obstacle avoidance method for the self-propelled device includes: Obtaining an image recognition result; and in response to the presence of a wire-like obstacle being indicated in both the image recognition result and the line laser recognition result, executing a corresponding obstacle avoidance operation based on the image recognition result and the line laser recognition result. The obstacle avoidance method for a self-propelled device according to claim 2.

4. Executing a corresponding obstacle avoidance operation based on the image recognition result and the line laser recognition result, Obtaining a first accuracy rate of the historical image recognition result and a second accuracy rate of the historical line laser recognition result respectively; determining a first weight corresponding to the image recognition result based on the first accuracy rate, and determining a second weight corresponding to the line laser recognition result based on the second accuracy rate; determining a comprehensive recognition result based on the image recognition result, the first weight, the line laser recognition result, and the second weight; and performing a corresponding obstacle avoidance operation on the wire-like obstacle based on the comprehensive recognition result. The obstacle avoidance method for a self-propelled device according to claim 3.

5. The obstacle avoidance method for the self-propelled device includes: acquiring image recognition results and line laser recognition results multiple times in response to the image recognition result indicating the presence of a wire-like obstacle and the line laser recognition result indicating the absence of a wire-like obstacle, or in response to the image recognition result indicating the absence of a wire-like obstacle and the line laser recognition result indicating the presence of a wire-like obstacle; and re-determining whether or not a wire-like obstacle exists based on the image recognition results and the line laser recognition results obtained multiple times. The obstacle avoidance method for a self-propelled device according to claim 3.

6. The image recognition result includes a position box corresponding to an obstacle, and the obstacle avoidance method for the self-propelled device includes: determining first position coordinate information of the wire obstacle based on the coordinates of each laser breakpoint in the line laser recognition image; Performing semantic segmentation on the image recognition result to determine whether each pixel point in the location box corresponding to the wire-like obstacle belongs to the wire-like obstacle; If the pixel point belongs to the wire-like obstacle, mark the pixel point to determine second position coordinate information of the wire-like obstacle; determining final position coordinate information based on the first position coordinate information and the second position coordinate information; and performing a corresponding obstacle avoidance operation based on the final position coordinate information.

4. The obstacle avoidance method for a self-propelled device according to claim 3.

7. The obstacle avoidance method for the self-propelled device includes: determining the start and end points of the wire obstacle; The method further includes executing a first obstacle avoidance operation in a first area where a start point and an end point of the wire-like obstacle exist, and executing a second obstacle avoidance operation in a second area other than the first area, wherein an obstacle avoidance distance of the first obstacle avoidance operation is greater than an obstacle avoidance distance of the second obstacle avoidance operation. The obstacle avoidance method for a self-propelled device according to claim 3.

8. Determining the start and end points of the wire obstacles includes: and recognizing the image recognition result through a target image recognition algorithm to determine the start and end points of the wire obstacle. The obstacle avoidance method for a self-propelled device according to claim 7.

9. Determining the start and end points of the wire obstacles includes: determining an end point position of the wire obstacle based on first position coordinate information of the wire obstacle; and determining the end point positions as the start and end points of the wire obstacle. The obstacle avoidance method for a self-propelled device according to claim 7.

10. Determining the start and end points of the wire obstacles includes: Obtaining multiple line laser recognition results from the self-propelled device, and determining all corresponding pixel points in the wire-like obstacle range based on the multiple line laser recognition results; performing clustering processing on all the pixel points to determine pixel point breakpoints within the wire obstacle range; determining the pixel point breakpoints as the start and end points of the wire obstacle. The obstacle avoidance method for a self-propelled device according to claim 7.

11. 1. An obstacle avoidance device for a self-propelled device based on a line laser, comprising: an acquisition unit configured to acquire a line laser recognition image of the self-propelled device; a result determining unit configured to determine whether a wire obstacle exists based on the line laser recognition image to obtain a line laser recognition result; a processing unit configured to perform a corresponding obstacle avoidance action based on the line laser recognition result; Obstacle avoidance device for self-propelled equipment.

12. 1. A self-propelled device including at least one processor and a memory communicatively coupled to the at least one processor, The memory stores instructions executable by the at least one processor, the instructions being configured to perform a method according to any one of claims 1 to 10. Self-propelled equipment.

13. A computer-readable storage medium having computer-executable instructions stored thereon, comprising: The computer-executable instructions are configured to perform the method of any one of claims 1 to 10. A computer-readable storage medium.

Images