A data processing method for a mobile robot sensor, a storage medium and an electronic device

Abstract

The application discloses a kind of mobile robot sensor collection data processing method and its storage medium, electronic equipment, in the working map of mobile robot, the scene that it is easy to appear fog is demarcated into demisting area;Including the following steps: multiple sensor acquisition data;Traversal all fog interference sensor acquisition data;Judge whether the data of fog interference sensor acquisition is in the demisting area in working map, if yes, then eliminate the data of fog interference sensor acquisition, otherwise continue to judge whether the next fog interference sensor acquisition data is in the demisting area in working map;Using the above technical solution prevents multiple sensor acquisition data from being affected by sensor that is easy to be interfered by fog, produces data deviation after data fusion, avoids misjudging fog as obstacle, improves the safety, stability when mobile robot runs.

Description

Technical Field

[0001] This invention relates to the field of mobile robot technology, and in particular to a data processing method for data collected by sensors in a mobile robot, as well as its storage medium and electronic equipment. Background Technology

[0002] With the improvement of people's living standards and the development of science and technology, mobile robots have been widely recognized by consumers and are gradually being applied in various industries because they replace some repetitive labor. However, when mobile robots are performing tasks, incidents such as collisions with people, falls, or impacts with objects are inevitable, posing a significant threat to people's lives and property. To avoid collisions, sensors are usually installed on mobile robots to identify obstacles. To more accurately identify obstacles, mobile robots are no longer limited to detecting obstacles using a single sensor. They typically use information fusion from multiple sensors (of different types) to perceive the presence of obstacles in the scene, compensating for the deficiencies of individual sensors and improving the environmental understanding and cognitive performance of mobile robots. This is an essential means to provide mobile robots with more accurate environmental information.

[0003] The sensors currently used on mobile robots mainly include LiDAR and depth cameras. However, LiDAR is easily affected by environmental interference. For example, in scenarios such as near cold storage entrances, refrigerators, or humidifiers (these scenarios are usually fixed), water vapor condenses into small water droplets that float in the air, forming a mist that often appears as a cloud of "white mist." LiDAR can easily identify this mist as an obstacle, greatly interfering with its operation. Furthermore, fusing data from LiDAR and other sensors can easily cause anomalies in the mobile robot's positioning and navigation. Summary of the Invention

[0004] Therefore, it is necessary to provide a data processing method for mobile robot sensors to solve the problem that data collected by sensors is easily affected by fog, leading to misjudgment of fog as obstacles and causing anomalies in various aspects of mobile robot, obstacle avoidance, positioning, and navigation.

[0005] To achieve the above objectives, the present invention provides a data processing method for mobile robot sensors. The mobile robot is equipped with multiple sensors for detecting whether there are obstacles. The multiple sensors are divided into sensors that are affected by fog and sensors that are not affected by fog. In the working map of the mobile robot, scenes that are prone to fog are designated as defogging zones.

[0006] Includes the following steps:

[0007] Data is collected by multiple sensors;

[0008] Iterate through the data collected by all sensors affected by fog;

[0009] Determine whether the data collected by the sensor affected by fog is within the defogging area in the working map. If so, discard the data collected by the sensor affected by fog. Otherwise, continue to determine whether the data collected by the next sensor affected by fog is within the defogging area in the working map.

[0010] Furthermore, the sensor affected by fog interference includes lidar.

[0011] Furthermore, the sensors unaffected by fog include depth cameras and ultrasonic radar.

[0012] Furthermore, in the step of determining whether the data collected by the sensor affected by fog is within the defogging area in the working map, if so, the data collected by the sensor affected by fog is discarded; otherwise, the process continues to determine whether the data collected by the next sensor affected by fog is within the defogging area in the working map.

[0013] It also includes determining whether the data collected by the sensor affected by fog is in the clearing area of ​​the working map. If the data collected by the sensor affected by fog is in both the defogging area and the clearing area of ​​the working map, the data collected by the sensor affected by fog is discarded; otherwise, the process continues to determine whether the data collected by the next sensor affected by fog is in the defogging area of ​​the working map.

[0014] Furthermore, the clearing area in the working map is obtained by transforming the clearing area in the robot coordinate system to the working map coordinate system.

[0015] Furthermore, the clearing zone of the robot coordinate system is defined in the coordinate system of the mobile robot as the sensing range of all sensors that are not affected by fog.

[0016] Furthermore, determining whether the data collected by the sensor affected by fog is within the cleared area of ​​the working map includes the following steps:

[0017] A ray is drawn from the data collected by the sensors affected by fog, and the number of intersections between this ray and the boundary of the cleared area on the working map is counted.

[0018] If the number of intersection points is odd, the data collected by the fog-affected sensor will be within the cleared area of ​​the working map; otherwise, the data collected by the fog-affected sensor will be outside the cleared area of ​​the working map.

[0019] Furthermore, determining whether the data collected by the sensor affected by fog is within the defogging area on the working map includes the following steps:

[0020] A ray is drawn from the data collected by the sensor affected by fog, and the number of intersections between this ray and the boundary of the defogging area on the working map is counted.

[0021] If the number of intersection points is odd, the data collected by the fog-affected sensor will be within the defogging area on the working map; otherwise, the data collected by the fog-affected sensor will be outside the defogging area on the working map.

[0022] A storage medium storing a computer program, which, when executed by a processor, implements the steps of the data processing method described above for data collected by the mobile robot's sensors.

[0023] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the data processing method for data collected by the mobile robot sensor described above.

[0024] Unlike existing technologies, the above-mentioned technical solution is mainly applied to mobile robots that use multi-sensor fusion perception. When the mobile robot passes through a scene prone to fog, after multiple sensors collect data, the data collected by sensors that are easily interfered with by fog are removed, and the data collected by the remaining sensors are fused to perceive whether there are obstacles in the scene. This prevents the data collected by multiple sensors from being affected by sensors that are easily interfered with by fog, which could cause data deviation after data fusion, misjudging fog as an obstacle, interfering with the perception of whether there are obstacles in the scene, and causing anomalies in various aspects of the mobile robot, such as obstacle avoidance, localization, and navigation. This improves the safety and stability of the mobile robot during operation. Attached Figure Description

[0025] Figure 1 This is a flowchart of a data processing method for mobile robot sensors according to the present invention;

[0026] Figure 2 This is the cleared area of ​​the mobile robot coordinate system of the present invention;

[0027] Figure 3 This refers to the defogging area around the humidifier in the working map described in this invention.

[0028] Figure 4 This refers to the defogging area at the entrance of the cold storage facility in the working map described in this invention.

[0029] Figure 5 This is a flowchart illustrating a data processing method for mobile robot sensors according to the present invention.

[0030] Explanation of reference numerals in the attached figures:

[0031] 10. Mobile robots

[0032] 101. Sensors unaffected by fog;

[0033] 20. Clearing area of ​​the mobile robot coordinate system;

[0034] 30. Humidifier;

[0035] 40. Defogging area around the humidifier;

[0036] 50. Cold storage;

[0037] 60. Defogging area at the entrance of the cold storage. Detailed Implementation

[0038] To explain in detail the technical content, structural features, objectives, and effects of the technical solution, the following description is provided in conjunction with specific embodiments and accompanying drawings.

[0039] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0040] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0041] In the description of this application, the term "and / or" is used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and A and B exist simultaneously. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" logical relationship.

[0042] In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order relationship between these entities or operations.

[0043] Unless otherwise specified, the use of terms such as “comprising,” “including,” “having,” or other similar expressions in this application is intended to cover non-exclusive inclusion, which does not exclude the presence of additional elements in a process, method, or product that includes the stated elements, such that a process, method, or product that includes a list of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such a process, method, or product.

[0044] Similar to the interpretation in the Patent Examination Guidelines, in this application, expressions such as "greater than," "less than," and "exceeding" are understood to exclude the stated number; expressions such as "above," "below," and "within" are understood to include the stated number. Furthermore, in the description of the embodiments in this application, "multiple" means two or more (including two), and similar expressions related to "multiple" are also interpreted in this way, such as "multiple groups" and "multiple times," unless otherwise explicitly specified.

[0045] In the description of the embodiments of this application, the space-related expressions used, such as "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "vertical," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," indicate the orientation or positional relationship based on the orientation or positional relationship shown in the specific embodiments or drawings. They are only for the purpose of describing the specific embodiments of this application or for the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0046] Unless otherwise expressly specified or limited, the terms "installation," "connection," "linking," "fixing," and "setting," as used in the description of the embodiments of this application, should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two components or the interaction between two components. For those skilled in the art to which this application pertains, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0047] Please see Figures 1 to 5This invention provides a data processing method for mobile robot sensors, mainly applied to mobile robots that use multiple sensors for fusion perception. When the mobile robot passes through a scene prone to fog, after multiple sensors collect data, the data collected by sensors susceptible to fog interference is discarded, and the remaining data are fused to perceive whether there are obstacles in the scene. This prevents data collected by multiple sensors from being affected by sensors susceptible to fog interference, which could lead to data deviation after data fusion, misjudging fog as an obstacle, interfering with the perception of whether there are obstacles in the scene, and causing anomalies in various aspects of the mobile robot, such as obstacle avoidance, positioning, and navigation. This improves the safety and stability of the mobile robot during operation.

[0048] The following is a detailed description of a data processing method for mobile robot sensors provided by the present invention. See also... Figure 1 As shown, a data processing method for mobile robot sensors is described. The mobile robot is equipped with multiple sensors to detect whether there are obstacles. The multiple sensors are divided into sensors that are affected by fog and sensors that are not affected by fog. In the working map of the mobile robot, scenes that are prone to fog are designated as defogging zones.

[0049] Includes the following steps:

[0050] Data is collected by multiple sensors;

[0051] Iterate through the data collected by all sensors affected by fog;

[0052] Determine whether the data collected by the sensor affected by fog is within the defogging area in the working map. If so, discard the data collected by the sensor affected by fog. Otherwise, continue to determine whether the data collected by the next sensor affected by fog is within the defogging area in the working map.

[0053] The aforementioned sensors are of different types, and each type can be one or more. Dividing the sensors into those susceptible to fog interference and those unaffected by fog interference refers to classifying different types of sensors according to the degree of fog interference they are affected by. The fog-affected sensors are those easily affected by fog, such as lidar; the fog-unaffected sensors are those less susceptible to fog, such as depth cameras and ultrasonic radar. The scenarios prone to fogging include, but are not limited to, the area around humidifiers, cold storage entrances, and refrigerator entrances. The defogging zones are defined based on the scenarios prone to fogging; see [link to relevant documentation]. Figure 3As shown, if the fogging occurs around the humidifier 30, then a circular area can be defined on the working map as the defogging zone 40, with the humidifier 30's air outlet as the center and the furthest distance between the center of the humidifier 30's air outlet and the fogging area as the radius. (See also...) Figure 4 As shown, if the foggy scene is at the entrance of cold storage room 50, then the width of the cold storage room 50 door can be used as one side of the defogging zone, and the furthest distance between the cold storage room door and the fog can be used as the other side of the defogging zone. A rectangular area can be defined on the working map as the defogging zone 60 at the entrance of the cold storage room. The aforementioned working map can be understood as a map used by the mobile robot during driving, obstacle avoidance, localization, and navigation, recording the layout of the mobile robot's working environment. The types of working maps include, but are not limited to, grid maps, feature maps, point cloud maps, topology maps, and semantic maps.

[0054] A further improvement to the above technical solution is that, in the step of determining whether the data collected by the sensor affected by fog is within the defogging area of ​​the working map, if so, the data collected by the sensor affected by fog is discarded; otherwise, the process continues to determine whether the data collected by the next sensor affected by fog is within the defogging area of ​​the working map.

[0055] It also includes determining whether the data collected by the sensor affected by fog is in the clearing area of ​​the working map. If the data collected by the sensor affected by fog is in both the defogging area and the clearing area of ​​the working map, the data collected by the sensor affected by fog is discarded. Otherwise, the process continues to determine whether the data collected by the next sensor affected by fog is in the defogging area of ​​the working map.

[0056] See Figure 2 As shown, the cleared area in the working map is obtained by transforming the cleared area 20 of the mobile robot coordinate system to the working map coordinate system. The cleared area 20 of the mobile robot coordinate system defines the area (sensor range) where the data collected by the fog-free sensor 101 is collected (the sensor's sensing range) in the mobile robot's coordinate system. This area can be defined as a triangle. By determining whether the data collected by the fog-free sensor is within the cleared area of ​​the working map, and whether the fog-free sensor's data is within the area of ​​the fog-free sensor's data, it is ensured that a fog-free sensor has collected data at the same location as the fog-free sensor's data. This data can be used to sense whether there are obstacles in the scene. This avoids blindly removing data collected by fog-free sensors in foggy scenes, which could lead to a situation where no other sensor's data can be used to sense obstacles, potentially causing the mobile robot to collide with actual obstacles.

[0057] See Figure 5 As shown, in order to further describe in detail a data processing method for mobile robot sensors, the present invention provides specific embodiments for understanding.

[0058] The sensor affected by fog interference is a lidar.

[0059] The data collected by the sensor when it is affected by fog is the laser point cloud;

[0060] During the operation of the mobile robot, the clearing area (laser clearing area) in the robot coordinate system is transformed to the working map coordinate system to obtain the clearing area in the working map;

[0061] Traverse the laser point cloud;

[0062] Determine if the laser point cloud is within the cleared area of ​​the working map; if so, proceed to the next step; otherwise, traverse the next laser point cloud.

[0063] Determine if the laser point cloud is within the defogging area of ​​the working map; if so, remove the laser point cloud; otherwise, traverse the next laser point cloud.

[0064] Continue until all laser point clouds have been traversed.

[0065] There are multiple methods for determining whether data collected by a fog-affected sensor is within a cleared area of ​​the working map, as long as it verifies whether the data is within that cleared area. In some embodiments, for each edge of the cleared area in the working map, the rotation direction (clockwise or counterclockwise) of the vector formed by that edge and the data collected by the fog-affected sensor is calculated; the rotation directions of all edges are summed to obtain a winding number; if the winding number is not 0, the data collected by the fog-affected sensor is within the cleared area of ​​the working map; if the winding number is 0, the data is outside the cleared area. In some embodiments, the pnPoly algorithm is used to determine whether the data collected by the fog-affected sensor is within the cleared area of ​​the working map, including the following steps:

[0066] A ray is drawn from the data collected by the sensors affected by fog, and the number of intersections between this ray and the boundary of the cleared area on the working map is counted.

[0067] If the number of intersection points is odd, the data collected by the fog-affected sensor will be within the cleared area of ​​the working map; otherwise, the data collected by the fog-affected sensor will be outside the cleared area of ​​the working map.

[0068] The pnPoly algorithm is a simple and efficient algorithm that can quickly determine whether data collected by a sensor affected by fog is within a cleared area on the working map. Its time complexity is O(n), where n is the number of edges in the cleared area of ​​the working map. It only requires traversing the edges of the cleared area without complex geometric calculations, making the pnPoly algorithm easy to understand and relatively simple to implement. Using the pnPoly algorithm helps reduce the computational complexity of determining whether data collected by a fog-affected sensor is within a cleared area on the working map, reducing computation time and thus shortening the determination time.

[0069] There are multiple methods for determining whether data collected by a sensor affected by fog is within the defogging area of ​​the working map, as long as it can be determined whether the data collected by the sensor affected by fog is within the defogging area of ​​the working map. In some embodiments, for each edge of the defogging area in the working map, the rotation direction (clockwise or counterclockwise) of the vector formed by the edge and the data collected by the sensor to be determined to be affected by fog is calculated; the rotation directions of all edges are summed to obtain a winding number; if the winding number is not 0, the data collected by the sensor affected by fog is within the defogging area of ​​the working map; if the winding number is 0, the data collected by the sensor affected by fog is outside the defogging area of ​​the working map. In some embodiments, the pnPoly algorithm is used to determine whether the data collected by the sensor affected by fog is within the defogging area of ​​the working map, including the following steps:

[0070] A ray is drawn from the data collected by the sensor affected by fog, and the number of intersections between this ray and the boundary of the defogging area on the working map is counted.

[0071] If the number of intersection points is odd, the data collected by the fog-affected sensor will be within the defogging area on the working map; otherwise, the data collected by the fog-affected sensor will be outside the defogging area on the working map.

[0072] The pnPoly algorithm is a simple and efficient algorithm that can quickly determine whether data collected by a sensor affected by fog is within the defogging zone on the working map. Its time complexity is O(n), where n is the number of edges in the defogging zone on the working map. It only requires traversing the edges of the defogging zone on the working map, without performing complex geometric calculations, making the pnPoly algorithm easy to understand and relatively simple to implement. Using the pnPoly algorithm helps reduce the computational complexity of determining whether data collected by a fog-affected sensor is within the defogging zone on the working map, reducing computation time and thus shortening the determination time.

[0073] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method. In this embodiment, the storage medium can be a storage medium disposed in an electronic device, allowing the electronic device to read the contents of the storage medium and achieve the effects of the present invention. Alternatively, the storage medium can be a separate storage medium connected to an electronic device, enabling the electronic device to read the contents of the storage medium and implement the method steps of the present invention.

[0074] The computer program involved in the embodiments can be stored in a computer device readable storage medium, which includes, but is not limited to, disks, magnetic tapes, magnetic cards, floppy disks, flash memory, optical disks, optical cards, read-only memory (ROM), random access memory (RAM), erasable programmable ROM (EPROM), and electrically erasable programmable ROM (EEPROM), etc., and also includes other biological, physical, or chemical structures that can achieve the same or equivalent functions as the storage media listed above, such as DNA, RNA, proteins, and other units with information storage capabilities. In specific embodiments, the storage medium involved can be one of the above-mentioned media types, or a combination of the above-mentioned media types. In different embodiments, the computer program involved in the embodiments can be centrally stored in a single medium, or distributed and stored in multiple media. The memory containing the computer device readable storage medium can be non-volatile memory or random access memory. These computer device readable storage media can be built into the device, or can be connected to the device involved in the embodiments as an external device or part of an external device. In some embodiments, the memory having a computer device readable storage medium is deployed locally; in other embodiments, the memory may be deployed remotely from the processor, for example, as a network-attached memory accessed via RF circuitry or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), or a suitable combination thereof, as long as computer device access to the memory is enabled. Furthermore, the computer program involved in the embodiments may be stored in plaintext / ciphertext form, or it may be designed as training data, integrated and recombined through model training and implicitly stored in the parameter states of a deep neural network or other machine learning model.

[0075] processor:

[0076] The processor described in the embodiments of this application can be implemented by hardware, firmware, software, or a combination thereof. It can be a circuit, one or more of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, or a microprocessor. It also includes other physical, biological, or chemical structures that can implement the same or equivalent functions as the processors listed above, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer program or method involved in the various embodiments of this application, or any combination of the steps mentioned therein.

[0077] It should be noted that although the above embodiments have been described herein, this does not limit the scope of patent protection of the present invention. Therefore, any changes and modifications made to the embodiments described herein based on the innovative concept of the present invention, or equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included within the scope of patent protection of the present invention.

Claims

1. A data processing method for data acquired by sensors in a mobile robot, characterized in that, The mobile robot is equipped with multiple sensors to detect obstacles. These sensors are divided into those affected by fog and those not affected by fog. Fog-prone areas are designated as defogging zones in the mobile robot's work map. Includes the following steps: Data is collected by multiple sensors; Iterate through the data collected by all sensors affected by fog; Determine whether the data collected by the sensor affected by fog is within the defogging area in the working map. If so, discard the data collected by the sensor affected by fog. Otherwise, continue to determine whether the data collected by the next sensor affected by fog is within the defogging area in the working map. In the step of determining whether the data collected by the sensor affected by fog is within the defogging area in the working map, if so, the data collected by the sensor affected by fog is discarded; otherwise, the process continues to determine whether the data collected by the next sensor affected by fog is within the defogging area in the working map. It also includes determining whether the data collected by the sensor affected by fog is within the clearing zone in the working map. The clearing zone in the robot coordinate system defines the sensing range of all sensors not affected by fog in the coordinate system of the mobile robot. If the data collected by the sensor affected by fog is simultaneously within the defogging zone and the clearing zone in the working map, then the data collected by the sensor affected by fog is discarded; otherwise, it continues to determine whether the data collected by the next sensor affected by fog is within the defogging zone in the working map. The process of determining whether the data collected by the sensor affected by fog is within the cleared area of ​​the working map includes the following steps: A ray is drawn from the data collected by the sensors affected by fog, and the number of intersections between this ray and the boundary of the cleared area on the working map is counted. If the number of intersection points is odd, the data collected by the fog-affected sensor will be within the cleared area of ​​the working map; otherwise, the data collected by the fog-affected sensor will be outside the cleared area of ​​the working map.

2. The data processing method for mobile robot sensors according to claim 1, characterized in that, The sensors affected by fog interference include lidar.

3. The data processing method for mobile robot sensors according to claim 1, characterized in that, The sensors that are unaffected by fog include ultrasonic radar.

4. The data processing method for mobile robot sensors according to claim 1, characterized in that, The clearing area in the working map is obtained by transforming the clearing area in the robot coordinate system to the working map coordinate system.

5. The data processing method for mobile robot sensors according to claim 1, characterized in that, The process of determining whether the data collected by the sensor affected by fog is within the defogging area on the working map includes the following steps: A ray is drawn from the data collected by the sensor affected by fog, and the number of intersections between this ray and the boundary of the defogging area on the working map is counted. If the number of intersection points is odd, the data collected by the fog-affected sensor will be within the defogging area on the working map; otherwise, the data collected by the fog-affected sensor will be outside the defogging area on the working map.

6. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the data processing method for mobile robot sensors as described in any one of claims 1 to 5.

7. An electronic device, characterized in that: It includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the data processing method for mobile robot sensor acquisition as described in any one of claims 1 to 5.

Images