Modeling method, device and equipment for robot simulation environment, medium and product

Abstract

The invention provides a modeling method and device for a robot simulation environment, equipment, a medium and a product. The method comprises the steps of obtaining an original image file of a robot operation environment; multiple different types of composition elements in the original image file are extracted, multiple different types of element sets are obtained, each type of element set comprises all the composition elements of the type, and the sizes of the composition elements in all the types of element sets are different; and generating a simulation model file of the robot operation environment according to the plurality of different types of element sets. The modeling efficiency of the robot simulation environment can be improved.

Description

Technical Field

[0001] This application relates to the field of robot simulation, and particularly to a modeling method, device, equipment, medium and product for a robot simulation environment. Background Art

[0002] Modeling of a robot simulation environment not only helps to reduce costs, improve safety, but also can speed up the R & D process and promote technological progress.

[0003] Currently, in the field of robotics, the Gazebo simulator is widely used as a modeling tool. Although Gazebo provides rich functions and flexibility. However, many users still face a large amount of manual configuration work when building a simulation environment, resulting in low modeling efficiency of the robot simulation environment. Summary of the Invention

[0004] Based on the above technical status quo, this application provides a modeling method, device, equipment, medium and product for a robot simulation environment, which can improve the modeling efficiency of the robot simulation environment.

[0005] To achieve the above technical objectives, this application specifically proposes the following technical solutions:

[0006] According to the first aspect of the embodiments of this application, a modeling method for a robot simulation environment is provided, including: obtaining an original image file of a robot operating environment; extracting various different types of constituent elements from the original image file to obtain a plurality of different types of element sets, where each type of element set includes each constituent element of that type, and the sizes of the constituent elements in each type of element set are different; generating a simulation model file of the robot operating environment according to the plurality of different types of element sets.

[0007] In some implementation manners, the extracting various different types of constituent elements from the original image file to obtain a plurality of different types of element sets includes: extracting wall elements from the original image file to obtain a first image file; searching for various different types of constituent elements in the first image file to obtain a plurality of different types of element sets.

[0008] In some implementation manners, searching for various different types of constituent elements in the first image file to obtain a plurality of different types of element sets includes: respectively extracting various different types of connected domain elements from the first image file according to a plurality of different preset conditions to obtain a plurality of different types of element sets; the plurality of different preset conditions are determined according to different connected domain areas.

[0009] In some implementations, the multiple different preset conditions include meeting the area of the first connected component, the area of the second connected component, or the area of the third connected component; wherein, according to the multiple different preset conditions, various different types of connected component elements are extracted from the first image file to obtain multiple different types of element sets, including: searching for the connected component elements in the first image file that meet the area of the first connected component and storing them in the first element set; traversing each pixel point in the first image file vertically, and storing the connected component elements composed of the pixel points that meet the area of the second connected component in the second element set; traversing each pixel point in the first image file horizontally, and storing the connected component elements composed of the pixel points that meet the area of the third connected component in the third element set, where the area of the second connected component and the area of the third connected component are greater than the area of the first connected component.

[0010] In some implementations, the area of the first connected component is 1*1; the area of the second connected component is n*1, where n is a positive integer greater than 1; the area of the third connected component is 1*m, where m is a positive integer greater than 1.

[0011] In some implementations, the method further includes: extracting obstacle elements from the original image file, and converting the coordinate values of the obstacle elements in the image coordinate system to the Gazebo coordinates to obtain the coordinate values of the obstacle elements in the Gazebo coordinates; wherein, according to the various different types of element sets, a simulation model file of the robot operating environment is generated, including: obtaining the simulation model file of the robot operating environment according to the various different types of element sets and the coordinate values of the obstacle elements in the Gazebo coordinate system.

[0012] In some implementations, obtaining the simulation model file of the robot operating environment according to the various different types of element sets and the coordinate values of the obstacle elements in the Gazebo coordinate system includes: generating an initial simulation model file according to the various different types of element sets; inserting the obstacle elements into the initial model file according to the coordinate values of the obstacle elements in the Gazebo coordinate system to obtain the simulation model file of the robot operating environment.

[0013] According to a second aspect of the embodiments of the present application, there is provided a modeling device for a robot simulation environment, including: an acquisition unit configured to acquire an original image file of a robot operating environment; an extraction unit configured to extract various different types of constituent elements from the original image file to obtain a plurality of different types of element sets, where each type of element set includes each constituent element of that type, and the sizes of the constituent elements in each type of element set are different from each other; and a generation unit configured to generate a simulation model file of the robot operating environment according to the plurality of different types of element sets.

[0014] According to a third aspect of the embodiments of the present application, there is provided an electronic device, including a memory and a processor; the memory is connected to the processor and is configured to store a program; the processor is configured to implement the modeling method of the robot simulation environment as described in the first aspect by running the program in the memory.

[0015] According to a fourth aspect of the embodiments of the present application, there is provided a storage medium, on which a computer program is stored, and when the computer program is run by a processor, the modeling method of the robot simulation environment as described in the first aspect is implemented.

[0016] According to a fifth aspect of the embodiments of the present application, there is provided a computer program product, including computer program instructions, and when the computer program instructions are run by a processor, the processor is caused to execute: the modeling method of the robot simulation environment as described in the first aspect.

[0017] A modeling method, device, equipment, medium, and product for a robot simulation environment provided by the embodiments of the present application. The method extracts various different types of constituent elements from an original image file of a robot operating environment to generate a plurality of different types of element sets. Each element set includes constituent elements of a specific type, and these elements are different in size. Subsequently, based on these multi-type element sets, a simulation model file of the robot operating environment is constructed. Since various elements of different sizes are used to describe the robot operating environment and elements of different scales are integrated, the complexity of model description can be simplified, the number of elements can be reduced, and the modeling efficiency of the robot simulation environment can be improved. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0019] Figure 1Schematic diagram of the running effect of the model file generated in the related art in the simulation environment;

[0020] Figure 2 Flowchart of a modeling method for a robot simulation environment provided by an embodiment of the present application;

[0021] Figure 3 Schematic diagram of an original image file provided by an embodiment of the present application;

[0022] Figure 4 Flowchart of an element extraction method provided by an embodiment of the present application;

[0023] Figure 5 Schematic diagram of the running environment after filtering obstacles provided by an embodiment of the present application;

[0024] Figure 6 Schematic diagram of a single block element satisfying the first connected domain area provided by an embodiment of the present application;

[0025] Figure 7 Schematic diagram of a vertical element satisfying the second connected domain area provided by an embodiment of the present application;

[0026] Figure 8 Schematic diagram of a horizontal element satisfying the third connected domain area provided by an embodiment of the present application;

[0027] Figure 9 Comparison effect diagram of different element extraction methods provided by an embodiment of the present application;

[0028] Figure 10 Schematic structural diagram of a modeling device for a robot simulation environment provided by an embodiment of the present application;

[0029] Figure 11 Schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0030] The technical solutions provided by the embodiments of the present application can be exemplarily applied to hardware devices such as processors, electronic devices, and servers (including cloud servers), or packaged into software programs to be run. When the hardware devices execute the processing procedures of the technical solutions of the embodiments of the present application, or when the above software programs are run, the automatic splitting of the target task and the automatic calling of the application program interfaces required for the task can be realized, and the purpose of completing the target task can be achieved. The embodiments of the present application only exemplarily introduce the specific processing procedures of the technical solutions of the present application, and do not limit the specific implementation forms of the technical solutions of the present application. Any technical implementation form that can execute the processing procedures of the technical solutions of the present application can be adopted by the embodiments of the present application.

[0031] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0032] Before introducing the solution of the present application, the related technologies will be introduced first:

[0033] Robot simulation software can build a highly realistic virtual environment, enabling developers to simulate, test, and optimize robots without physical hardware. Such tools provide comprehensive support from proof of concept to actual deployment during the development process of robots.

[0034] As the development process progresses, the role of these simulation tools becomes increasingly important. In the initial stage of research and development, simulation software can be used to verify the feasibility of algorithms and design concepts; after entering the testing stage, the software can simulate and reproduce diverse actual application scenarios to support the evaluation of the performance and reliability of robots; in the control stage, the simulation platform can also interact with actual hardware to achieve remote control and monitoring of robots. In this way, developers can continuously utilize the advantages of simulation software throughout the product life cycle to ensure that the tasks in each stage can be completed efficiently.

[0035] Different simulation platforms have their own characteristics and technical advantages to meet the requirements of different types of robot projects. For example, some platforms focus on providing an easy-to-use graphical interface, while others emphasize high-performance computing capabilities or the realism of physical simulation. For developers seeking efficient modeling and rapid iteration, the ability to automatically generate model files is one of the key factors to improve work efficiency.

[0036] Currently, the Simulink simulation platform can automatically generate model files by combining image processing technology and a script editing language. This involves extracting the key information of the input formula from a picture file and automatically generating the required Simulink model file through the script editing language. However, this method is mainly applicable to the Simulink platform and is not compatible with other platforms (such as Gazebo), thus limiting its universality for cross-platform applications.

[0037] The Gazebo platform is one of the most widely used simulation platforms in the field of robotics. It provides some open source tools in the online community to reduce the difficulty of getting started with robotics simulation. However, the model accuracy of these tools in complex scenarios is often insufficient. High complexity may cause excessive resource consumption when the simulation software is running, or even cause the program to crash. In addition, in simple scenarios, there may be certain errors in the model output, affecting the accuracy and practicality of the simulation.

[0038] Figure 1 The schematic diagram is a diagram showing the running effect of the model file generated in the related technology in the simulation environment. Figure 1 As shown in Figure 1, although the model files automatically generated by these open source tools can roughly restore the expected design, there are certain deviations in the details. Specifically, these errors are reflected in the constructed virtual environment, where the connection between the walls is not perfectly realized, and obvious faults and gaps appear (such as Figure 1 As shown in the marked area in the figure, the final output model deviates from the original design intent. This not only affects the visual fidelity of the simulation, but may also have a negative impact on the accuracy of the simulation results.

[0039] Therefore, although Gazebo is one of the most widely used simulation platforms in the field of robotics, in actual operation, engineers still rely on manual construction of the environment, including calling Gazebo's own models, or using community and self-built plug-in models, and need to manually adjust the model parameters in the interface or retrieve and modify the model parameters based on the model file; finally, according to actual use feedback, continuous adjustments are made until the expected simulation effect is achieved. This method has low modeling efficiency and is difficult to meet the needs of rapid iteration.

[0040] In summary, the efficiency and accuracy of generating robot simulation environment model files in related technologies need to be improved.

[0041] In view of this, the embodiments of the present application are dedicated to providing a modeling method, device, equipment, medium and product for a robot simulation environment, which are described in detail one by one in the following embodiments.

[0042] Exemplary method

[0043] Figure 2 The following is a flow chart of a method for modeling a robot simulation environment provided in an embodiment of the present application. Figure 2 As shown, the modeling method of the robot simulation environment provided in this embodiment includes steps S101-S103:

[0044] S101, obtaining an original image file of the robot operating environment.

[0045] The robot in this embodiment can be a floor cleaning robot, a warehouse logistics robot, an agricultural robot, or a detection robot for exploring dangerous environments, etc.

[0046] The robot operating environment refers to the physical environment where the robot works, that is, the actual physical space where the robot is located when performing tasks. This physical space has a spatial layout, such as the size, shape, and obstacle distribution of the robot's working area. In a factory, the robot operating environment may be a fixed workstation, while in a home environment, the robot operating environment may be the layout of different rooms.

[0047] The original image file is an image of the physical environment when the robot is working, and can be obtained by at least two of the following methods:

[0048] In some implementation manners, the original image file can be obtained by manual drawing by the user. For example, for a floor cleaning robot, the user can manually depict the room layout, furniture positions, and other important features to provide an accurate working environment model for the robot.

[0049] In some implementation manners, the original image file can also be obtained by using the map scanning function of the robot itself. For example, some floor cleaning robots or other intelligent mobile devices equipped with advanced sensors and algorithms can build an environmental map during autonomous movement. These devices use a variety of sensing technologies such as lidar (LIDAR), cameras, and ultrasonic sensors to achieve high-precision scanning of the surrounding space, process the collected data, generate a detailed environmental map, and save it as the original image file. The original image file will be further introduced below with reference to the accompanying drawings:

[0050] Figure 3 It is a schematic diagram of an original image file provided by an embodiment of the present application. As Figure 3 shown, this original image file is a top view of the operating environment of a home floor cleaning robot. Through the top view, the overall layout of the room can be observed, including the positions of the walls, the distribution of doors and windows, and various other elements such as furniture placed inside the room.

[0051] These elements are represented by different icons or graphics in the figure. For example, a rectangle may represent a table or a bed, while smaller circles or irregular shapes may be chairs, lamps, or other decorative items. Through this kind of schematic description, the spatial configuration inside the room can be intuitively understood, providing basic information for the subsequent modeling of the robot simulation environment.

[0052] After obtaining the original image file, the original image file can be used for the modeling of the robot simulation environment, specifically as follows:

[0053] S102. Extract various different types of constituent elements from the original image file to obtain multiple element sets of different types. Each element set of a type includes all constituent elements of that type, and the sizes of the constituent elements in each element set of different types are different.

[0054] In step S102, various different types of constituent elements are extracted from the original image file through an element extraction method (or called an element cutting method) to obtain multiple element sets of different types. Among them, there are multiple element extraction methods in step S102, which can be specifically as follows:

[0055] Figure 4 It is a flowchart of an element extraction method provided by an embodiment of the present application. As Figure 4 shown, step S102 includes step S401 and step S402:

[0056] S401. Extract wall elements from the original image file to obtain a first image file.

[0057] In the working environment of the robot, the wall is the most common and crucial structural element. It not only defines the physical boundary of the space but also constitutes the basic framework of the entire environment. All other objects and furniture are arranged and placed within the space defined by the wall. Therefore, in order to accurately simulate and understand the environment where the robot will operate, it is first necessary to accurately extract the wall elements from the original image file.

[0058] To adapt to the Gazebo platform and ensure the accuracy of modeling. In step S401, first, the original image file needs to be converted from the image coordinate system to the Gazebo coordinate system, and then element extraction is performed in the Gazebo coordinate system. There are multiple implementation methods for its element extraction, which can be specifically as follows:

[0059] In some implementation methods, in step S401, the outlines and lines representing the wall can be identified and separated through an image processing algorithm, thereby generating a first image file containing wall elements.

[0060] The original image file is usually a color image. In a color image, the wall usually has a relatively uniform color and texture, while furniture and other items may show more details and variations. By performing grayscale processing on the original image file to reduce the complexity of color information and convert it into a grayscale image, the visual difference between the wall and other elements can be enhanced, facilitating subsequent feature extraction.

[0061] Feature extraction can adopt an edge detection algorithm (such as the Canny or Sobel operator) to identify the significant boundaries in the image, that is, the lines corresponding to the wall edges. Thus, the regions with large intensity changes in the image, that is, the boundaries between objects, are highlighted.

[0062] In some other implementation manners, machine learning or deep learning models, such as convolutional neural networks (CNNs), can also be utilized to distinguish walls from other obstacles according to color, texture, and context information, and perform semantic segmentation on the image. This method can not only identify wall elements but also label other key elements in the image, achieving precise separation.

[0063] Figure 5 A schematic diagram of an operating environment after filtering obstacles provided by an embodiment of the present application. As Figure 5 shown, after the above processing, only the wall contours and lines are retained in the image, and the overall layout of the room can be clearly defined. This refined image not only enables the spatial structure to provide a basis for subsequent element extraction steps.

[0064] Continue to refer to Figure 4 , after step S401, the element extraction process may further include step S402.

[0065] S402. Search for various different types of wall component elements in the first image file to obtain various different types of wall element sets.

[0066] Specifically, step S402 includes the following steps a1:

[0067] Step a1. Extract various different types of connected component elements from the first image file according to multiple different preset conditions to obtain multiple different types of wall element sets; the multiple different preset conditions are determined according to different connected component areas.

[0068] Among them, a connected component refers to a group of pixel points in an image that have the same attributes (such as color, texture, or brightness) and are connected to each other. In this embodiment, the area size of the connected component is used as the main classification basis to identify and classify the connected component elements in the first image file.

[0069] In step a1, when extracting connected component elements that meet specific area requirements according to multiple different preset conditions, for each preset condition, a corresponding set of wall elements will be obtained.

[0070] The number of preset conditions can be set to 2 - 5, such as 3. Taking 3 preset conditions as an example, the specific implementation process of step a1 will be introduced in detail below:

[0071] In some application scenarios, when a robot is deployed in a specific operating environment, the layout diagram of the environment is an important basis for its navigation and task planning. Among them, walls, as the main elements constituting the spatial structure, occupy a large proportion in the floor plan.

[0072] In actual architectural environments, walls usually present regular geometric shapes, mainly in the form of long strips extending horizontally or vertically. However, in addition to these basic linear wall structures, there may be some special areas in actual architectural environments that may contain protrusions or depressions of non-standard shapes, such as niches, columns or other decorative structures. Although these features are not the main components of the wall, they are equally important for the robot's perception and action. Therefore, when creating an environmental model, different preset conditions can be set for the three different types of wall elements mentioned above, namely standard horizontal and vertical long strip walls and special protruding structures, for accurate extraction.

[0073] In addition, in the Gazebo simulation environment, the basic building block includes a cuboid-shaped entity, which can be intuitively understood as a box in a three-dimensional space. Users can flexibly adjust its size parameters, including length, width, and height, according to specific simulation needs to adapt to different types of simulation scene requirements.

[0074] The three different types of wall elements mentioned in the foregoing, i.e., standard horizontal and vertical long strip walls and protruding parts of special shapes, all meet the definition of the basic element. Therefore, the three different types of elements can be represented by customized cuboids, and can be combined and modified as needed to accurately reflect the complex structure in the actual environment. Therefore, a plurality of different types of wall component elements can be searched in the first image file according to a plurality of different preset conditions, thereby ensuring that all extracted elements can be integrated into the simulation model to realize the modeling of the robot simulation environment.

[0075] Specifically, the plurality of different preset conditions may include a first preset condition, a second preset condition, and a third preset condition; wherein the first preset condition includes satisfying a first connected domain area; the second preset condition includes satisfying a second connected domain area; and the third preset condition includes satisfying a third connected domain area. Furthermore, the second connected domain area and the third connected domain area are greater than the first connected domain area.

[0076] On this basis, step a1 classifies the connected domain elements according to the first preset condition, the second preset condition and the third preset condition, and specifically includes the following steps a11, a12 and a13:

[0077] Step a11, searching the first image file for connected domain elements that satisfy the first connected domain area, and storing them in the first wall element set.

[0078] Specifically, all connected domain elements with a connected domain area equal to the area of the first connected domain can be searched out through a depth-first search algorithm and stored in the first wall element set. The area of the first connected domain can be defined as the smallest unit, and the elements of the smallest unit can be obtained through step a11, such as a single pixel point.

[0079] Step a12: Vertically traverse each pixel point in the first image file, and store the connected domain elements composed of pixel points that meet the second connected domain area in the second wall element set.

[0080] Among them, by performing a vertical traversal on the first image file and scanning each pixel point column by column, connected domains that meet the second connected domain area standard can be found and added to the second wall element set. Vertical traversal helps to discover wall parts that extend in the vertical direction and meet the second connected domain area standard. Through vertical traversal, wall structures larger than a single pixel, such as corner walls and door frame edges, can be obtained.

[0081] Step a13: Horizontally traverse each pixel point in the first image file, and store the connected domain elements composed of pixel points that meet the third connected domain area in the third wall element set, where the second connected domain area and the third connected domain area are greater than the area of the first connected domain.

[0082] In step a13, a search strategy similar to that in step a12 is adopted, but step a13 is a horizontal traversal, that is, scanning each pixel point in the image row by row to identify connected domain elements that meet the third connected domain area requirements and classifying them into the third wall element set. These connected domain elements meet the requirements of the third connected domain area, that is, their area is greater than the area of the first connected domain. Through step a13, some connected domain elements with a large width and a small height can be identified, such as long and thin wall parts.

[0083] Through steps a11 to a13, not only can connected domains in different directions be effectively detected, but various types of wall elements can also be accurately classified and collected. Thus, it provides data support for subsequent 3D reconstruction and ensures the accuracy and integrity of model reconstruction.

[0084] It should be noted that the execution order of steps a11 to a13 is not fixed and can be flexibly adjusted. These three steps can be performed in any combined order according to the actual situation without affecting the final result. This means that in actual operation, steps a11, a12, and a13 can be executed in any order.

[0085] In the first image file, different wall parts may correspond to connected regions of different areas. Specifically, large connected regions usually represent entire walls or large partitions; medium-sized connected regions may represent smaller wall parts, door frames, or other similar structures; small connected regions often correspond to slender wall parts or decorative wall elements. Based on this, corresponding criteria for the area of connected regions can be set for different types of wall features to more accurately identify and classify wall elements. Among them, the areas of connected regions under different preset conditions can be set as follows:

[0086] Specifically, the area of the second connected region can be set to a size of n*1, where n is a positive integer greater than 1. The area of the third connected region can be set to a size of 1*m, where m is a positive integer greater than 1.

[0087] That is to say, each element in the first set of wall elements is a single pixel of size 1*1. Each element in the second set of wall elements has a size of n*1, corresponding to 1 connected region in each row of the image. Each element in the third set of wall elements has a size of 1*m, corresponding to 1 connected region in each column of the image. The following introduces the sizes of elements in different types of element sets with reference to the accompanying drawings:

[0088] Figure 6 Schematic diagram of a single element satisfying the area of the first connected region provided by an embodiment of the present application. As Figure 6 shown, the area of the first connected region can be set to a single pixel point of size 1*1; the area of the first connected region helps to capture details at the smallest unit level.

[0089] Figure 7 Schematic diagram of a vertical element satisfying the area of the second connected region provided by an embodiment of the present application. As Figure 7 shown, each wall element corresponding to the area of the second connected region is composed of n rows but only one column of pixels, which is suitable for representing slender wall features extending in the vertical direction, such as wall corners or vertical borders.

[0090] Figure 8 Schematic diagram of a horizontal element satisfying the area of the third connected region provided by an embodiment of the present application. As Figure 8 shown, each wall element corresponding to the area of the third connected region is composed of m columns but only one row of pixels, which is suitable for identifying slender wall features stretched in the horizontal direction, such as window sill lines or horizontal decorative bands.

[0091] This embodiment effectively detects and classifies wall features of different shapes and sizes according to the areas of connected regions under different preset conditions, thereby reducing the number of individual elements and improving the modeling efficiency.

[0092] Continue to refer toFigure 2 , the method for modeling the robot simulation environment in this embodiment further includes the following step S103.

[0093] S103. Generate a simulation model file for the robot operating environment according to multiple different types of element sets.

[0094] Among them, for multiple different types of element sets, the xml library of python can be used to process multiple different types of element sets and edit them into corresponding model elements. In this process, various wall elements and other structural information are first compiled into an SDF-compatible format to ensure that each element can be accurately represented in the three-dimensional simulation space, thereby generating a basic SDF (Simulation Description Format) file, that is, an initial simulation model file. This initial simulation model file defines the basic structure and layout of the environment.

[0095] In step S102, by identifying and extracting multiple different types of element sets, the basic framework of the robot operating environment can be constructed. After these diverse elements are combined, the wall structure part in the robot operating environment can be accurately reconstructed. In addition, the robot operating environment may also contain various obstacle elements other than walls, such as furniture like coffee tables, desks, and chairs, which will all affect the navigation and operation of the robot. Therefore, the method in this embodiment further includes:

[0096] Convert the original image file from the image coordinate system to the Gazebo coordinate system, and determine the coordinate values of the obstacle elements in the Gazebo coordinates according to the converted original image file.

[0097] On this basis, step S103 includes: obtaining a simulation model file for the robot operating environment according to multiple different types of element sets and the coordinate values of the obstacle elements in the Gazebo coordinate system.

[0098] In some embodiments, an initial simulation model file can be generated according to multiple different types of element sets; according to the coordinate values of the obstacle elements in the Gazebo coordinate system, the obstacle elements are inserted into the initial model file to obtain a simulation model file for the robot operating environment.

[0099] Specifically, it is necessary to use the specific coordinate values of the obstacle elements determined in the previous conversion process as position indexes to insert the non-model elements to be inserted into the initial simulation model file. Thereby ensuring that all non-wall obstacles can be accurately represented in the simulation environment, so as to completely reflect the actual operating environment of the robot.

[0100] These non-model elements may include other obstacles in the floor plan besides walls, such as coffee tables, desks, chairs, beds, etc. These obstacles are crucial for enriching the simulation environment. After completing the above series of edits, run the verification by opening Gazebo and loading the generated SDF file to confirm the validity of the file and the accuracy of the simulation environment, thus completing the entire process of robot simulation modeling.

[0101] Through the above steps, a comprehensive simulation model file of the robot operating environment can be obtained. This file not only contains the basic infrastructure of the environment but also includes all important obstacle elements and their accurate position information. Thus, the simulation model is closer to the actual situation, providing a more realistic test environment for the robot and helping to more effectively evaluate and optimize the performance of the robot.

[0102] In the foregoing embodiments, an element extraction method is introduced, that is, a specific implementation process of generating a robot simulation model file by identifying and extracting various different types of element sets in the original image file. However, according to the foregoing description, since the connected domain elements in the horizontal and vertical directions are actually composed of basic units of size 1*1 (i.e., the first connected domain elements). Therefore, another element extraction method can also be provided, that is, by traversing each pixel point in the first image file and adding the pixel points representing the wall features to the fourth wall element set one by one. And generating a modeling file of the robot simulation environment based on the fourth wall element set containing all the pixel points of the wall features.

[0103] Figure 9 Schematic diagrams of different element extraction methods provided by embodiments of this application. As Figure 9 shown, the left image presents a method of extraction that completely relies on single elements. Among them, the numerical numbers 1 to 20 respectively represent each independent wall element in the fourth wall element set, and each element corresponds to a basic unit of size 1x1.

[0104] The right image shows the results obtained by various different types of element extraction methods. Among them, the connected domains composed of rectangular frames corresponding to the same numerical numbers identify a specific element set. For example, the connected domains identified by the numbers 1, 2, and 3 in the figure belong to the second wall element set; the number 4 identifies the first wall element set; and the connected domains identified by the numbers 5 and 6 constitute the third wall element set.

[0105] Compared with the method of using only single - block elements for robot simulation modeling, the element extraction method by integrating multiple types of element sets can reduce the number of elements required, thereby improving the efficiency of simulation environment modeling. At the same time, it can also enhance the quality and authenticity of the model. The resulting simulation environment is closer to the actual situation, providing a more reliable and realistic platform for robot behavior testing.

[0106] By verifying the above two different element extraction methods in multiple scenarios, the following comparisons can be made: Among them, in Scenario 1, the number of elements in the modeling method using only single - block elements is 1837, while the number of elements in the multiple - type element extraction method is 194; in Scenario 2, the number of elements in the modeling method using only single - block elements is 3560, while the number of elements in the multiple - type element extraction method is 367; in Scenario 4, the number of elements in the modeling method using only single - block elements is 4586, while the number of elements in the multiple - type element extraction method is 473.

[0107] Through the comparison of multiple groups of experimental data, it can be found that the multiple - type element extraction method can effectively reduce the number of finally generated models and fully match the environmental information of the input original image file.

[0108] In the above - mentioned embodiments, it is introduced that in step S401, the original image file is converted from the image coordinate system to the Gazebo coordinate system, and then the element extraction is carried out in the Gazebo coordinate system. However, in some embodiments, the element extraction can also be directly carried out in the image coordinates corresponding to the original image file. After the extraction is completed, the extracted element set is then converted to the Gazebo coordinate system. Finally, in step S103, according to multiple different - type element sets in the Gazebo coordinate system, a simulation model file of the robot running environment is generated.

[0109] Regarding the positioning of obstacle elements, it can be achieved during the process of converting the original image file from the image coordinate system to the Gazebo coordinate system in step S401. During this process, not only the coordinate conversion is completed, but also accurate position information is provided for the obstacle elements, thereby ensuring that these obstacle elements can be correctly mapped and simulated when creating the simulation model of the robot running environment.

[0110] In the embodiments of the present application, by analyzing the original image file of the robot's operating environment, various different types of constituent elements are identified and extracted, and then multiple element sets of specific types are constructed. Each set contains a series of constituent elements with specific characteristics, and these elements differ in size and shape. Next, based on these multi-type and multi-scale element sets, a simulation model file of the robot's operating environment can be efficiently constructed. By integrating elements with different sizes and shapes to describe the robot's operating environment, not only can the complexity of the model be simplified, but also the total number of required elements can be significantly reduced. This can not only improve the efficiency of the modeling process, but also make the generated simulation environment more accurate and efficient.

[0111] In summary, the modeling method for a robot simulation environment provided by the embodiments of the present application can greatly improve the modeling efficiency while ensuring the simulation accuracy.

[0112] Exemplary device

[0113] Corresponding to the above-mentioned modeling method for a robot simulation environment, the embodiments of the present application also provide a modeling device for a robot simulation environment. Figure 10 It is a schematic structural diagram of a modeling device for a robot simulation environment provided by the embodiments of the present application. As Figure 10 shown, the modeling device for a robot simulation environment provided by the embodiments of the present application includes: an acquisition unit 1001, an extraction unit 1002, and a generation unit 1003; wherein, the acquisition unit 1001 is used to acquire the original image file of the robot's operating environment; the extraction unit 1002 is used to extract various different types of constituent elements from the original image file to obtain multiple different types of element sets, and each type of element set includes each constituent element of that type, and the sizes of the constituent elements in each type of element set are different; the generation unit 1003 is used to generate the simulation model file of the robot's operating environment according to the multiple different types of element sets.

[0114] In some embodiments, when the extraction unit 1002 extracts various different types of constituent elements from the original image file to obtain multiple different types of element sets, it includes: extracting wall elements from the original image file to obtain a first image file; searching for various different types of wall constituent elements in the first image file to obtain various different types of wall element sets.

[0115] In some embodiments, the extraction unit 1002 searches for various different types of wall composition elements in the first image file to obtain various different types of wall element sets, including: respectively extracting various different types of connected domain elements from the first image file according to multiple different preset conditions to obtain multiple different types of wall element sets; the multiple different preset conditions are determined according to different connected domain areas.

[0116] In some embodiments, the multiple different preset conditions include meeting a first connected domain area, a second connected domain area, and a third connected domain area; wherein, the extraction unit 1002 extracts various different types of connected domain elements from the first image file according to the multiple different preset conditions to obtain various different types of wall element sets, including: searching for connected domain elements in the first image file that meet the first connected domain area and storing them in the first wall element set; longitudinally traversing each pixel point in the first image file and storing the connected domain elements composed of pixel points that meet the second connected domain area in the second wall element set; transversely traversing each pixel point in the first image file and storing the connected domain elements composed of pixel points that meet the third connected domain area in the third wall element set, and the second connected domain area and the third connected domain area are greater than the first connected domain area.

[0117] In some embodiments, the first connected domain area is 1*1; the second connected domain area is n*1, where n is a positive integer greater than 1; the third connected domain area is 1*m, where m is a positive integer greater than 1.

[0118] In some embodiments, the generation unit 1003 is further configured to perform the following steps: convert the original image file from the image coordinate system to the Gazebo coordinate system, and determine the coordinate values of the obstacle elements in the Gazebo coordinates according to the converted original image file; obtain the simulation model file of the robot operating environment according to the various different types of element sets and the coordinate values of the obstacle elements in the Gazebo coordinate system.

[0119] In some embodiments, the generation unit 1003 obtains the simulation model file of the robot operating environment according to the various different types of element sets and the coordinate values of the obstacle elements in the Gazebo coordinate system, including: generating an initial simulation model file according to the various different types of element sets; inserting the obstacle elements into the initial model file according to the coordinate values of the obstacle elements in the Gazebo coordinate system to obtain the simulation model file of the robot operating environment.

[0120] The modeling device for the robot simulation environment provided in this embodiment belongs to the same inventive concept as the modeling method for the robot simulation environment provided in the above embodiments of the present application. It can execute the modeling method for the robot simulation environment provided in any of the above embodiments of the present application and has the corresponding functional modules and beneficial effects for executing the modeling method for the robot simulation environment. For technical details not described in detail in this embodiment, reference can be made to the specific processing content of the modeling method for the robot simulation environment provided in the above embodiments of the present application, which will not be elaborated here.

[0121] The functions implemented by the above acquisition unit 1001, extraction unit 1002, and generation unit 1003 can be respectively implemented by the same or different processors, which is not limited in the embodiments of the present application.

[0122] It should be understood that the units in the above device can be implemented in the form of a processor invoking software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor invokes the instructions stored in the memory to implement any of the above methods or the functions of each unit of the device. The processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory inside the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of a hardware circuit. By designing the hardware circuit, the functions of some or all of the units can be realized. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and by designing the logical relationship of the components in the circuit, the functions of some or all of the above units are realized. Again, in another implementation, the hardware circuit can be implemented by a PLD. Taking FPGA as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured through a configuration file to realize the functions of some or all of the above units. All the units of the above device can be all implemented in the form of a processor invoking software, or all implemented in the form of a hardware circuit, or some implemented in the form of a processor invoking software, and the remaining part implemented in the form of a hardware circuit.

[0123] In the embodiments of the present application, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and running capabilities, such as a CPU, microprocessor, GPU, or DSP, etc.; in another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or can be reconfigured. For example, the processor is a hardware circuit implemented by an ASIC or PLD, such as an FPGA, etc. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as a type of ASIC, such as an NPU, TPU, DPU, etc.

[0124] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0125] In addition, each unit in the above device can be integrated in whole or in part, or can be independently implemented. In one implementation, these units are integrated together and implemented in the form of an SOC. The SOC can include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The types of the at least one processor can be different, such as including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0126] Exemplary electronic device

[0127] Embodiments of the present application propose an electronic device. Refer to Figure 11 As shown, the electronic device includes:

[0128] A memory 200 and a processor 210;

[0129] Wherein, the memory 200 is connected to the processor 210 and is used for storing programs;

[0130] The processor 210 is used for implementing the modeling method of the robot simulation environment disclosed in any of the above embodiments by running the program stored in the memory 200.

[0131] Specifically, the above electronic device may further include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0132] The processor 210, the memory 200, the communication interface 220, the input device 230, and the output device 240 are interconnected with each other via a bus. Among them:

[0133] The bus may include a path for transmitting information between various components of the computer system.

[0134] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or may be an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention solution. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0135] The processor 210 may include a main processor, and may also include a baseband chip, a modem, etc.

[0136] The memory 200 stores the program for implementing the technical solution of the present invention, and may also store an operating system and other critical services. Specifically, the program may include program code, and the program code includes computer operation instructions. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk memory, a flash memory, etc.

[0137] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor, etc.

[0138] The output device 240 may include a device for allowing information to be output to a user, such as a display screen, a printer, a speaker, etc.

[0139] The communication interface 220 may include a device of any transceiver type for communicating with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0140] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the steps of any one of the robot simulation environment modeling methods provided in the above embodiments of the present application.

[0141] An embodiment of the present application also provides a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the modeling method of the robot simulation environment introduced in any of the above embodiments. For the specific processing process and its beneficial effects, reference may be made to the embodiments of the modeling method of the robot simulation environment described above.

[0142] Exemplary computer program product and storage medium

[0143] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions. When the computer program instructions are run by a processor, the processor is caused to execute the steps in the modeling method of the robot simulation environment according to various embodiments of the present application described in any of the above embodiments of this specification.

[0144] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0145] In addition, an embodiment of the present application may also be a storage medium, on which a computer program is stored. The computer program is executed by a processor to perform the steps in the modeling method of the robot simulation environment according to various embodiments of the present application described in any of the above embodiments of this specification. Specifically, the following steps may be implemented:

[0146] Step S101: Obtain an original image file of the robot operating environment.

[0147] Step S102: Extract various different types of constituent elements from the original image file to obtain a plurality of different types of element sets. Each type of element set includes each constituent element of that type, and the sizes of the constituent elements in each type of element set are different;

[0148] Step S103: Generate a simulation model file of the robot operating environment according to the plurality of different types of element sets.

[0149] For the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0150] It should be noted that the embodiments in this specification are all described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0151] The steps in the methods of the embodiments of this application can be adjusted, combined, and deleted according to actual needs. The technical features recorded in each embodiment can be replaced or combined.

[0152] The modules and sub-modules in the devices and terminals in the embodiments of this application can be combined, divided, and deleted according to actual needs.

[0153] In the several embodiments provided by this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are only illustrative. For example, the division of modules or sub-modules is only a logical function division. In actual implementation, there can be other division methods. For example, multiple sub-modules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of devices or modules can be in electrical, mechanical, or other forms.

[0154] The modules or sub-modules described as separate components can be or may not be physically separated. The components as modules or sub-modules can be or may not be physical modules or sub-modules, that is, they can be located in one place, or can be distributed to multiple network modules or sub-modules. Some or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, in each embodiment of the present application, each functional module or sub-module can be integrated into one processing module, or each module or sub-module can exist physically alone, or two or more modules or sub-modules can be integrated into one module. The above-mentioned integrated modules or sub-modules can be implemented in the form of hardware, or in the form of software functional modules or sub-modules.

[0156] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0157] The steps of the method or algorithm described in combination with the embodiments disclosed in this article can be directly implemented by hardware, software units executed by a processor, or a combination of the two. The software units can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0158] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0159] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A modeling method for a robot simulation environment, characterized in that: include: Get the original image file of the robot's operating environment; Extracting multiple different types of component elements from the original image file to obtain multiple different types of element sets, each type of element set including each component element of the type, and the sizes of the component elements in each type of element set are different; A simulation model file of the robot operating environment is generated according to the plurality of different types of element sets.

2. The method according to claim 1, characterized in that: The extracting of multiple different types of constituent elements from the original image file to obtain multiple different types of element sets includes: Extract wall elements from the original image file to obtain a first image file; A plurality of different types of wall component elements are searched in the first image file to obtain a plurality of different types of wall element sets.

3. The method according to claim 2, characterized in that Searching for multiple different types of wall elements in the first image file to obtain multiple different types of wall element sets, including: According to a plurality of different preset conditions, a plurality of different types of connected domain elements are extracted from the first image file to obtain a plurality of different types of wall element sets; The multiple different preset conditions are determined according to different connected domain areas.

4. The method according to claim 3, characterized in that The plurality of different preset conditions include satisfying a first connected domain area, a second connected domain area, and a third connected domain area; The method of extracting multiple different types of connected domain elements from the first image file according to multiple different preset conditions to obtain multiple different types of wall element sets includes: Searching for connected domain elements satisfying the first connected domain area in the first image file, and storing them in the first wall element set; Traversing each pixel point in the first image file vertically, and storing connected domain elements composed of pixel points satisfying the second connected domain area into a second wall element set; Each pixel point in the first image file is traversed horizontally, and connected domain elements composed of pixel points satisfying the third connected domain area are stored in a third wall element set, and the second connected domain area and the third connected domain area are greater than the first connected domain area.

5. The method according to claim 4, characterized in that The area of ​​the first connected domain is 1*1; The area of ​​the second connected domain is n*1, where n is a positive integer greater than 1; The area of ​​the third connected domain is 1*m, where m is a positive integer greater than 1.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Convert the original image file from the image coordinate system to the Gazebo coordinate system, and determine the coordinate value of the obstacle element in the Gazebo coordinate system according to the converted original image file; The step of generating a simulation model file of the robot operating environment according to the plurality of different types of element sets includes: A simulation model file of the robot operating environment is obtained according to the plurality of different types of element sets and the coordinate values ​​of the obstacle elements in the Gazebo coordinate system.

7. The method according to claim 6, characterized in that The obtaining of the simulation model file of the robot operating environment according to the plurality of different types of element sets and the coordinate values ​​of the obstacle elements in the Gazebo coordinate system includes: Generate an initial simulation model file according to the plurality of different types of element sets; According to the coordinate value of the obstacle element in the Gazebo coordinate system, the obstacle element is inserted into the initial model file to obtain a simulation model file of the robot operating environment.

8. A modeling device for a robot simulation environment, characterized in that: include: An acquisition unit, used for acquiring an original image file of the robot's operating environment; An extraction unit is used to extract multiple different types of component elements from the original image file to obtain multiple different types of element sets, each type of element set includes each component element of the type, and the sizes of the components in each type of element set are different; A generating unit is used to generate a simulation model file of the robot operating environment according to the plurality of different types of element sets.

9. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 7 by running the program in the memory.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, cause the processor to implement the method as claimed in any one of claims 1 to 7.

Images