Interaction control method of map model

Abstract

The invention discloses an interaction control method of a map model. The interaction control method comprises the steps of building a three-dimensional space based on a graphic development kit; loading and rendering each object model in a three-dimensional space through a resource loading function included in the graphic development kit, and forming a map model by each object model; setting a camera model in a three-dimensional space based on a camera control tool class included in the graphic development kit; then establishing a mapping relationship between a corresponding touch event included in the model rendering tool class and a touch event processing function included in the camera control tool class; and calling a touch event processing function included in the camera control tool class on the basis of the mapping relationship every time in response to an interactive operation on the terminal, and then adjusting the pose of the camera model or the visual angle of the camera model through the touch event processing function, thereby observing the map model from different visual angles through the camera model.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an interactive control method for a map model. Background Art

[0002] With the rapid development of smart home technology, a floor sweeping robot has become an indispensable smart home device in modern families. As an important part of the floor sweeping robot, the three-dimensional map technology provides a great enhancement for the path planning, cleaning efficiency, and obstacle avoidance ability of the floor sweeping robot through precise spatial perception and intelligent algorithms. The three-dimensional map technology can recognize the home space in a multi-dimensional manner, including information such as height, width, and depth. This multi-dimensional recognition enables the floor sweeping robot to more accurately locate its own position, intelligently plan the cleaning path, and adapt to changes in the home environment.

[0003] When developing a terminal application that can effectively display the movement trajectory of the floor sweeping robot and the characteristics of its environment across platforms, for example, when creating a high-performance 3D graphics application on an Android device, the map model displayed on the screen of the Android device is generally collected and constructed from a single camera perspective for human-computer interaction. Even during human-computer interaction, only the display position of the map model is artificially changed, and the map model actually collected is not switched to be displayed from other perspectives of the camera. Therefore, the map model is not displayed in all directions from multiple camera perspectives of the floor sweeping robot. Summary of the Invention

[0004] The present application discloses an interactive control method for a map model, and the specific technical solution is as follows: An interactive control method for a map model, which is executed by a terminal configured with an Android operating system; the interactive control method includes: building a three-dimensional space based on a graphics development kit; loading and rendering each object model in the three-dimensional space through a resource loading function included in the graphics development kit, and then combining each object model into a map model; setting a camera model in the three-dimensional space based on a camera control tool class included in the graphics development kit; then controlling a corresponding touch event included in a model rendering tool class to establish a mapping relationship with a touch event processing function included in the camera control tool class; whenever an interactive operation on the terminal is responded to, calling the touch event processing function included in the camera control tool class based on the mapping relationship, and then adjusting the pose or perspective of the camera model through the touch event processing function, so that when the map model is fixedly set in the three-dimensional space, the map model can be observed from different perspectives through the camera model.

[0005] Compared with the prior art, in response to an interaction operation, the present application controls the changes in the perspective and pose of the camera model by calling relevant processing functions, and is adapted to display the map model from multiple camera perspectives when the map model is fixedly set, and can equivalently achieve the rotation movement, movement, reduction, enlargement, etc. of the map model from the perspective of the user observing the screen, which is not equivalent to transforming the map model observed from a single camera perspective (including rotation transformation, translation transformation, and scaling transformation), so as to optimize the display effect of the map model in the three-dimensional space and enhance the multi-dimensional space cognition.

[0006] Further, the method of adjusting the pose of the camera model or the perspective of the camera model through the touch event processing function includes at least one of the following: controlling the camera model to perform an arc movement based on the touch event processing function included in the camera control tool class, so that the areas of the map model at different perspectives are displayed on the screen of the terminal; controlling the perspective translation of the camera model based on the touch event processing function included in the camera control tool class, so that the map model is displayed as being dragged on the screen of the terminal; enlarging the perspective of the camera model based on the touch event processing function included in the camera control tool class, so that the image of the map model displayed on the screen of the terminal is reduced; reducing the perspective of the camera model based on the touch event processing function included in the camera control tool class, so that the image of the map model displayed on the screen of the terminal is enlarged; wherein, the map model is fixedly immovable in the three-dimensional space.

[0007] In summary, the user holds the terminal and keeps the relative position between the terminal and the object model in the environment where the robot is located unchanged. When the user performs an interaction operation on the terminal to cause a pose transformation or a perspective transformation (including perspective translation and perspective scaling) of the camera model, but does not directly perform an interaction operation on the map model displayed on the screen of the terminal (such as clicking or swiping on the map model); as long as a touch occurs during the interaction operation on the screen of the terminal, a touch event is generated, causing a change in the pose of the camera model or the perspective of the camera model, which can be uniformly defined as controlling the perspective of the camera model; then during the process of responding to the interaction operation, different perspective areas of the map model can be displayed on the screen of the terminal, or the map model can be displayed as being dragged, or the map model can be displayed as being scaled, realizing a comprehensive and controllable display of various visual effects of the map model.

[0008] Further, in response to an interaction operation on the screen of the terminal, the touch event acquisition function included in the model rendering tool class is used to acquire the touch event corresponding to the interaction operation on the screen of the terminal, and the touch event corresponding to the interaction operation on the screen of the terminal is set as the touch event included in the model rendering tool class; the event type determination function included in the model rendering tool class is used to determine the event type of the touch event included in the model rendering tool class; in combination with the mapping relationship and the event type of the touch event, the touch event processing function included in the camera control tool class is called, and then the pose or viewing angle of the camera model is configured through the touch event processing function included in the camera control tool class, so that the camera model makes an arc motion or the viewing angle of the camera model is adjusted, realizing the update of the displayed map model without calling the touch event processing function included in the model rendering tool class.

[0009] Further, the camera model is a camera tool in the renderer included in the camera control tool class, forming a camera tool with an adjustable viewing angle in the three-dimensional space; after the three-dimensional space is built, the view initialization function in the rendering method subclass is called through the renderer included in the camera control tool class, and the initial position of the camera model is set directly above the map model along the vertical axis of the space coordinate system of the three-dimensional space. It supports moving or changing the viewing angle in response to the interaction operation.

[0010] Further, a background light is configured for the three-dimensional space, and the illumination brightness of the point light source is set to be greater than the brightness of the background light of the three-dimensional space, where the brightness at the projection position of the point light source is configured to decay as the distance from the point light source increases; the origin of the space coordinate system of the three-dimensional space is set as the center of the map model; a point light source is configured directly above the map model along the vertical axis of the space coordinate system, so that when the pose or viewing angle of the camera model changes, the point light source emits light towards the map model at a fixed angle, and the illumination brightness of the point light source does not change due to the change of the interaction operation.

[0011] Further, the object model is a ground model, or a wall model, or a robot model, or a working base station model, or other obstacle models that can be recognized by the mobile robot; whenever an object model is recognized, the recognized object model is loaded into the three-dimensional space through the resource loading function included in the graphics development kit, and the object model is rendered in the three-dimensional space through the rendering file included in the graphics development kit to configure the material information of the object model. Improve the object recognition function.

[0012] Further, in the case where the map model includes a ground model and multiple wall models, the rendered ground model displays the movement trajectory of the robot on the screen of the terminal; the loaded wall models are displayed on the screen of the terminal as a set of vertex coordinates of the walls. It can adapt to the map protocol and navigation path protocol of the preset floor cleaning robot.

[0013] Further, the method for loading and rendering each object model in the three-dimensional space through the resource loading function included in the graphics development package includes: defining a model rendering tool class from the graphics development package; loading each object model in the three-dimensional space through the resource loading function of the model rendering tool class; rendering each object model through the effect rendering function of the model rendering tool class; and displaying each rendered object model. Thus, loading and rendering each object model in the three-dimensional space is achieved, that is, by defining (which can also be understood as creating) a model rendering tool class and calling the relevant functions it includes to render and display the map model.

[0014] Further, the method for building a three-dimensional space based on the graphics development package includes: calling a renderer from the graphics development package and docking the renderer into the model rendering tool class; setting a spatial coordinate system through the renderer, thereby determining the construction of the three-dimensional space; and initializing the three-dimensional space, each object model, point light source, and material information by calling the view initialization function through the renderer. Thus, the scene arranged in the three-dimensional space is initialized through the view initialization function in the rendering method subclass, and further, the initialization and display of the three-dimensional space of the graphics development package can be realized.

[0015] Further, the interactive control method further includes: listening to the touch operations on the terminal through the touch event listening function included in the model rendering tool class and then feeding back to the screen of the terminal; in response to the listened touch operations, obtaining the touch events corresponding to the touch operations on the terminal through the event pass-through function; wherein, the interactive operations on the terminal include touch operations on the terminal; the touch events include touch events; wherein, the touch operations on the terminal include single-finger swiping the screen and double-finger swiping the screen, respectively corresponding to controlling the camera model to make an arc movement and controlling the perspective of the camera model to translate and adjust the angle value simultaneously. Thus, in the case where the map model is fixedly set in the three-dimensional space, the map model can be observed from different perspectives through the camera model.

[0016] Further, in the case where the interaction operation on the terminal changes, after the interaction operation that changes occurs on the terminal, the pose or perspective of the camera model is adjusted through a touch event handling function. When the pose or perspective of the camera model changes, the map model displayed on the screen of the terminal is triggered to be updated to change the perspective of the map model for external display. Description of the Drawings

[0017] Figure 1 This is a flowchart of an interactive control method for a map model disclosed in this embodiment. Embodiment

[0018] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described and explained below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application. The various non-limiting embodiments of the present application will be specifically introduced below.

[0019] The present application discloses an interactive control method for a map model. The interactive control method is executed by a terminal configured with an Android operating system. The implementation environment of the interactive control method includes: a terminal, a robot, walls, a ground in the indoor environment where the robot is located, and various obstacles of different shapes and sizes that may exist on the ground.

[0020] The interactive control method can be used to display a map model. The displayed map model is a three-dimensional map model constructed based on a rendering engine, and provides a series of human-computer interaction functions based on the map model, including using a camera model (which can be understood as triggered and controlled by a camera control tool class) in a renderer under an integrated library of OpenGL ES for user interaction, etc. This helps users, without applying interaction operations to the map model, to equivalently achieve the effects of flipping, dragging, and zooming of the map model through interaction operations on the camera model, so that users of a touch terminal can comprehensively and controllably display the entire map model effect.

[0021] As Figure 1 shown, the interactive control method includes the following processes: Build a three-dimensional space based on a graphics development package; in the present application, the terminal can call out the graphics development package within the Android operating system, and then call relevant initialization functions in a rendering method subclass within the graphics development package to build a three-dimensional space, forming a spatial coordinate system for marking the map model. The rendering method subclass is used to render a curved surface view. This three-dimensional space can be used to represent the coordinate space of the motion environment where the robot is located.

[0022] The way for the Android system to implement the three-dimensional space usually involves using a graphics library, a cross-platform graphics application programming interface, and a framework to create and render three-dimensional scenes. In the actual running scenario, OpenGL is a cross-platform graphics application programming interface (API) used to specify the software and hardware programming interfaces in three-dimensional (3D) graphics processing hardware. However, due to performance and portability reasons, the use of OpenGL on mobile devices such as terminals is relatively troublesome. To facilitate the use of OpenGL on terminals, a subset OpenGL ES (OpenGL for Embedded System) is created under OpenGL. OpenGL ES is a lightweight 3D graphics API for mobile devices, which allows developers to create high-performance 3D graphics applications on Android devices. Based on this, the graphics development package is found, so as to provide a cross-platform and feature-complete 3D graphics library interface for terminals through OpenGL ES. This application does not limit which version of OpenGL ES is specifically used.

[0023] The graphics development package is preferably the integrated library Rajawali3D of OpenGL ES. Rajawali3D is an open-source 3D graphics engine based on the Java language, belonging to a cross-platform graphics development package, and is designed for the development of high-performance graphics applications such as scientific computing visualization. The graphics development package is an object-oriented framework on top of OpenGL, which can liberate developers from implementing and optimizing low-level graphics calls, and can provide many additional utility tools for the rapid development of graphics applications, and is suitable for three-dimensional modeling on various Android device terminals.

[0024] The graphics development package focuses on creating high-performance 3D graphics applications on the Android platform. It provides a set of tools and libraries that enable developers to easily create realistic three-dimensional rendering effects, including functions such as model loading, material rendering, lighting effects, animation, and interaction by calling the provided related classes and defining class functions. After considering the development difficulty, development cost, and the impact of functions on the application, Rajawali3D is selected as the graphics development package to form a 3D map implementation engine for the sweeping robot.

[0025] Load and render each object model in a three-dimensional space through the resource loading function included in the graphics development package, and then combine each object model into a map model; it can be understood that a three-dimensional map model is displayed through some rendering tool classes (such as the SurfaceView class) included in the graphics development package. Among them, the terminal involved in this application can be a terminal using the Android operating system. When using Rajawali3D on the Android operating system, it can be implemented by attaching to the SurfaceView class. Therefore, the rendering tool class required to load and render each object model in a three-dimensional space can be the SurfaceView class.

[0026] Preferably, the SurfaceView class is a display class function provided by Rajawali3D for presenting the rendering effect of a three-dimensional space scene in the Android operating system. When the use of Rajawali3D on the Android operating system is implemented through the SurfaceView class, in some embodiments, before rendering each object model, a SurfaceView class is customized, which can specifically include creating a renderer inherited from Rajawali3D and capable of being docked to the SurfaceView for displaying a three-dimensional scene to build a three-dimensional space scene.

[0027] Based on the camera control tool class included in the graphics development package, set up a camera model in a three-dimensional space, and then control the corresponding touch events included in the model rendering tool class to establish a mapping relationship with the touch event processing function included in the camera control tool class, so as to bind the corresponding touch events included in the model rendering tool class to the touch event processing function included in the camera control tool class.

[0028] In this application, first define the camera control tool class from the graphics development package, for example, select the pre-set camera control tool class function or camera control tool function from Rajawali3D; then set up a camera model in a three-dimensional space through the camera control tool class. This camera model inherits from the camera control tool class and can be a subclass or a function defined by attaching to the camera control tool class. Specifically, the camera model is equivalent to a virtual model constructed in a three-dimensional space using the camera control tool class included in the graphics development package and is used as a reference observation object for the map model, independent of the hardware device. However, the relevant control functions included in the camera control tool class can adjust the viewing angle of the camera model or change the position of the camera model in a three-dimensional space. In some embodiments, the camera model can provide a camera control effect similar to a virtual trackball (Arcball). Therefore, during the rotation process of the camera model similar to a virtual track in a three-dimensional space, an effect similar to flipping the map model can be achieved.

[0029] Since the map model is configured not to be subjected to interaction operations, including not directly moving, rotating or scaling the map model through the model rendering tool class, and there is a pre-set mapping relationship between the corresponding touch events included in the model rendering tool class and the touch event handling functions included in the model rendering tool class, and the interaction operations on the terminal originally correspond to the corresponding touch events included in the model rendering tool class. That is, without other restrictions, in the prior art, the touch event corresponding to the interaction operation on the terminal is preferentially obtained through the screen event acquisition function included in the model rendering tool class, and the event type of the touch event is determined through the event type judgment function included in the model rendering tool class. Therefore, in this application, a mapping relationship is established between the corresponding touch events included in the model rendering tool class and the touch event handling functions included in the camera control tool class, so that when the corresponding touch events included in the model rendering tool class are obtained in response to an interaction operation, the touch event handling functions included in the camera control tool class are directly called, instead of calling the touch event handling functions included in the model rendering tool class, so as to change the camera model through the touch event handling functions instead of changing the map model through the touch event handling functions.

[0030] Whenever an interaction operation on the terminal is responded to, the touch event handling functions included in the camera control tool class are called based on the mapping relationship, and then the pose or viewing angle of the camera model is adjusted through the touch event handling functions, including controlling the rotation of the camera model, the arc movement of the camera model, the translation of the camera model or the viewing angle translation, or controlling the viewing angle scaling of the camera model. Therefore, in response to the user's interaction operation, the acquisition field of view of the camera model changes, so that in the case where the map model is fixedly set in the three-dimensional space, the map model can be observed from different viewing angles through the camera model. That is, the user can control the viewing angle of the camera model through drag gestures, rotation gestures and zoom gestures, and can also use two-finger sliding on the screen to control the simultaneous translation and scaling of the viewing angle of the camera model to observe the map model in an all-round and controllable manner.

[0031] Compared with the prior art, in response to an interaction operation, this application controls the change of the viewing angle and pose of the camera model by calling relevant processing functions, which is adapted to display the map model from multiple camera viewing angles in the case where the map model is fixedly set. From the perspective of the user observing the screen, it can equivalently achieve the rotation movement, movement, reduction, enlargement, etc. of the map model, which is not equivalent to transforming the map model observed from a single camera viewing angle (including rotation transformation, translation transformation and scaling transformation), so as to optimize the display effect of the map model in the three-dimensional space and strengthen the multi-dimensional space cognition.

[0032] After introducing the basic principles and implementation environment of the present application, the interactive control method provided by the present application will be described below in conjunction with several optional embodiments.

[0033] As an embodiment, the method of adjusting the pose or viewing angle of the camera model through the touch event processing function includes at least one of the following: Based on the touch event processing function included in the camera control tool class, control the camera model to perform an arc movement. For example, it can rotate around the map model (or one of the object models to be observed), so that different viewing angle regions of the map model are displayed on the screen of the terminal, and the map model is displayed on the screen at the same position; thus, it is equivalent to realizing the corresponding flip of the map model in three-dimensional space. Therefore, the arc movement (or circular movement) of the camera model can be realized through the interactive operation applied by the user, so as to equivalently realize the flip effect of the map model, and then the map model can be displayed in all viewing angles.

[0034] The arc movement of the camera model may include the rotation action of the camera model. The calculation process involved in the rotation of the camera model includes multiplying a pixel point by a unit quaternion through Quaternion (quaternion) to represent the rotation of the map model in three-dimensional space. Then the camera model rotates along a virtual track in three-dimensional space. From the imaging angle of the camera model, it is equivalent to realizing the effect of flipping the map model. And the function of flipping the map model can be realized by adding restrictions on the rotation angle to keep the map model facing forward in the user's field of view and through the logic of user interaction touch.

[0035] Based on the touch event processing function included in the camera control tool class, control the viewing angle of the camera model to translate, so that the map model is displayed as being dragged on the screen of the terminal, and the map model is displayed on the screen at the same position; then the corresponding area of the map model can be observed from the translated camera viewing angle in three-dimensional space; among them, the translation of the viewing angle of the camera model may be the translation of the position of the camera model, or the control of the viewing angle to translate alone without the camera translating, which is equivalent to realizing the corresponding translation of the map model in three-dimensional space, forming the phenomenon of dragging the map model.

[0036] Based on the touch event handling function included in the camera control tool class, the viewing angle of the camera model is enlarged, so that the image of the map model displayed on the screen of the terminal is reduced, and the map model is displayed on the screen at the same position; specifically, during the process of executing the touch event handling function included in the camera control tool class, the multiple by which the viewing angle of the camera model is enlarged is positively correlated with the multiple by which the image of the map model displayed on the screen of the terminal is reduced. In addition, based on the touch event handling function included in the camera control tool class, the viewing angle of the camera model is reduced, so that the image of the map model displayed on the screen of the terminal is enlarged, and the map model is displayed on the screen at the same position; specifically, during the process of executing the touch event handling function included in the camera control tool class, the multiple by which the viewing angle of the camera model is reduced is positively correlated with the multiple by which the image of the map model displayed on the screen of the terminal is enlarged. Therefore, by triggering the dragging and zooming of the camera model through interactive operations, the display angle of the map model is expanded.

[0037] It should be noted that the map model is fixed in the three-dimensional space, that is, the map model does not support automatic rotation and translation in the three-dimensional space. In this embodiment, only after the camera model responds to the interactive operation, composes the map model and displays it on the screen of the terminal, the user can perform interactive operations such as single-finger rotation, two-finger zooming in, two-finger zooming out, single-finger sliding or single-finger moving on the terminal to interact with the camera model; when the pose or viewing angle of the camera model changes, it causes the three-dimensional image of the map model displayed on the screen of the terminal to produce visual effects of flipping, translation, scaling transformation or a combination of two of them.

[0038] In some interactive scenarios, when the user single-finger slides the screen, it triggers the camera model to rotate around the center of the map model, causing the displayed map model to produce a flipping effect; when the user two-finger slides the screen, it triggers the viewing angle translation and FOV value adjustment of the camera model through the corresponding touch event handling function to simultaneously achieve the dragging and scaling effects of the same map model.

[0039] In summary, the user holds the terminal and keeps the relative position between the terminal and the object model in the environment where the robot is located unchanged. When the user performs an interaction operation on the terminal to cause a pose transformation or a perspective transformation of the camera model (including perspective translation and perspective zoom), but does not directly perform an interaction operation on the map model displayed on the screen of the terminal (such as clicking or swiping on the map model); as long as an interaction operation is performed on the screen of the terminal to cause a touch, a touch event is generated, causing a change in the pose of the camera model or the perspective of the camera model, which can be uniformly defined as controlling the perspective of the camera model; then during the process of responding to the interaction operation, different perspectives of the map model can be displayed on the screen of the terminal, or the map model can be displayed as being dragged, or the map model can be displayed as being stretched, realizing comprehensive and controllable display of various visual effects of the map model.

[0040] Specifically, in response to an interaction operation on the screen of the terminal, through the touch event acquisition function included in the model rendering tool class, the touch event corresponding to the interaction operation on the screen of the terminal is acquired, and the touch event corresponding to the interaction operation on the screen of the terminal is set as the touch event included in the model rendering tool class; wherein, in the relevant graphics libraries, application programming interfaces, and frameworks of the graphics development kit, the model rendering tool class is pre-set to present the rendering effect of a three-dimensional space scene in the Android operating system; the model rendering tool class, as a display class, can define a touch event acquisition function and correspond to the touch event corresponding to the interaction operation.

[0041] Through the event type judgment function included in the model rendering tool class, the event type of the touch event included in the model rendering tool class is determined; in this embodiment, the touch event acquisition function included in the model rendering tool class can intercept the touch events occurring on the screen of the terminal, so that the event type judgment function included in the model rendering tool class can judge the event type of the touch event based on the touch events intercepted by the touch event acquisition function included in the model rendering tool class. Among them, the event types of the touch events included in the model rendering tool class can include single-finger rotation event, single-click event, double-click event, single-finger touch event, two-finger touch event, two-finger zoom-in event, two-finger zoom-out event, swipe event, and so on. Optionally, the touch event can also include other types of events, which are not limited in this application.

[0042] Combined with the mapping relationship and the event type of the touch event, the touch event handling function included in the camera control tool class is called, that is, based on the mapping relationship, the touch event handling function included in the camera control tool class is mapped from the event type of the touch event included in the model rendering tool class. However, the touch event handling function included in the model rendering tool class is not directly defined and called within the model rendering tool class based on the event type of the touch event included in the model rendering tool class. Then, the pose or perspective of the camera model is configured through the touch event handling function included in the camera control tool class, so that the camera model can be translated or rotated in the three-dimensional space, and the perspective of the camera model can also be zoomed in or out (i.e., perform a scaling transformation). Before the map model displayed on the screen at the same position does not change relative to the camera model, corresponding movements, flips, or scalings occur, realizing the update of the displayed map model without calling the touch event handling function included in the model rendering tool class. On this basis, in response to the interaction operation on the screen of the terminal, the three-dimensional display effect of the map model can be optimized by calling the touch event handling function included in the camera control tool class, and the multi-dimensional map environment space can be recognized from different camera perspectives.

[0043] It should be noted that touch events of different event types correspond to different processing logics, and different processing logics correspond to codes corresponding to different values of the touch event handling function. That is, different time types correspond to different values of the touch event handling function. After determining the event type of the touch event, the response to the interaction operation can be achieved by executing the code corresponding to the value of the touch event handling function corresponding to the event type.

[0044] In some embodiments, for the event type of the touch event included in the same model rendering tool class, the touch event handling function included in the called camera control tool class and the touch event handling function included in the called model rendering tool class are not necessarily the same.

[0045] As an embodiment, the camera model is a camera tool in the renderer included in the camera control tool class, forming a camera tool with an adjustable perspective in the three-dimensional space, equivalent to the function of a camera tool, and can adjust parameters such as camera focal length and depth of field; it supports moving or changing the perspective in response to the interaction operation. From the perspective of the graphics library, the camera model is set based on the camera control class functions called from the graphics development package. As a camera tool in the pre-created renderer, it is specifically limited to use in the three-dimensional space, but in this embodiment, the device's camera is not called as the camera model.

[0046] After building the three-dimensional space, the renderer included in the camera control tool class is used to call the view initialization function in the rendering method subclass to set the initial position of the camera model directly above the map model along the vertical axis of the spatial coordinate system of the three-dimensional space; relative to the map model, the viewing angle of the camera model is adjustable and the position of the camera model is also variable. Therefore, by calling the touch event handling function included in the camera control tool class, different angular regions of the map model can be brought into the viewing angle of the camera model, and the map model that originally fell within the viewing angle of the camera model can also be scaled, meeting the user's need to observe the map model omnidirectionally through the camera model.

[0047] Optionally, the present application can also configure background light for the three-dimensional space, and set the illumination brightness of the point light source to be greater than the brightness of the background light of the three-dimensional space. Here, the point light source is used to simulate the light source center where the light covers the map model, that is, the point light source is set in the three-dimensional space in the form of a light source model to describe the ambient light situation in the three-dimensional space. The position points where the light emitted by the point light source passes through or is reflected in the three-dimensional space are all given brightness, and the given brightness is greater than the brightness of the background light of the three-dimensional space. The background light of the three-dimensional space is pre-configured low-intensity ambient white light, so configuring the background light is to configure a low-intensity ambient white light background for the three-dimensional space where the map model is located.

[0048] The brightness at the projection position of the point light source weakens as the distance from the point light source increases; in particular, within the effective illumination coverage range of the point light source, the constant attenuation factor is set to 50, the linear attenuation factor is set to 1, the quadratic factor for attenuation according to the square of the distance is set to 0.09, and the additional quadratic attenuation factor is set to 0.032.

[0049] The present application can also set the origin of the spatial coordinate system of the three-dimensional space to the center of the map model; configure a point light source directly above the map model along the vertical axis (such as the z-axis) of the spatial coordinate system, which can guide the light emitted by the point light source to vertically enter directly above the map model; when the pose or viewing angle of the camera model changes, the point light source emits light towards the map model at a fixed angle, and the illumination brightness of the point light source does not change due to the change of the interaction operation. The purpose is to keep the lighting effect unchanged relative to the user's touch on the terminal. Therefore, no matter how the camera model changes, the lighting effect in the three-dimensional space on the map model is relatively fixed, realizing that the lighting effect remains unchanged relative to the user's manipulation.

[0050] As an embodiment, the object model is a ground model, or a wall model, or a robot model, or a working base station model, or other obstacle models that can be recognized by a mobile robot to form the map model, which is applicable to represent the map in an indoor environment, such as a three-dimensional map required for a sweeping robot to navigate or a three-dimensional map required for a user to observe the indoor environment omnidirectionally. The other obstacle models that can be recognized by the mobile robot may include household items such as beds, dining tables, sofas, wardrobes, etc., and some small items such as clothes and shoes.

[0051] Whenever an object model is recognized, the recognized object model is loaded into the three-dimensional space through the resource loading function included in the graphics development package, and the object model is rendered in the three-dimensional space through the rendering file included in the graphics development package to configure the material information of the object model, where the material information includes the surface size and the surface illumination type. Here, the recognition action can be executed by the terminal, while the construction of the object model depends on the image information collected by an external sensor. Among them, when all object models are loaded together, it can be regarded as completing the loading of the map model.

[0052] Based on the classification of the foregoing object models, in this embodiment, recognizing an object model needs to adapt to the map protocol and navigation path protocol of a preset sweeping robot; when loading and rendering an object model, its distribution position needs to be considered. For example, various objects in a home environment, such as furniture and toys, can be recognized through the map model, so as to assist the sweeping robot to avoid these obstacles, ensure the safety and efficiency of the cleaning process, and improve the operation convenience and user experience of the sweeping robot; the composition of the object model also needs to be considered. If it can be split into relatively small pixels, it will increase the model rendering pressure.

[0053] As for the rendering file, it is default configured by an MTL file; for example, a furniture model is established by a dedicated model-making software, then loaded into the three-dimensional space through the resource loading function LoaderOBJ, and configured by its corresponding material rendering file (such as an MTL file), including the configuration of materials such as diffuse reflection and specular reflection on the model surface, making the map model more beautiful and practical.

[0054] In some embodiments, there is a ground model and multiple wall models in the map model, so that the map model is surrounded by a ground model and multiple wall models above it.

[0055] The rendered ground model displays the movement trajectory of the robot on the screen of the terminal; wherein, after loading the ground model in the three-dimensional space, the two-dimensional coordinate position information of the robot is included in the ground model to form the movement trajectory of the robot. The movement trajectory of the robot can include the starting point of movement, the movement direction, and the length of the movement trajectory, and supports adjustment; through the effect rendering function of the model rendering tool class, the ground surface of the ground model is set to a specular reflection or diffuse reflection effect to reflect the light reflection effect of the ground surface material, and the glossiness of the ground surface of the ground model is set to zero. The size of the ground surface and the type of ground surface material of the ground model can also be set.

[0056] The wall model loaded into the three-dimensional space through the resource loading function is displayed on the screen of the terminal as composed of the vertex set of the wall. Specifically, each wall model can be regarded as a wall, and a wall model can be divided into a cube composed of multiple squares or multiple triangles. The cube, square, and triangle are all composed of vertices, reducing the rendering pressure. The effect rendering function of the model rendering tool class can be used to set the wall surface of the wall model to a specular reflection or diffuse reflection effect to reflect the light reflection effect of the ground surface material, and set the glossiness of the wall surface of the wall model to zero. The size of the wall surface and the type of wall surface material of the wall model can also be set.

[0057] As an embodiment, the method for loading and rendering each object model in the three-dimensional space through the resource loading function included in the graphics development package includes: Define a model rendering tool class from the graphics development package; in the graphics development package, instantiate and define the model rendering tool class through the Android application, thereby starting the drawing thread of the custom SurfaceView class (corresponding to the model rendering tool class) instance, which is regarded as completing the definition of the model rendering tool class, so that the view initialization function of the Render (renderer) subclass instance can be called back; the functions included in the graphics development package can be stored in the dynamic library corresponding to the graphics development package, and the dynamic library corresponding to the graphics development package is pre-generated.

[0058] Load each object model in the three-dimensional space through the resource loading function of the model rendering tool class; wherein, this resource loading function can be the LoaderOBJ function. By using the LoaderOBJ function, the loading of the map model can be synchronized when the above-mentioned drawing thread is started. Furthermore, the map model is rendered by calling back the effect rendering function of the Render subclass instance.

[0059] Render each object model through the effect rendering function of the model rendering tool class so that the rendered object models can be displayed. Then, the rendered object models can be displayed on the screen of the terminal. Thus, it realizes the loading and rendering of each object model in the three-dimensional space, that is, by defining (which can also be understood as creating) a model rendering tool class and calling the relevant functions it includes to render and display the map model.

[0060] Based on the above embodiments, the method for building a three-dimensional space based on the graphics development kit includes: Set up a renderer based on the graphics development kit and dock the renderer into the model rendering tool class. It can be understood that the renderer here can be sourced from the model rendering tool class, equivalent to a subclass of the rendering method defined in the model rendering tool class. Then, set up the space coordinate system through the renderer, and thus determine the construction of the three-dimensional space. At the same time, call the view initialization function through the renderer to initialize the three-dimensional space, each object model, the point light source (the point light source required to be configured according to the foregoing embodiments), and the material information (the material information required to be rendered according to the foregoing embodiments); that is, initialize the scene arranged in the three-dimensional space through the view initialization function in the subclass of the rendering method, and then the initialization and display of the three-dimensional space of the graphics development kit can be realized.

[0061] Preferably, set the origin of the space coordinate system as the center point of the subsequent composed map model, and then initialize the three-dimensional space. Further, after building the three-dimensional space, call the view initialization function in the subclass of the rendering method through the renderer included in the camera control tool class, and set the initial position of the camera model directly above the map model along the vertical axis of the space coordinate system of the three-dimensional space; the renderer included in the camera control tool class can perform the same function as the function of the subclass of the rendering method defined in the model rendering tool class, and the scope of action of the renderer included in the camera control tool class is limited to the setting of the camera model.

[0062] As an embodiment, the interactive control method further includes: Through the touch event listening function included in the model rendering tool class, the touch operations on the terminal are listened to and then fed back to the screen of the terminal. Here, the feedback is to call back the registered listener function to the interface event handling class in the graphics development package through the touch event listening function, and then respond to the touch operations on the terminal listened to through the registered listener function. In response to the listened touch operations, the touch events corresponding to the touch operations on the terminal are obtained through the event passthrough function, and correspondingly, they can be set as the touch events included in the model rendering tool class. It should be noted that the interaction operations on the terminal include the touch operations on the terminal; the touch events include the touch events; among them, the touch operations on the terminal include single-finger swiping the screen and double-finger swiping the screen, which respectively correspond to controlling the camera model to perform an arc movement and controlling the perspective of the camera model to translate and the angle value to be adjusted simultaneously. Thus, when the map model is fixedly set in the three-dimensional space, the map model can be observed from different perspectives through the camera model.

[0063] Generally, the touch operations on the terminal include single-finger rotation, swiping the screen, double-finger zooming in, and double-finger zooming out, which can respectively correspond to controlling the camera model to rotate around the map model, controlling the perspective of the camera model to translate, enlarging the perspective of the camera model, and reducing the perspective of the camera model, or can also correspond to a combination of two transformation methods of the camera model.

[0064] Specifically, in this embodiment, after establishing a mapping relationship between the corresponding touch events included in the control model rendering tool class and the touch event processing functions included in the camera control tool class, there are: whenever listening to and responding to the user's single-finger swiping the screen, the camera model performs an arc movement, and the displayed map model equivalently shows a flipping effect. Whenever listening to and responding to the user's double-finger swiping the screen, the perspective translation of the camera model and the angle adjustment of the perspective of the camera model (including enlarging or reducing the perspective) are simultaneously realized, and the displayed map model equivalently shows a dragging effect and a zooming effect.

[0065] As an embodiment, when the interaction operation on the terminal changes, after responding to the changed interaction operation on the terminal, the pose or perspective of the camera model is adjusted through the touch event processing function. When the pose or perspective of the camera model changes, the map model displayed on the screen of the terminal is triggered to be updated to change the perspective of the map model for external display; the specific update forms include enlarging the displayed map model; reducing the displayed map model; rotating the displayed map model; moving the displayed map model.

[0066] Among them, the change in the viewing angle of the camera model includes the translation of the viewing angle of the camera model, the enlargement or reduction of the viewing angle of the camera model; the change in the pose of the camera model includes the arc movement of the camera model.

[0067] In some embodiments, when the interaction operation on the terminal is single-finger rotation, the arc movement of the camera model is controlled through the touch event processing function; when the interaction operation on the terminal is a slide (the touch event is a slide event), the translation of the viewing angle of the camera model is controlled through the touch event processing function, which is equivalent to the translation of the camera model; when the interaction operation on the terminal is two-finger zoom-in, the viewing angle of the camera model is enlarged through the touch event processing function; when the interaction operation on the terminal is two-finger zoom-out, the viewing angle of the camera model is reduced through the touch event processing function. Of course, when the interaction operation on the terminal is two-finger slide, the viewing angle of the camera model can also be enlarged or reduced through the touch event processing function while controlling the translation of the same camera model; when the interaction operation on the terminal is single-finger slide, the arc movement or rotation of the camera model can be controlled through the touch event processing function.

[0068] It should be noted that touch events of different event types correspond to different processing logics. By determining the event type of the touch event, the corresponding touch event processing function can be determined according to the determined touch event type.

[0069] This application also provides a terminal, which includes a processor, a memory, and a network interface. The memory is used to store computer program code that can run on the processor. The processor is used to implement the interaction control method provided in any embodiment of this application when executing the computer program code. The network interface is used to implement input and output functions and can also perform information interaction with the sweeping robot, including receiving the images collected by the sweeping robot and the instantaneously constructed map information.

[0070] In more possible implementation manners, the terminal may further include other hardware, which is not limited in this application.

[0071] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be in various forms. For example, in different examples, the computer-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or a combination thereof. Specifically, the computer-readable medium can also be paper or other suitable media capable of printing programs. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the interaction control method provided in any embodiment of the present application.

[0072] The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the interaction control method provided in any embodiment of the present application.

[0073] In the present application, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. The term "plurality" refers to two or more, unless otherwise clearly defined.

[0074] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the disclosure herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include common general knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and examples are only considered exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0075] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. An interactive control method for a map model, characterized in that, the interactive control method is executed by a terminal configured with an Android operating system; the interactive control method includes: Constructing a three-dimensional space based on a graphics development kit; Loading and rendering each object model in the three-dimensional space through a resource loading function included in the graphics development kit, and then forming a map model by combining each object model; Based on a camera control tool class included in the graphics development kit, setting a camera model in the three-dimensional space, and then controlling a corresponding touch event included in a model rendering tool class to establish a mapping relationship with a touch event processing function included in the camera control tool class; Whenever an interaction operation on the terminal is responded to, based on the mapping relationship, calling a touch event processing function included in the camera control tool class, and then adjusting the pose or viewing angle of the camera model through the touch event processing function, so that when the map model is fixedly arranged in the three-dimensional space, the map model can be observed from different viewing angles through the camera model.

2. The interactive control method according to claim 1, characterized in that, the method of adjusting the pose or viewing angle of the camera model through the touch event processing function includes at least one of the following: Based on the touch event processing function included in the camera control tool class, controlling the camera model to perform an arc motion, so that areas of the map model at different viewing angles are displayed on the screen of the terminal; Based on the touch event processing function included in the camera control tool class, controlling the viewing angle translation of the camera model, so that the map model is displayed as being dragged on the screen of the terminal; Based on the touch event processing function included in the camera control tool class, enlarging the viewing angle of the camera model, so that the image of the map model displayed on the screen of the terminal is reduced; Based on the touch event processing function included in the camera control tool class, reducing the viewing angle of the camera model, so that the image of the map model displayed on the screen of the terminal is enlarged; wherein, the map model is fixedly immovable in the three-dimensional space.

3. The interactive control method according to claim 2, characterized in that, In response to an interaction operation on the screen of the terminal, through a touch event acquisition function included in the model rendering tool class, acquiring a touch event corresponding to the interaction operation on the screen of the terminal, and setting the touch event corresponding to the interaction operation on the screen of the terminal as the touch event included in the model rendering tool class; Determining the event type of the touch event included in the model rendering tool class through an event type judgment function included in the model rendering tool class; Combined with the mapping relationship and the event type of the touch event, call the touch event handling function included in the camera control tool class, and then configure the pose of the camera model or the perspective of the camera model through the touch event handling function included in the camera control tool class, so that the camera model makes an arc movement or the perspective of the camera model is adjusted, realizing the update of the displayed map model without calling the touch event handling function included in the model rendering tool class.

4. The interactive control method according to claim 3, wherein, the camera model is a camera tool in the renderer included in the camera control tool class, forming a camera tool with adjustable perspective in the three-dimensional space; wherein, after building the three-dimensional space, the view initialization function in the rendering method subclass is called through the renderer included in the camera control tool class, and the initial position of the camera model is set directly above the map model along the vertical axis of the space coordinate system of the three-dimensional space.

5. The interactive control method according to claim 2, wherein, configure background light for the three-dimensional space, and set the illumination brightness of the point light source to be greater than the brightness of the background light of the three-dimensional space, wherein the brightness at the projection position of the point light source is configured to attenuate as the distance from the point light source increases; set the origin of the space coordinate system of the three-dimensional space as the center of the map model; configure a point light source directly above the map model along the vertical axis of the space coordinate system, so that when the pose of the camera model or the perspective of the camera model changes, the point light source emits light towards the map model at a fixed angle, and the illumination brightness of the point light source does not change due to the change of the interaction operation.

6. The interactive control method according to claim 1, wherein, the object model is a ground model, or a wall model, or a robot model, or a working base station model, or other obstacle models that can be recognized by the mobile robot; whenever an object model is recognized, the recognized object model is loaded into the three-dimensional space through the resource loading function included in the graphics development kit, and the object model is rendered in the three-dimensional space through the rendering file included in the graphics development kit to configure the material information of the object model, wherein the material information includes surface size and surface illumination type.

7. The interactive control method according to claim 6, wherein, when there is a ground model and multiple wall models in the map model, the rendered ground model displays the movement trajectory of the robot on the screen of the terminal; the loaded wall model is displayed on the screen of the terminal as composed of a set of vertex coordinates of the wall.

8. The interactive control method according to claim 1, wherein, the method of loading and rendering each object model in the three-dimensional space through the resource loading function included in the graphics development kit includes: define a model rendering tool class from the graphics development kit; load each object model into the three-dimensional space through the resource loading function of the model rendering tool class; Render each object model through the effect rendering function of the model rendering tool class; Display each object model obtained by rendering.

9. The interactive control method according to claim 8, characterized in that, the method for building a three-dimensional space based on the graphics development package includes: Call a renderer from the graphics development package and dock the renderer into the model rendering tool class; Set a spatial coordinate system through the renderer, and then determine that the three-dimensional space is built; Call a view initialization function through the renderer to initialize the three-dimensional space, each object model, point light sources, and material information.

10. The interactive control method according to claim 2, characterized in that, the interactive control method further includes: Monitor touch operations on the terminal through the touch event listening function included in the model rendering tool class, and then feedback to the screen of the terminal; In response to the monitored touch operation, obtain a touch event corresponding to the touch operation on the terminal through an event pass-through function; wherein, the interactive operation on the terminal includes a touch operation on the terminal; the touch event includes a touch event; wherein, the touch operation on the terminal includes single-finger swiping the screen and double-finger swiping the screen, respectively corresponding to controlling the camera model to make an arc motion and controlling the perspective of the camera model to translate and adjust the angle value simultaneously.

11. The interactive control method according to claim 10, characterized in that, When the interactive operation on the terminal changes, after responding to the changed interactive operation on the terminal, adjust the pose or the perspective of the camera model through a touch event processing function. When the pose or the perspective of the camera model changes, trigger an update of the map model displayed on the screen of the terminal to change the perspective of the map model for external display; wherein, the change in the perspective of the camera model includes the translation of the perspective of the camera model, the perspective of the camera model being enlarged or reduced; the change in the pose of the camera model includes the camera model making an arc motion.

Images