Information transmission method, device and storage medium

Abstract

Embodiments of the present application provide an information transmission method and device and a storage medium. In the embodiments of the present application, an autonomous mobile device can collect an environment image during a work process and transmit the environment image to a terminal device. Before being transmitted to the terminal device, it can be determined whether the environment image contains a target partition reference. If the environment image contains the target partition reference, the environment information of a target area in the environment image divided by the target partition reference is identified, and the environment image is filtered according to the environment information. Then, the target image obtained through the filtering process is provided to the terminal device. The target image from which some environment information is filtered can reduce the data transmission amount on the basis of meeting the demand for remote monitoring or intelligent management and control, and / or protect user privacy and improve service quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to an information transmission method, device and storage medium. BACKGROUND

[0002] Mobile robots are gradually used in people's daily work and life, bringing great convenience to people. Many mobile robots are installed with cameras, and positioning and obstacle avoidance can be achieved based on the environment images collected by the cameras. With the help of the cameras, many robots support remote monitoring or intelligent management and control functions. Specifically, the robots provide the images collected by the cameras to a server, and the server provides the images to a terminal device, and a user realizes remote monitoring or intelligent management and control of the environment where the robot is located through the terminal device. However, there are still some deficiencies in the remote monitoring or intelligent management and control services provided by the robots for users, and the service quality is not high. SUMMARY

[0003] The present application provides an information transmission method, device and storage medium, which can improve the service quality of the remote monitoring or intelligent management and control services provided by the robots for users.

[0004] The present application provides an information transmission method, which is suitable for an autonomous mobile device. The method comprises: in a work process, collecting an environment image by using a visual sensor; in a case where a target partition reference is contained in the environment image, identifying environment information in a target region contained in the environment image; performing information filtering on the environment image according to the environment information in the target region to obtain a first target image; and transmitting the first target image to a terminal device bound with the autonomous mobile device, the target region being a functional region in a work environment divided by the target partition reference.

[0005] The present application also provides an autonomous mobile device, which comprises: a device body, a memory and a processor being arranged on the device body; the memory is used for storing a computer program; the processor is coupled with the memory and is used for executing the computer program, so as to: in a work process, collect an environment image by using a visual sensor; in a case where a target partition reference is contained in the environment image, identify environment information in a target region contained in the environment image; perform information filtering on the environment image according to the environment information in the target region to obtain a first target image; and transmit the first target image to a terminal device bound with the autonomous mobile device, the target region being a functional region in a work environment divided by the target partition reference.

[0006] The present application also provides a computer readable storage medium storing a computer program, when the computer program is executed by a processor, the processor is caused to realize the steps in the information transmission method provided by the present application.

[0007] In the embodiments of the present application, the autonomous mobile device can collect an environment image during the operation and transmit it to the terminal device. Before transmission to the terminal device, it can be determined whether the environment image contains a target partition reference, and in the case where the environment image contains the target partition reference, the environment information of the target region contained in the environment image is identified, and the environment image is filtered accordingly, and then the target image obtained by filtering is provided to the terminal device. Wherein, the target image which transmits some environment information filtered, on the basis of meeting the demand of remote monitoring or intelligent control, can also reduce the data transmission amount, and / or protect the user privacy, improve the service quality. BRIEF DESCRIPTION OF DRAWINGS

[0008] The accompanying drawings, which are included to provide a further understanding of the present application, form a part of the present application and illustrate the illustrative embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0009] Figure 1 A flowchart of an information transmission method provided for the illustrative embodiments of the present application;

[0010] Figure 2 A schematic diagram of a 3D environment map provided for the illustrative embodiments of the present application;

[0011] Figure 3a A schematic diagram of a local environment map under the perspective of an autonomous mobile device provided for the illustrative embodiments of the present application;

[0012] Figure 3b A schematic diagram of an environment information containing target region under the perspective of an autonomous mobile device provided for the illustrative embodiments of the present application;

[0013] Figure 3c A schematic diagram of filtering environment information of a target region under the perspective of an autonomous mobile device provided for the illustrative embodiments of the present application;

[0014] Figure 4 A structural schematic diagram of an autonomous mobile device provided for the illustrative embodiments of the present application;

[0015] Figure 5 A structural schematic diagram of a sweeping robot provided for the illustrative embodiments of the present application. DETAILED DESCRIPTION

[0016] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in connection with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0017] At present, robots can realize positioning and obstacle avoidance based on environment images collected by cameras. With the help of cameras, many robots support remote monitoring or intelligent control functions, but there are deficiencies, resulting in low service quality. For example, there may be a problem of large data transmission volume and waste of resources. Or, in the process of remote monitoring or intelligent control, user privacy may be involved. Optionally, the user privacy can be protected by turning off the camera, but the camera carries other functions such as positioning or mapping function, and turning off the camera will affect the normal work of the robot. In view of the technical problems faced, in the embodiments of the present application, the autonomous mobile device collects environment images during work and transmits them to the terminal device bound thereto; and before transmitting the environment images to the terminal device, it can be judged whether the environment images contain target partition reference, and in the case that the environment images contain target partition reference, the environment information of the target area divided by the target partition reference contained in the environment images is identified, and the environment images are filtered accordingly, so as to provide the target images obtained by filtering to the terminal device. As can be seen, on the basis of meeting the demand of remote monitoring or intelligent control, the environment images with some environment information filtered out are transmitted, which can also reduce the data transmission volume and / or protect the user privacy and improve the service quality.

[0018] The technical solutions provided by the embodiments of the present application will be described in detail below in connection with the drawings.

[0019] Figure 1 A flowchart of an information transmission method provided by an exemplary embodiment of the present application is shown in FIG. Figure 1 As shown in the figure, the method comprises:

[0020] S101, collecting environment images by using a visual sensor during work;

[0021] S102, in the case that the environment images contain target partition reference, identifying the environment information in the target area contained in the environment images, the target area being a functional area in the work environment divided by the target partition reference.

[0022] S103, filtering the environment images according to the environment information in the target area contained in the environment images to obtain first target images;

[0023] S104, transmit the first target image to a terminal device bound with the autonomous mobile device.

[0024] The autonomous mobile device in the embodiments of the present application can be any mechanical device capable of highly autonomously moving in a designated environment space, for example, a robot, a purifier, an unmanned carrier vehicle, etc. The robot can be a household robot, such as a sweeping robot, a home companion robot, etc., or a commercial robot, such as a cruising robot, a guiding robot or a service robot for assisting users to handle various affairs in places such as shopping malls, supermarkets, banks, hospitals, airports or railway stations, and a carrier robot for use in various warehouses, etc.

[0025] In the embodiments of the present application, the autonomous mobile device has a traveling device and can autonomously move in a current environment to perform a task; in addition, the autonomous mobile device also has a visual sensor, such as a monocular camera, a binocular camera or a depth camera (Red Green Blue-Depth map, RGB-D), etc. The autonomous mobile device can collect environment images of the environment in which the autonomous mobile device is located by using the visual sensor during the task.

[0026] On one hand, the autonomous mobile device can locate its position and pose according to the collected environment images and an existing environment map, realize autonomous movement and provide a basis for performing a task. Alternatively, the environment map can be constructed by the autonomous mobile device using simultaneous localization and mapping (SLAM) technology, or can be generated by other devices in advance and imported into the autonomous mobile device. The environment map can be 2D or 3D, such as the example schematic diagram of a 3D environment map shown in FIG. 2. Figure 2

[0027] On the other hand, the autonomous mobile device can also provide remote monitoring or intelligent management and control services for users based on the collected environment images. For example, the collected environment images are transmitted to a terminal device bound with the autonomous mobile device in advance, such as a user's mobile phone, tablet computer or wearable device, etc. The user can view the environment images in the working environment of the autonomous mobile device through the terminal device, obtain information in the working environment of the autonomous mobile device and realize remote monitoring; or according to the obtained information, the user can also intelligently manage and control other devices in the working environment (such as air conditioners, water heaters, televisions, etc. in a home working environment) through the autonomous mobile device.

[0028] ​In the embodiment, in the process of implementing the remote monitoring or intelligent management and control function by the autonomous mobile device, in order to improve the service quality, some environment information in the environment image can be filtered in the granularity of the functional area in the working environment, so as to further meet other requirements on the basis of meeting the remote monitoring or intelligent management and control requirements, and improve the service quality. The functional area refers to an area with a clear functionality in the working environment, and the functionality here refers to the main function of the area. Taking a family scene as an example, the bedroom is a functional area, and its main function is to provide a place for the user to rest; the living room is a functional area, and its main function is to provide a place for the user to relax and entertain guests; the kitchen is a functional area, and its main function is to provide a place for the user to cook food; the study is a functional area, and its main function is to provide a place for the user to study and read books; and so on.

[0029] In the remote monitoring or intelligent management and control scene, the user can have different monitoring or intelligent management and control requirements for different functional areas. For example, the user wants to monitor the situation in the living room, kitchen, balcony and other areas, but wants to protect the privacy and safety of the bathroom or bedroom. In view of this, the autonomous mobile device can filter the environment information containing the bathroom or bedroom in the environment image to achieve the purpose of protecting the user's privacy and safety while meeting the remote monitoring or intelligent management and control requirements, ensuring data security and improving service quality. For another example, the user's study has confidential documents, and the user does not want to leak the situation of the study area. At the same time, the user also wants to monitor the situation in the kitchen, living room, bedroom and balcony and other areas. Then the autonomous mobile device can filter the environment information containing the study area in the environment image to achieve the purpose of protecting the privacy of the study area while meeting the remote monitoring or intelligent management and control requirements, ensuring data security and improving service quality. For another example, in some application scenarios, the user only wants to focus on the situation in a working area, for example, the user only wants to focus on the situation in the children's room, and does not pay much attention to the situation in the living room, balcony and kitchen and other areas. Then the autonomous mobile device can filter the environment information in the environment image except for the children's room to achieve the purpose of saving transmission bandwidth while meeting the remote monitoring or intelligent management and control requirements, improving data transmission speed and fluency, and improving service quality.

[0030] In the embodiment, the functional area refers to an area divided by a partition reference object, rather than an arbitrary area in the working environment. The partition reference object refers to an object with a partition function in the working environment, such as a wall, a door, a door frame, a glass wall, a partition or a screen. Based on this, for a target area in the working environment related to user privacy or needing to be focused on, the autonomous mobile device can determine whether the environment information in the target area is contained in the captured environment image according to the target partition reference object. The target partition reference object refers to a partition reference object that divides the target area, which can be a door, a door frame, a glass wall, a partition or a screen. That is, the autonomous mobile device can determine whether the target partition reference object is contained in the collected environment image; if the target partition reference object is contained in the environment image, it means that the environment information in the target area divided by the target partition reference object is contained in the environment image, and thus it can be identified that the environment information in the target area is contained in the environment image. Furthermore, according to the user's remote monitoring or intelligent management needs, the environment image can be filtered according to the environment information in the target area contained in the environment image to obtain a first target image; and the first target image is transmitted to a terminal device bound with the autonomous mobile device. In this way, the user can view the environment image in the working environment of the autonomous mobile device through the terminal device, obtain the information in the working environment of the autonomous mobile device, and thus realize remote monitoring or intelligent management.

[0031] In the embodiment, the autonomous mobile device can determine the target area and the corresponding target partition reference object, but the implementation of the autonomous mobile device determining the target area and the corresponding target partition reference object is not limited, which is illustrated as follows:

[0032] In an optional embodiment P1, the user can pre-set the target area through the terminal device. For example, in a privacy protection scenario, the user can set a bathroom or a bedroom and the like related to user privacy as the target area; or, in a scenario of focusing on a certain area, the user can set a child room and the like which the user wants to focus on as the target area. One way in which the user pre-sets the target area through the terminal device includes: the terminal device can show the user an environment map, which can be a 2D environment map or a 3D environment map; the user selects the target area based on the environment map shown by the terminal device; the terminal device generates configuration information in response to the selection operation of the user, the configuration information including identification information of the target area configured by the user, the identification information of the target area being semantic information or position information of the target area in the environment map, depending on the environment map; and the terminal device sends the configuration information to the autonomous mobile device, which receives the configuration information sent by the terminal device and marks the target area in the environment map of the autonomous mobile device based on the identification information of the target area carried in the configuration information. Further, the identification information of the target partition reference can be determined according to the identification information of the target area. In this embodiment, the autonomous mobile device pre-stores or maintains the identification information of various partition references existing in the working environment and the corresponding relationship between the partition references and the functional partitions. Alternatively, the identification information of the partition references and the corresponding relationship between the partition references and the functional partitions can be embodied in the environment map.

[0033] In yet another optional embodiment P2, at least one of the area features and the partition reference features can be pre-set on the autonomous mobile device, or pre-configured by the user on the autonomous mobile device, and these information are used for the autonomous mobile device to identify the target area and the corresponding target partition reference. The autonomous mobile device can identify the target area and the corresponding target partition reference in the working environment based on the at least one of the set area features and the partition reference features, and mark the target area and the target partition reference in the environment map. The area features refer to some feature information that can describe the area, for example, can be the objects contained in the area, such as the toilet or bathtub in the bathroom, the bed in the bedroom, or the semantic information of the area, such as the information of the bedroom, kitchen and the like; the partition reference features refer to the features of the objects in the working environment that have the partition effect, for example, can be the width, height and position of the door and the like. The following is an example:

[0034] Example A1: The autonomous mobile device can identify a target region and a target partition reference in the working environment in combination with the preset region features and the preset partition reference features. For example, if the preset region features are toilet and bathtub, and the preset partition reference features are the position of the door, the autonomous mobile device can identify the bathroom region where the toilet and bathtub are located as the target region according to the position of the door, and mark the bathroom as the target region in the environment map, while marking the door of the bathroom as the target partition reference corresponding to the bathroom.

[0035] Example A2: The autonomous mobile device can identify a target region and a target partition reference in the working environment according to the preset partition reference features alone. For example, if the preset partition reference features are the width, height and position of the door, the autonomous mobile device can identify the study room region divided by the door as the target region according to the width, height and position of the door, and mark the study room region as the target region in the environment map, while marking the door of the study room region as the target partition reference corresponding to the study room.

[0036] In this embodiment, when the target partition reference is included in the environment image, the environment information in the target region included in the environment image can be identified, and the environment image is filtered according to the environment information in the target region included in the environment image. Specifically, the environment image is filtered according to the environment information in the target region included in the environment image in combination with the user's demand for the target region in the process of remote monitoring or intelligent management and control. For example, if the user wants to protect the privacy information in the target region, the environment information in the target region included in the environment image can be occluded, blurred or replaced to obtain a first target image, so as to protect the user's privacy and ensure the safety of data transmission. For another example, if the user wants to focus on the information in the target region, the information other than the environment information in the target region in the environment image can be occluded, blurred or replaced to obtain a first target image, so as to let the user focus on the information in the target region, and reduce the data transmission amount, improve the data transmission efficiency, save network flow resources, etc.

[0037] Further, in some cases, the target subzone reference object can not be included in the environment image captured by the autonomous mobile device. The target subzone reference object not being included in the environment image does not mean that the environment information in the target region is not included in the environment image. For example, the autonomous mobile device is outside the target region and moves to a position far away from the target region, and the field of view range of the visual sensor cannot cover the target subzone reference object at this time. In this case, the environment information in the target region is not included in the environment image captured by the visual sensor. For another example, the autonomous mobile device is inside the target region, and the field of view range of the visual sensor can also not cover the target subzone reference object at this time. In this case, the environment information in the target region is included in the environment image captured by the visual sensor. In view of this, for the case that the target subzone reference object is not included in the environment image, the positional relationship between the autonomous mobile device and the target region can be determined, the environment image is filtered according to the positional relationship to obtain a second target image, and the second target image is transmitted to the terminal device to meet the needs of the user for remote monitoring or intelligent management and control.

[0038] In the case that the target subzone reference object is not included in the environment image, the environment image is filtered according to the positional relationship between the autonomous mobile device and the target region, which needs to be combined with the needs of the user for the target region in the process of remote monitoring or intelligent management and control. For example, if the user wants to protect the private information in the target region, when the positional relationship between the autonomous mobile device and the target region indicates that the autonomous mobile device is inside the target region, it is considered that most or all of the information in the environment image comes from the target region. At this time, if the environment information in the target region included in the environment image is shielded, blurred or replaced, the meaning of the obtained second target image will not be great, so a local environment map corresponding to the environment image can be obtained as the second target image, and the second target image is transmitted to the terminal device to meet the needs of the user for remote monitoring or intelligent management and control. For another example, if the user wants to focus on the information in the target region, when the positional relationship between the autonomous mobile device and the target region indicates that the autonomous mobile device is inside the target region, it is considered that most or all of the information in the environment image comes from the target region. In this case, the environment image can not be filtered, but the environment image is directly provided as the second target image to the terminal device, so that the user can focus on the information in the target region, and the needs of the user for remote monitoring or intelligent management and control are met.

[0039] Further optionally, if the user wants to protect the privacy information in the target area, when the positional relationship between the autonomous mobile device and the target area indicates that the autonomous mobile device is located outside the target area, it means that the environment information in the target area is not contained in the environment image, so the environment image can not be filtered, but directly transmitted to the terminal device, meeting the needs of the user for remote monitoring or intelligent management. If the user wants to focus on the information in the target area, when the positional relationship between the autonomous mobile device and the target area indicates that the autonomous mobile device is located outside the target area, it means that the environment information in the target area is not contained in the environment image, at this time, the environment image has no meaning to the user, and the environment image can not be transmitted to the terminal device.

[0040] In the embodiments of the present application, first, whether the target partition reference is contained in the environment image can be determined in combination with the position and pose of the autonomous mobile device; in the case that it is determined that the target partition reference is not contained in the environment image, the positional relationship between the autonomous mobile device and the target area can be further determined in combination with the position and pose of the autonomous mobile device. Specifically, the position and pose of the autonomous mobile device can be located according to the existing environment map and the collected environment image; according to the position and pose of the autonomous mobile device, the field of view range of the vision sensor on the autonomous mobile device can be determined; wherein the pose of the autonomous mobile device mainly refers to the orientation of the autonomous mobile device, further, in combination with the installation position relationship of the vision sensor on the autonomous mobile device, the field of view range of the vision sensor can be determined; if the target partition reference falls within the field of view range of the vision sensor, it indicates that the vision sensor can capture the environment information in the target area, so it can be determined that the environment image contains the environment information in the target area, and the environment image is filtered according to the environment information in the target area to obtain a first target image, and the first target image is transmitted to the terminal device.

[0041] If the target partition reference does not fall within the field of view range of the vision sensor, it cannot be determined whether the vision sensor can capture the environment information in the target area, which has a certain relationship with the positional relationship between the autonomous mobile device and the target area. In view of this, whether the autonomous mobile device is located inside or outside the target area can be identified based on the position of the autonomous mobile device. In the case that the target partition reference is not contained in the environment image, if the autonomous mobile device is located inside the target area, it indicates that the environment image contains the environment information in the target area; if the autonomous mobile device is located outside the target area, it indicates that the environment image does not contain the environment information in the target area.

[0042] Wherein whether the target partition reference falls within the field of view range of the vision sensor can be determined in the following ways, but is not limited to the following ways:

[0043] Method D1: Input the environmental image into the image recognition model to detect the target partition reference object; if the environmental image contains the target partition reference object, it means that the target partition reference object falls within the field of view of the visual sensor; otherwise, it is determined that the target partition reference object does not fall within the field of view of the visual sensor.

[0044] Method D2: The autonomous mobile device knows the world coordinates of the target area reference object, i.e., the coordinates of the target area reference object in the environmental map, and knows the transformation relationship between the world coordinate system and the sensor coordinate system. Based on this transformation relationship, the world coordinates of the target area reference object can be transformed to the sensor coordinate system to obtain the coordinates of the target area reference object in the sensor coordinate system. Furthermore, the field of view of the visual sensor includes several position coordinates in the sensor coordinate system. If these position coordinates include the coordinates of the target area reference object in the sensor coordinate system, then the target area reference object is determined to fall within the field of view of the visual sensor; otherwise, it is determined that the target area reference object is not within the field of view of the visual sensor. The sensor coordinate system refers to the coordinate system used by the visual sensor; the world coordinate system refers to the coordinate system used by the autonomous mobile device.

[0045] Method D3: On one hand, the environmental image is input into the image recognition model to detect the target partition reference object. If the environmental image contains the target partition reference object, the coordinates of the target partition reference object in the environmental image are obtained as the first coordinate. On the other hand, according to the transformation relationship between the world coordinate system and the sensor coordinate system, the known world coordinates of the target partition reference object are transformed to the sensor coordinate system to obtain the coordinates of the target partition reference object in the sensor coordinate system, which are denoted as the second coordinate. Then, the third coordinate is calculated based on the first and second coordinates. Optionally, the third coordinate can be the average of the first and second coordinates, but it is not limited to this. If the position coordinates in the sensor coordinate system covered by the field of view of the vision sensor include the third coordinate, it is determined that the target partition reference object falls within the field of view of the vision sensor; otherwise, it is determined that the target partition reference object does not fall within the field of view of the vision sensor.

[0046] In methods D2 and D3 above, assuming the environment map is a 3D map, then assuming the world coordinates of the target partition reference object obtained from the 3D map are P w =(x,y,z) T Given that the intrinsic and extrinsic parameters of the vision sensor are the intrinsic parameter matrix K and the extrinsic parameter matrix T, respectively, we can use formula P... uv =KTP w Calculate the coordinates P of the target partition reference object in the sensor coordinate system. uv =(u,v,1) T .

[0047] In the embodiments of the present application, the manner of information filtering of the environment image will be different considering different application scenarios and requirements. The following will be exemplarily described in two application scenarios, and four cases in each application scenario:

[0048] In one application scenario W1, the user wants to realize remote monitoring or intelligent management and control of the situation in the working environment through the autonomous mobile device, while at the same time, wants to protect the privacy of certain functional areas (such as target areas) in the working environment. In view of this application requirement, the autonomous mobile device specifically performs the following operations to meet the user's such application requirement:

[0049] Case B1: The environment image collected by the autonomous mobile device using the visual sensor includes the target partition reference, and the autonomous mobile device is located outside the target area, which means that the field of view range of the visual sensor at least partially covers the target area, that is, the environment image includes the environment information in the target area, then the environment information in the target area contained in the environment image is occluded, blurred or replaced to obtain a first target image; the first target image is transmitted to the terminal device bound with the autonomous mobile device. The first target image includes the environment information in the non-target area.

[0050] Case B2: The environment image collected by the autonomous mobile device using the visual sensor includes the target partition reference, and the autonomous mobile device is located in the target area, that is, the environment image includes the environment information in the target area, then the environment information in the target area contained in the environment image is occluded, blurred or replaced to obtain a first target image; the first target image is transmitted to the terminal device bound with the autonomous mobile device. The first target image includes the environment information in the non-target area.

[0051] In cases B1 and B2, the autonomous mobile device can identify the environment information in the target area contained in the environment image according to the target partition reference contained in the environment image. Specifically, the world coordinates of the target partition reference in the world coordinate system are also determined and known, so the world coordinates of the target partition reference in the environment image can be determined according to the known world coordinates of the target partition reference; according to the coordinates of the target partition reference in the environment image, the environment information in the target area contained in the environment image can be determined; for example, it can be the environment information behind, in front of or on the side of the target partition reference.

[0052] Case B3: the autonomous mobile device does not include the target partition reference in the environment image collected by the visual sensor, and the autonomous mobile device is located in the target region. Regardless of the angle of the visual sensor, most or all of the environment information collected by the visual sensor is from the target region. Therefore, after performing the occlusion, blurring or replacement processing on the environment information in the target region included in the environment image, the image will become substantially meaningless or completely meaningless. Based on this, the local environment map corresponding to the environment image can be obtained, that is, the local environment map under the current view angle of the autonomous mobile device is sent to the terminal device as the target image. The user cannot obtain detailed information in the target region through the local environment map, but at least can obtain some basic information in the region and view angle of the autonomous mobile device, so that the image obtained by the user has certain meaning. These basic information belong to the information included in the environment map, and generally do not involve user privacy.

[0053] In this embodiment, the implementation of obtaining the local environment map corresponding to the environment image is not limited, for example, the environment image can be directly recognized by image recognition, the environment information therein is recognized, and the local environment map matched in the environment information is obtained from the existing environment map. For another example, the autonomous mobile device can collect environment images while positioning the autonomous mobile device. The autonomous mobile device can obtain position information and pose information based on the position information, pose information and field of view angle of the visual sensor. The field of view range of the autonomous mobile device can be determined, and the local environment map corresponding to the environment image is obtained by using the field of view range.

[0054] Case B4: the autonomous mobile device does not include the target partition reference in the environment image collected by the visual sensor, and the autonomous mobile device is located outside the target region. Regardless of the angle of the visual sensor, the environment image collected by the visual sensor does not include the environment information in the target region, which means that it will not involve user privacy. Therefore, the environment image does not need to be filtered, and can be directly transmitted to the terminal device.

[0055] In another application scenario W2, the user wants to realize remote monitoring or intelligent management and control of a specific region (i.e. the target region) in the working environment through the autonomous mobile device, and does not want to consume too much traffic resource and storage resource on the terminal device. In view of this application requirement, the autonomous mobile device specifically performs the following operations to meet the application requirement of the user:

[0056] Case C1: the autonomous mobile device includes the target partition reference in the environment image collected by the vision sensor, and the autonomous mobile device is located in the target region, which means that the environment image contains the environment information in the target region, and may also contain the environment information in the non-target region. Since the target region is the region that the user wants to focus on, the environment information in the non-target region contained in the environment image can be occluded, blurred, or replaced to obtain a first target image. The first target image is transmitted to the terminal device bound to the autonomous mobile device. The non-target region refers to other regions in the field of view of the vision sensor except the target region.

[0057] Case C2: the autonomous mobile device includes the target partition reference in the environment image collected by the vision sensor, and the autonomous mobile device is located outside the target region, which means that the field of view of the vision sensor at least partially covers the target region, that is, the environment image includes the environment information in the target region, and further, the environment image also contains the environment information in the non-target region. In view of this, the environment information in the non-target region contained in the environment image can be occluded, blurred, or replaced to obtain a first target image. The first target image is transmitted to the terminal device bound to the autonomous mobile device. The non-target region refers to other regions in the field of view of the vision sensor except the target region.

[0058] Case C3: the autonomous mobile device does not include the target partition reference in the environment image collected by the vision sensor, and the autonomous mobile device is located outside the target region. Regardless of the angle of the vision sensor, the environment image collected by the vision sensor does not contain the environment information in the target region. Optionally, if the environment image is completely occluded, blurred, or replaced, the image will become meaningless. Based on this, a local environment map corresponding to the environment image can be obtained, that is, a local environment map under the current perspective of the autonomous mobile device is taken as a target image and sent to the terminal device. Although the user cannot obtain detailed information in the target region through the local environment map, at least some basic information under the current region and perspective of the autonomous mobile device can be obtained, so that the image obtained by the user has certain meaning. For the implementation of the autonomous mobile device obtaining the local environment map corresponding to the environment image, refer to the foregoing embodiments, which will not be described here. Alternatively, the environment image can also be discarded and not transmitted to the terminal device, thereby saving traffic resources.

[0059] Case C4: the autonomous mobile device does not include the target sub-area reference in the environment image collected by the visual sensor, and the autonomous mobile device is located in the target area. In this case, no matter which angle the visual sensor is directed to, most or all of the environment information collected by the visual sensor is from the target area, and the target area is the area that the user wants to focus on. In this case, the environment image can not be filtered, and the environment image can be directly transmitted to the terminal device bound to the autonomous mobile device.

[0060] In this embodiment, after the first target image or the second target image is generated by the above method, the first target image or the second target image can be transmitted to the terminal device bound to the autonomous mobile device. In this embodiment, the implementation of the autonomous mobile device providing the first or second target image to the terminal device is not limited. In an optional embodiment, the user is in the same environment as the autonomous mobile device, and the terminal device of the user can establish a connection relationship with the autonomous mobile device through Bluetooth, wireless, or other wireless technologies. The autonomous mobile device directly transmits the first or second target image to the terminal device. In another optional embodiment, the autonomous mobile device is not in the same environment as the terminal device, and the autonomous mobile device establishes a binding relationship with the terminal device. The autonomous mobile device can provide the first or second target image to the server, and the server provides the first or second target image to the terminal device based on the binding relationship between the terminal device and the autonomous mobile device.

[0061] In the embodiments of the present application, the autonomous mobile device can collect environment images and transmit them to the terminal device during operation. Before transmission to the terminal device, it can be determined whether the environment image contains a target sub-area reference. In the case where the environment image contains a target sub-area reference, the environment information containing the target area in the environment image is identified, and the environment image is filtered accordingly. The target image obtained by filtering is then provided to the terminal device. The transmission of the target image filtered to remove some environment information can reduce data transmission volume and / or protect user privacy and improve service quality while meeting the needs of remote monitoring or intelligent control.

[0062] Scenario-based embodiment 1:

[0063] In a home environment, at least a living room and a children's room are included. The autonomous mobile device is implemented as a companion robot. The user wants to remotely monitor the baby in the children's room through the companion robot and save network traffic. The user establishes a connection relationship with the companion robot through a terminal device (such as a mobile phone), and sets the focus area as the children's room on the terminal device.

[0064] The user can issue a command to the accompanying robot to accompany the child to the child's room when going out, and the accompanying robot locates itself in the living room after receiving the accompanying command. Further, the accompanying robot plans a travel path from the current location (living room) to the child's room and moves to the child's room along the path. During the movement, the accompanying robot collects environmental images while moving and determines whether the door of the child's room is in the environmental images. If the determination result is yes, the accompanying robot blurs the environmental information in the environmental images except for the child's room to reduce the data volume in the transmission process and improve the transmission speed. The accompanying robot transmits the locally blurred environmental images to the terminal device of the user, so that the user can remotely monitor the situation of the child's room through the locally blurred environmental images, such as determining whether the child has dangerous or abnormal behavior, and enhancing the user's experience.

[0065] After entering the child's room, the environmental images collected by the accompanying robot are all environmental information in the child's room, so the accompanying robot can not filter the environmental images but directly transmit the environmental images to the terminal device of the user for monitoring the situation of the child's room, such as determining whether the child has dangerous or abnormal behavior. It is explained herein that in this scenario, the accompanying robot can continuously collect environmental images to form a video stream and transmit the video stream to the terminal device of the user.

[0066] Scenario-based embodiment 2:

[0067] In a home environment, the self-moving device is implemented as a sweeping robot, and the user can remotely monitor the home environment during the execution of the sweeping task by the sweeping robot. The home environment of the user includes at least a bedroom, a living room, a bathroom, and a kitchen. The user establishes a binding relationship with the sweeping robot through a terminal device and sets the privacy area as the bedroom and the bathroom through the terminal device.

[0068] The sweeping robot can start to execute the sweeping task according to the sweeping instruction remotely issued by the user or the sweeping time of the set timed sweeping task. During the execution of the sweeping task by the sweeping robot, the sweeping robot collects environmental images around for positioning and navigation. During this process, when the user needs to view the environmental information under the perspective of the sweeping robot, the user can issue a remote monitoring command to the sweeping robot through the terminal device. The sweeping robot receives the remote monitoring command, locates the current position thereof, assumes that the current position of the sweeping robot is the bedroom, and determines that the bedroom is a privacy area. Then, the sweeping robot replaces the environmental images collected in the bedroom with a local environmental map under the current perspective thereof and transmits the local environmental map to the server. The server provides the local environmental map to the terminal device, and the user views the local environmental map of the bedroom under the perspective of the sweeping robot through the terminal device, as shown in FIG. 2. Figure 3a ​

[0069] After a period of time, the sweeping robot moves to the living room for cleaning, at this time, the sweeping robot is positioned to be currently located in the living room, and the living room belongs to a non-private area, so the field of view range of the camera on the sweeping robot is determined according to the attitude of the sweeping robot; it is found that the field of view range of the camera contains the door of the bedroom, which means that the field of view range of the camera can cover the bedroom area, as shown in Figure 3b In this case, although the sweeping robot is in the living room, it can collect environmental information in the bedroom through the door of the bedroom. In order to protect the user's privacy, the sweeping robot can identify that the door of the bedroom is contained in the environmental image, and use a whiteboard to shield the door in the environmental image, as shown in Figure 3c , so as to shield the environmental information in the bedroom and achieve the purpose of protecting the user's privacy. Further, the sweeping robot sends the environmental image shielded by the whiteboard to the server, and the server sends the environmental image to the terminal device bound with the sweeping robot, so that the user can monitor the home environment.

[0070] With the continuation of the cleaning task, the sweeping robot repeatedly performs the above-mentioned actions until the cleaning task is completed or a remote monitoring end instruction sent by the user is received, thereby meeting the user's remote monitoring needs while protecting the user's privacy.

[0071] Scenario embodiment 3:

[0072] In a home environment, the user can use the sweeping robot as a smart butler in the smart home Internet of Things, and intelligently manage and control the smart home devices in the home environment through the sweeping robot. The sweeping robot can perform a cleaning task in the home environment, or can perform a tour in the home environment, so that the user can intelligently manage and control the smart home devices in the home environment through the sweeping robot. During the execution of the cleaning task or the tour, the sweeping robot can collect environmental images in the home environment through the camera, and the user can observe the use state of the smart home devices through the environmental images collected by the sweeping robot, thereby realizing intelligent management and control of the smart home devices. In order to realize intelligent management and control of the smart home devices, the user binds the terminal device with the sweeping robot through the terminal device, and sets the privacy area as the bedroom and the bathroom through the terminal device.

[0073] The sweeping robot can start to move to the living room to perform the sweeping task according to a sweeping instruction remotely sent by the user or when the sweeping time of the set timing sweeping task arrives. During the sweeping task performed by the sweeping robot, the user sends an instruction to the sweeping robot to acquire the working state of the air conditioner, and then the sweeping robot, after receiving the instruction, locates the current position thereof as the living room, further determines the field of view range of the camera on the sweeping robot in combination with the current posture thereof, and judges whether the air conditioner arranged in the living room is located in the current field of view range. If the air conditioner is not located in the current field of view range, the sweeping robot can adjust the position and posture thereof until the air conditioner is located in the current field of view range. In the case that the air conditioner is located in the current field of view range, the sweeping robot further judges whether the door of the bedroom is contained in the current field of view range. If the door of the bedroom is contained in the current field of view range, it means that the environment image collected by the sweeping robot at the current position and posture contains the door of the bedroom, and it also means that the environment information in the bedroom is included in the environment image, and the sweeping robot can recognize the door contained in the environment image and shield the door in the environment image by using a whiteboard. Then, the environment image after shielding is provided to the server and sent to the terminal device by the server. The environment image after shielding contains the working state information of the air conditioner, such as the current gear, wind speed and current indoor temperature of the air conditioner, which can be used for the user to check the working state of the air conditioner.

[0074] Further, if the user thinks that the current indoor temperature is appropriate, the user can also send an instruction to the sweeping robot through the terminal device to control the air conditioner to be turned off, and the sweeping robot sends a turn-off instruction to the air conditioner to turn off the air conditioner. Or, if the user thinks that the current wind speed of the air conditioner is too high, the user can send an instruction to the sweeping robot through the terminal device to control the air conditioner to reduce the wind speed, and the sweeping robot sends an instruction to the air conditioner to reduce the wind speed to control the air conditioner to reduce the wind speed. Or, if the user thinks that the current gear of the air conditioner is too low, the user can also send an instruction to the sweeping robot through the terminal device to control the air conditioner to increase the gear, and the sweeping robot sends an instruction to the air conditioner to increase the gear to control the air conditioner to increase the gear, thereby realizing intelligent management and control of the air conditioner.

[0075] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can also be executed by different devices as the execution subject. For example, the execution subject of steps S101 to S104 can be device A; for another example, the execution subject of steps S101 to S103 can be device A, and the execution subject of step S104 can be device B; and the like.

[0076] In addition, in some of the processes described in the above embodiments and the accompanying drawings, a plurality of operations are included in a specific order, but it should be clear that these operations can be executed in the order in which they appear in this document or in parallel, and the serial numbers of the operations, such as S101, S102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel.

[0077] Figure 4 A structural schematic diagram of an autonomous mobile device is provided for an exemplary embodiment of the present application. As shown in the figure, the autonomous mobile device includes a device body 40, and the device body 40 is provided with a memory 44 and a processor 45. Figure 4 The memory 44 is used to store computer programs and can be configured to store various other data to support operations on the autonomous mobile device. Examples of these data include instructions for any application program or method operating on the autonomous mobile device, messages, pictures, videos, etc.

[0078] The memory 44 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0079] The processor 45 is coupled to the memory 44 and is used to execute the computer programs in the memory 44 for: during a job process, collecting an environment image using a visual sensor; in the case that the environment image contains a target partition reference, identifying the environment information in a target region contained in the environment image; according to the environment information in the target region, filtering the environment image to obtain a first target image; and transmitting the first target image to a terminal device bound to the autonomous mobile device, the target region being a functional region in a working environment divided by the target partition reference.

[0080] In an optional embodiment, the processor 45 is further configured to: in the case that the environment image does not contain the target partition reference, determine the positional relationship between the autonomous mobile device and the target region; if the positional relationship indicates that the autonomous mobile device is located within the target region, obtain a local environment map corresponding to the environment image as a second target image, and transmit the second target image to the terminal device.

[0081] In an optional embodiment, the processor 45 is further configured to: in the case that the environment image does not contain the target partition reference, determine the positional relationship between the autonomous mobile device and the target region; if the positional relationship indicates that the autonomous mobile device is located within the target region, obtain a local environment map corresponding to the environment image as a second target image, and transmit the second target image to the terminal device.

[0082] In an optional embodiment, the processor 45 is configured to, when obtaining the first target image, perform occlusion, blurring or replacement processing on the environmental information in the target region included in the environment image to obtain the first target image, or perform occlusion, blurring or replacement processing on the information other than the environmental information in the target region in the environment image to obtain the first target image.

[0083] In an optional embodiment, the processor 45 is further configured to: locate the position and pose of the autonomous mobile device according to the existing environment map and the environment image; determine the field of view range of the vision sensor according to the position and pose of the autonomous mobile device; and determine that the environment image contains the target partition reference if the target partition reference falls within the field of view range of the vision sensor.

[0084] Further, when determining whether the target partition reference falls within the field of view range of the vision sensor, the processor 45 is configured to: input the environment image into an image recognition model to detect the target partition reference, and acquire the coordinates of the target partition reference in the environment image as first coordinates if the environment image is detected to contain the target partition reference; convert the world coordinates of the known target partition reference to the sensor coordinate system to obtain second coordinates, and calculate third coordinates according to the first coordinates and the second coordinates; and determine that the target partition reference falls within the field of view range of the vision sensor if the third coordinates are located within the field of view range of the vision sensor.

[0085] In an optional embodiment, the processor 45 is further configured to: receive configuration information sent by a terminal device, the configuration information including identification information of a target region configured by a user, and determine identification information of a target partition reference according to the identification information of the target region; and the identification information of the target region or the target partition reference is semantic information or position information of the target region or the target partition reference in an environment map.

[0086] In an optional embodiment, the processor 45 is further configured to: identify the target region and the target partition reference in the working environment based on at least one of the set region features and the partition reference features, and mark the target region and the target partition reference in the environment map.

[0087] Further, as shown in Figure 4 the autonomous mobile device further includes a communication component 46, a display 47, a power supply component 48, an audio component 49 and other components. Figure 4 only some components are shown schematically in the drawings, and the autonomous mobile device does not necessarily include only the components shown in the drawings. It should be noted that Figure 4 the autonomous mobile device can include other components. Figure 4The components in the dashed box are optional components, rather than mandatory components, and can be determined according to the product form of the autonomous mobile device.

[0088] Correspondingly, the embodiment of the present application also provides a computer readable storage medium storing a computer program, which can implement each step performed by the autonomous mobile device in the information transmission method embodiment when executed.

[0089] The implementation form of the autonomous mobile device can be a sweeping robot, a purifier, a cleaning machine, etc. According to different implementation forms, the autonomous mobile device can also include other components or modules. In the following, the autonomous mobile device is taken as a sweeping robot for example, as shown in Figure 5 The sweeping robot 500 includes a mechanical body 501, one or more processors 502, one or more memories 503 storing computer instructions, and a communication component 504 on the mechanical body 501. The communication component 504 can be a Wifi module, an infrared module, or a Bluetooth module, etc.

[0090] In addition to the one or more processors 502, the communication component 504, and the one or more memories 503, the mechanical body 501 is also provided with some basic components of the sweeping robot 500, such as a ranging sensor (single-point laser sensor or multi-point laser sensor) 509 for upward ranging, a vision sensor 506, a power component 507, a driving component 508, etc. The vision sensor can be a camera, a camera, etc. Optionally, the driving component 508 can include a driving wheel, a driving motor, a universal wheel, etc. Further, as shown in Figure 5 The sweeping robot 500 can also include a cleaning component 505, which can include a cleaning motor, a cleaning brush, a dusting brush, a dust suction fan, etc. The basic components included in different sweeping robots 500 and the composition of the basic components will be different, and the embodiments of the present application are only some examples. It is worth noting that the components shown in the dashed box in Figure 5 are optional components, rather than necessary components.

[0091] It is worth noting that the one or more processors 502 and the one or more memories 503 can be arranged inside the mechanical body 501, or can be arranged on the surface of the mechanical body 501.

[0092] The mechanical body 501 is an execution mechanism on which the robot 500 relies to complete a task, and can execute the operation specified by the processor 502 in a determined environment. Among them, the mechanical body 501 to some extent embodies the appearance form of the sweeping robot 500. In this embodiment, the appearance form of the sweeping robot 500 is not limited, for example, it can be circular, elliptical, triangular, convex polygon, etc.

[0093] One or more memories 503, mainly used for storing computer instructions, which can be executed by one or more processors 502, so that one or more processors 502 can control robot 500 to perform corresponding tasks. In addition to storing computer instructions, one or more memories 503 can also be configured to store other various data to support operations on robot 500. Examples of these data include instructions for any application or method operating on robot 500, environment map, signal strength map, etc. of the environment / scene where robot 500 is located.

[0094] One or more processors 502, which can be regarded as the control system of robot 500, can be used to execute computer instructions stored in one or more memories 503 to control robot 500 to perform corresponding tasks. For example, one or more processors 502 can execute computer instructions stored in one or more memories 503 to implement the steps of the information transmission method in the foregoing method embodiments, so that the execution of the task can more and more meet the scene requirements and user's intention, improve the service quality of the robot, and improve the user experience.

[0095] In addition to the above operations, one or more processors 502 executing computer instructions stored in one or more memories 503 can also be used for: during the task, using a visual sensor to collect an environment image, identifying environment information in a target area contained in the environment image in the case that the environment image contains a target partition reference; filtering the environment image according to the environment information in the target area to obtain a first target image; and transmitting the first target image to a terminal device bound with the autonomous mobile device, the target area being a functional area in a working environment divided by the target partition reference. For details, please refer to the foregoing embodiments, which will not be repeated here.

[0096] The embodiments of the present application also provide an information transmission device, which can be applied to an autonomous mobile device, and the device comprises: an acquisition module, an identification module, a filtering module, and a transmission module. The acquisition module is used for collecting an environment image using a visual sensor on the autonomous mobile device during the task of the autonomous mobile device. The identification module is used for identifying environment information in a target area contained in the environment image in the case that the environment image contains a target partition reference. The filtering module is used for filtering the environment image according to the environment information in the target area to obtain a first target image. The transmission module is used for transmitting the first target image to a terminal device bound with the autonomous mobile device, the target area being a functional area in a working environment divided by the target partition reference.

[0097] In an optional embodiment, the apparatus further comprises a determining module and an obtaining module. The determining module is configured to determine the positional relationship between the autonomous mobile device and the target region when the target region reference is not contained in the environment image. The obtaining module is configured to obtain a local environment map corresponding to the environment image as a second target image when the positional relationship indicates that the autonomous mobile device is located within the target region. The transmitting module is further configured to transmit the second target image to the terminal device.

[0098] In an optional embodiment, the filtering module is specifically configured to perform occlusion, blurring or replacement processing on the environment information in the target region contained in the environment image to obtain the first target image, or perform occlusion, blurring or replacement processing on the information other than the environment information in the target region in the environment image to obtain the first target image.

[0099] In an optional embodiment, the apparatus further comprises a positioning module, a determining module and a judging module. The positioning module is configured to position the position and pose of the autonomous mobile device according to the existing environment map and the environment image. The determining module is configured to determine the field of view range of the visual sensor according to the position and pose of the autonomous mobile device. The judging module is configured to determine that the target region reference is contained in the environment image when the target region reference falls within the field of view range of the visual sensor.

[0100] Further optionally, the judging module is specifically configured to input the environment image into an image recognition model to detect the target region reference, and obtain the coordinates of the target region reference in the environment image as the first coordinates when it is detected that the environment image contains the target region reference. The world coordinates of the known target region reference are converted to the sensor coordinate system to obtain the second coordinates, and the third coordinates are calculated according to the first coordinates and the second coordinates. If the third coordinates are located within the field of view range of the visual sensor, it is determined that the target region reference falls within the field of view range of the visual sensor.

[0101] The information transmission apparatus provided in the embodiment can collect an environment image during the operation of the autonomous mobile device and transmit the environment image to a terminal device. Before being transmitted to the terminal device, it can be determined whether the environment image contains a target region reference, and when the environment image contains the target region reference, the environment information of the target region divided by the target region reference contained in the environment image can be identified, and the environment image can be filtered accordingly. Then, the target image obtained by filtering is provided to the terminal device. The target image filtered to remove some environment information can reduce the data transmission amount on the basis of meeting the demand for remote monitoring or intelligent control, and / or protect user privacy and improve service quality.

[0102] The above Figures 4-5The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0103] The above Figures 4-5 The display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0104] The above Figures 4-5 The power supply component provides power to the various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0105] The above Figures 4-5 The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0106] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0107] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0108] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0109] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0110] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0111] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory. The memory is an example of computer-readable media.

[0112] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0113] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0114] The above only describes the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. An information transmission method, suitable for an autonomous mobile device, characterized in that, The method comprises: During the operation, an environment image is collected by using a visual sensor; In the case that the environment image contains a target partition reference, environment information in a target region contained in the environment image is identified; According to the environment information in the target region, the environment image is information filtered to obtain a first target image; The first target image is transmitted to a terminal device bound with the autonomous mobile device, and the target region is a functional region in the operation environment divided by the target partition reference; In the case that the environment image does not contain the target partition reference, the environment image is transmitted according to the positional relationship between the autonomous mobile device and the target region and the demand of a user for the target region during remote monitoring or intelligent management.

2. The method of claim 1, wherein, Further comprising: In the case that the environment image does not contain the target partition reference, the positional relationship between the autonomous mobile device and the target region is determined; If the positional relationship indicates that the autonomous mobile device is located in the target region, a local environment map corresponding to the environment image is obtained as a second target image, and the second target image is transmitted to the terminal device.

3. The method of claim 1, wherein, According to the environment information in the target region, the environment image is information filtered to obtain a first target image, comprising: The environment information in the target region contained in the environment image is subjected to occlusion, blurring or replacement processing to obtain a first target image.

4. The method of claim 1, wherein, According to the environment information in the target region, the environment image is information filtered to obtain a target image, comprising: Information other than the environment information in the target region in the environment image is subjected to occlusion, blurring or replacement processing to obtain a first target image.

5. The method according to any one of claims 1 to 4, characterized in that, Further comprising: According to the existing environment map and the environment image, the position and pose of the autonomous mobile device are located; According to the position and pose of the autonomous mobile device, the field of view range of the visual sensor is determined; If the target partition reference falls within the field of view range of the visual sensor, it is determined that the environment image contains the target partition reference.

6. The method of claim 5, wherein, Determining whether the target partition reference falls within the field of view range of the visual sensor, comprising: The environment image is input into an image recognition model for detection of the target partition reference, and in the case that the environment image is detected to contain the target partition reference, the coordinates of the target partition reference in the environment image are obtained as first coordinates; The world coordinates of the known target partition reference are converted to a sensor coordinate system to obtain second coordinates, and the third coordinates are calculated according to the first coordinates and the second coordinates; If the third coordinates are located within the field of view range of the visual sensor, it is determined that the target partition reference falls within the field of view range of the visual sensor.

7. The method according to any one of claims 1 to 4, characterized in that, Further comprising: Configuration information sent by the terminal device is received, the configuration information comprising identification information of a target region configured by a user, and identification information of a target partition reference is determined according to the identification information of the target region; The identification information of the target region or the target sub-region reference is semantic information or position information of the target region or the target sub-region reference in the environment map.

8. The method according to any one of claims 1 to 4, characterized in that, Further comprising: Based on at least one of the set region features and the sub-region reference features, the target region and the target sub-region reference in the working environment are identified, and the target region and the target sub-region reference are marked in the environment map.

9. An autonomous mobile device, comprising: Comprise: The device body is provided with a memory and a processor; The memory is used for storing a computer program; The processor is coupled with the memory and is used for executing the computer program, so as to: In the working process, the environment image is collected by using the visual sensor; In the case that the target sub-region reference is contained in the environment image, the environment information in the target region contained in the environment image is identified; According to the environment information in the target region, the environment image is filtered to obtain a first target image; The first target image is transmitted to a terminal device bound with the autonomous mobile device, and the target region is a functional region in the working environment divided by the target sub-region reference; In the case that the target sub-region reference is not contained in the environment image, according to the position relationship between the autonomous mobile device and the target region, and combining the demand of the user in the remote monitoring or intelligent management and control process on the target region, the environment image is transmitted and processed.

10. A computer readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the processor realizes the steps in the method of any one of claims 1-8.

Images