Human-computer interaction method and electronic equipment

By using electronic devices to display surveillance images in intelligent service robot applications, users can use the global and real-time cameras to interact with surveillance images to issue instructions to the robot, solving the problem of optimization of human-computer interaction methods, achieving more efficient and accurate interaction, and improving the intelligence of the robot.

CN119883070APending Publication Date: 2025-04-25HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311389067.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-10-24
Publication Date
2025-04-25

AI Technical Summary

Technical Problem

In the process of applying intelligent service robots, how to optimize human-computer interaction methods, give full play to the existing underlying positioning and navigation methods, simplify the user interaction process, and enrich interaction functions to improve the intelligence of the robot.

Method used

By displaying surveillance images on electronic devices, users can use the global and real-time nature of the camera to interact with the surveillance images, issue instructions to the robot, instructing the robot to move to the target position to perform tasks. The method includes modeling the environment in which the robot is located, matching the camera and map, obtaining external parameters of the camera, and then converting the pixel coordinates in the monitoring image to the map coordinate system of the target environment.

Benefits of technology

This method enriches the interaction functions, improves the user experience, enables users to convey instructions more conveniently and accurately, optimizes the human-computer interaction method, and improves the intelligence of the robot.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119883070A_ABST
    Figure CN119883070A_ABST
Patent Text Reader

Abstract

A man-machine interaction method is applied to an electronic device with a display screen. The method comprises: in response to a viewing operation for a monitoring image of a target environment, displaying the monitoring image, at least one camera being configured in the target environment, and the monitoring image being acquired by the at least one camera; in response to a selection operation for a target area in the monitoring image, a first instruction is issued to a robot located in a target environment, and the first instruction is used for instructing the robot to move to a target position in the target environment to execute a task, the target position is obtained by converting pixel coordinates of the target area into a coordinate system of a map of the target environment. Thus, in the application process of the robot, a user can interact with the monitoring image by using the globality and the real-time performance of the camera, the interaction function is enriched, the user experience is improved, and instructions can be conveyed more conveniently and accurately.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of information technology (IT) technology, and in particular to a human-computer interaction method and electronic equipment. Background Art

[0002] In recent years, with the continuous updating and development of software and hardware technologies, intelligent service robots have achieved a huge leap forward, with their intelligence level constantly improving, showing a huge market demand, and have been widely used in education, medical care, logistics and other industries. The technologies involved in intelligent robots are relatively complex. Generally speaking, positioning navigation, human-computer interaction and environmental interaction are the three essential underlying technologies of intelligent service robots. With the rapid development of big data and artificial intelligence technology, the capabilities of intelligent robots in environmental perception and navigation and obstacle avoidance have been greatly enhanced. These perception and positioning technologies provide a huge guarantee for the improvement of the intelligence level of service robots. However, in the process of robot application, how to optimize the human-computer interaction method, give full play to the existing underlying positioning and navigation methods, simplify the user interaction process, and enrich the interactive functions, has important significance and more direct impact on the improvement of the robot's intelligence level. Therefore, in the process of robot application, how to optimize the human-computer interaction method is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a human-computer interaction method, electronic device, computer storage medium and computer product, which can optimize the human-computer interaction mode and enhance the user experience.

[0004] In a first aspect, the present application provides a human-computer interaction method, which is applied to an electronic device with a display screen. The method includes: in response to a viewing operation of a surveillance image of a target environment, displaying the surveillance image, wherein at least one camera is configured in the target environment, and the surveillance image is acquired by at least one camera; in response to a selection operation of a target area in the surveillance image, issuing a first instruction to a robot located in the target environment, wherein the first instruction is used to instruct the robot to move to a target position in the target environment to perform a task, wherein the target position is obtained by converting the pixel coordinates of the target area into the coordinate system of a map of the target environment.

[0005] In this way, during the robot application process, users can take advantage of the globality and real-time nature of the camera to interact with surveillance images, enrich the interactive functions, improve the user experience, and convey instructions more conveniently and accurately.

[0006] In a possible implementation, before displaying the monitoring image, the process further includes: modeling the environment in which the robot is located to obtain a map of the target environment; matching the camera with the map to obtain the external parameters of the camera under the map, wherein the external parameters are used to convert the pixel coordinates in the monitoring image to the coordinate system of the map. In this way, the pixel coordinates in the monitoring image can be subsequently converted to the coordinate system of the map of the target environment using the external parameters of the camera.

[0007] In a possible implementation, the camera is matched with the map to obtain the external parameters of the camera under the map, including: determining the external parameters based on the pixel coordinates of the target point in the monitoring image and the position coordinates of the target point in the map. In this way, the external parameters of the camera can be obtained, which is convenient for subsequent coordinate conversion.

[0008] In a possible implementation, the method further includes: in response to a marking operation on a target object in a monitoring image, recording a target position of the target object in a target environment, wherein the target position is obtained by converting pixel coordinates of the target object in the monitoring image into a coordinate system of a map of the target environment. In this way, the robot can be conveniently controlled to operate the target object or reach the location of the target object through the monitoring image, thereby improving the user experience.

[0009] In a possible implementation, after recording the target position of the target object in the target environment, the method further includes: detecting the position of the target object in real time in the monitoring image; updating the target position based on the position of the target object in the monitoring image when the position of the target object changes; and outputting prompt information when the target object disappears in the monitoring image, the prompt information being used to prompt that the target object disappears. In this way, the target object can be well managed.

[0010] In a possible implementation, the method further includes: in response to a binding operation on the target object and the target user, binding the target object to the target user. In this way, different target objects can be managed for different users, thereby achieving accurate management of the target objects.

[0011] In one possible implementation, the tasks performed by the robot include one or more of the following: a cleaning task, a pickup or delivery task, or a cargo counting task.

[0012] In a second aspect, the present application provides an electronic device comprising: a display screen; one or more processors; and a memory; wherein one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by one or more processors, enable the electronic device to perform the method described in the first aspect.

[0013] In a third aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on an electronic device, the electronic device executes the method described in the first aspect.

[0014] In a fourth aspect, the present application provides a computer program product, characterized in that when the computer program product is run on an electronic device, the electronic device executes the method described in the first aspect.

[0015] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic diagram of a robot application scenario provided by an embodiment of the present application;

[0017] Figure 2 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0018] Figure 3 It is a flowchart of a human-computer interaction method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0020] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0021] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0022] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0023] For example, Figure 1 FIG. 1 is a schematic diagram showing a robot application scenario provided by an embodiment of the present application. Figure 1 As shown, at least one camera 120 and a robot 130 are arranged in the house 110. Exemplarily, the camera 120 can also be called a camera or a surveillance camera. In this scenario, the environment in the house 110 can be modeled in advance to obtain a map of the house 110; and the surveillance image collected by the camera 120 is matched with the map of the house 110, so that the surveillance image collected by the camera 120 has spatial information. Exemplarily, the map of the house 110 can be a two-dimensional map or a three-dimensional map, which can be determined according to actual conditions and is not limited here. Finally, the user can access the surveillance image collected by the camera 120 through the electronic device 140 and interact with the surveillance image to issue instructions to the robot 130, for example, control the robot 130 to move from position A to position B, etc. In this way, during the robot application process, the user can use the globality and real-time nature of the camera to interact with the surveillance image, enrich the interactive function, improve the user experience, and convey instructions more conveniently and accurately. Exemplarily, the surveillance image can refer to a visual image or video stream captured by a camera.

[0024] It is understandable that the robot 130 in the embodiment of the present application can be an automated system that can perform tasks, manipulate objects or complete work, and usually has perception, decision-making and execution capabilities, and can perform various tasks without direct intervention. Exemplarily, the robot 130 may include: sensors, control systems and actuators, etc. Among them, sensors are sensors used to perceive the surrounding environment, such as cameras or lidars. The control system is a computer system responsible for processing sensor data, executing decisions and controlling the movement of the robot. The actuator is a mechanical component used to perform robot tasks, such as a motor or a wheel. Of course, the robot 130 may also include more or fewer components, such as a communication module for communicating with other devices, etc., which can be determined according to actual conditions and is not limited here.

[0025] The electronic device 140 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device and / or a smart city device, etc. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device 140.

[0026] For example, Figure 2 FIG. 1 is a schematic diagram showing the structure of an electronic device provided by an embodiment of the present application. Figure 2 As shown, the electronic device 140 may include: a processor 141 , a memory 142 , a display screen 143 , a sensor module 144 and a communication module 145 .

[0027] The processor 141 is the computing core and control core of the electronic device 140. The processor 141 may include one or more processing units. For example, the processor 141 may include one or more of an application processor (AP), a modem, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU). Among them, different processing units may be independent devices or integrated into one or more processors. In this embodiment, the NPU may be a neural network (NN) computing processor, which can quickly process input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn. The NPU can realize applications such as intelligent cognition of the electronic device 140, such as image recognition, face recognition, speech recognition, text understanding, text recognition, etc. In some embodiments, the processor 141 may be used to model the environment in which the robot 130 is located to obtain a map of the environment in which the robot 130 is located. For example, a laser scanner can be used to perform laser scanning on the environment where the robot 130 is located to obtain three-dimensional point cloud data in the environment; then, the processor 141 can process the point cloud data to create a map of the environment where the robot 130 is located. Of course, the modeling of the environment where the robot 130 is located can also be completed by other devices, which can be determined according to the actual situation and is not limited here. In some embodiments, the processor 141 can match the camera 120 with the modeled map to obtain the external parameters of the camera 120 under the map; of course, it can also be completed by other devices, which can be determined according to the actual situation and is not limited here. Exemplarily, the camera 120 can be matched with the modeled map by coordinate conversion to obtain the external parameters of the camera 120 in the map. For example: let Pc be the coordinate of P in the coordinate system centered on the camera 120, and Pw be the coordinate of P in the coordinate system related to the map. A rotation matrix R and a translation vector t can be used to transform Pc into Pw, where Pc=RPw+t. By deducing the formula, the external parameter T of the camera 120 under the map can be obtained.

[0028] The memory 142 may store a program, which may be executed by the processor 141, so that the processor 141 executes the method performed by the electronic device 140 provided in the embodiment of the present application. The memory 142 may also store data. The processor 141 may read the data stored in the memory 142. The memory 142 and the processor 141 may be provided separately. Optionally, the memory 142 may also be integrated in the processor 141. In some embodiments, the memory 142 may contain a map of the environment in which the robot 130 is located.

[0029] The display screen 143 can be used to display images, videos, etc. The display screen 143 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc.

[0030] The sensor module 144 may include a touch sensor. The touch sensor may be used to detect a touch operation on or near the electronic device 140. Exemplarily, the touch sensor may be implemented in a variety of ways such as resistive, capacitive, infrared, and surface acoustic waves. The touch sensor may be disposed on the display screen 143, and the touch sensor and the display screen 143 may form a touch screen, also known as a "touch screen". Of course, the touch sensor may also be disposed on the surface of the electronic device 140, at a different location from the display screen 143. In some embodiments, the sensor module 140 may also include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, an ambient light sensor, or a bone conduction sensor, etc.

[0031] The communication module 145 may include at least one of a mobile communication module and a wireless communication module. When the communication module 145 includes a mobile communication module, the communication module 145 may provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 140. For example, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), new radio (NR), etc.

[0032] The communication module 145 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The communication module 145 may receive electromagnetic waves by at least two antennas including an antenna, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modem for demodulation. The communication module 145 may also amplify the signal modulated by the modem, and convert it into electromagnetic waves for radiation through the antenna. In some embodiments, at least some functional modules of the communication module 145 may be arranged in the processor 141. In some embodiments, at least some functional modules of the communication module 145 may be arranged in the same device as at least some modules of the processor 141.

[0033] When the communication module 145 includes a wireless communication module, the communication module 145 can provide wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), nearfield communication technology (NFC), infrared technology (IR), etc., which are applied to the electronic device 140. The communication module 145 can be one or more devices integrating at least one communication processing module. The communication module 145 receives electromagnetic waves via an antenna, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 141. The communication module 145 can also receive the signal to be sent from the processor 141, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna.

[0034] Optionally, the electronic device 140 may also communicate with external devices through various interfaces, such as a universal serial bus (USB) interface, an Ethernet interface, a firewire interface, etc.

[0035] It is understandable that this embodiment Figure 2 The illustrated structure does not constitute a specific limitation on the electronic device 140. In other embodiments, the electronic device 140 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0036] Combine the following Figure 3 , the technical solution of the embodiment provided in this application is described in detail. For example, Figure 3 FIG. 1 is a flow chart of a human-computer interaction method provided in an embodiment of the present application. Figure 3 As shown, the human-computer interaction method may include the following steps:

[0037] S301. In response to a viewing operation of a surveillance image of a target environment, the surveillance image is displayed. At least one camera is configured in the target environment, and the surveillance image is acquired by the at least one camera.

[0038] In this embodiment, a user can select to view the surveillance image captured by a camera in a target environment on an electronic device. The target environment is configured with at least one camera. Then, the electronic device can display the surveillance image on its display screen. The surveillance image can be captured by at least one camera. Exemplarily, the target environment can be, but is not limited to, a home environment or an office environment.

[0039] S302. In response to a selection operation on a target area in the surveillance image, a first instruction is issued to a robot located in a target environment, wherein the first instruction is used to instruct the robot to move to a target position in the target environment to perform a task, wherein the target position is obtained by converting pixel coordinates of the target area into a coordinate system of a map of the target environment.

[0040] In this embodiment, after the electronic device displays the monitoring image, the user can select a target area in the monitoring image. Then, the electronic device can issue instructions to the robot located in the target environment to instruct the robot to move to the target position selected by the user to perform a task, such as a cleaning task, a pickup or delivery task, or a cargo inventory task, etc. The target position is obtained by converting the pixel coordinates of the target area into the map coordinate system of the target environment. During the conversion process, the map coordinate system of the target environment can be regarded as a world coordinate system. In this way, the coordinates of the pixel points in the monitoring image can be converted to the world coordinate system through the intrinsic parameters of the camera and its extrinsic parameters in the world coordinate system.

[0041] In this way, during the robot application process, users can take advantage of the globality and real-time nature of the camera to interact with surveillance images, enrich the interactive functions, improve the user experience, and convey instructions more conveniently and accurately.

[0042] In some embodiments, before S301, the environment in which the robot is located can be modeled first, and the camera can be matched with the map obtained by the modeling to obtain the external parameters of the camera under the modeled map (i.e., the rotation matrix and the translation vector). In this way, in S302, the position that the robot needs to reach can be calculated based on the area selected by the user on the monitoring image. Exemplarily, the external parameters can be determined based on the pixel coordinates of the target point in the monitoring image and the position coordinates of the target point in the map. For example, the pixel coordinates of the target point can be converted to a camera coordinate system centered on the camera, and the position of the target point in the camera coordinate system and the position of the target point on the map of the target environment can be processed through the conversion relationship between the camera coordinate system and the coordinate system of the map of the target environment, so that the external parameters of the camera under the map of the target environment can be obtained.

[0043] In some embodiments, the user can mark the target object on the monitoring image. Then, the electronic device can convert the pixel coordinates of the target object in the monitoring image to the coordinate system of the map of the target environment to obtain the target position of the target object in the target environment. Finally, the electronic device can record the target position. In this way, the user can easily control the robot to operate the target object or reach the location of the target object through the monitoring image. For example, when the robot is a restaurant service robot, the user can mark the location of each dining table in advance, and then the user can click on the area where a dining table is located on the monitoring image. At this time, the location of the area has already been recorded, so there is no need to calculate the position, and the robot can be directly issued instructions to the location of the dining table.

[0044] In addition, after recording the target position of the target object in the target environment, the position of the target object can also be detected in real time in the monitoring image. When the position of the target object changes, the target position can be updated based on the position of the target object in the monitoring image. When the target object disappears in the monitoring image, a prompt message can be output to inform the user that the target object disappears. The prompt message is used to prompt the disappearance of the target object. In this way, the target object can be well managed.

[0045] In addition, the user can also bind the target object with the target user, so that a connection between the target object and the target user can be established. After the electronic device obtains the binding operation issued by the user, it can bind the target object with the target user. After that, the electronic device can manage different target objects for different users, realizing accurate management of the target objects. For example, when the cup of user A disappears in the surveillance image, the electronic device can directly send a prompt message to user A without sending a prompt message to user B.

[0046] It is understandable that the order of execution of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, the various embodiments described above can be combined according to actual conditions, and the combined solutions are still within the scope of protection of the present application.

[0047] Based on the method in the above embodiment, the embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computing device cluster including at least one computing device, the computing device cluster executes the method described in the above embodiment. Exemplarily, the computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0048] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product including instructions. When the computer program product is run on a computing device cluster including at least one computing device, the computing device cluster executes the method in the above embodiment.

[0049] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0050] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0051] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0052] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A human-computer interaction method, characterized in that: Applied to an electronic device having a display screen, the method comprises: In response to a viewing operation for a surveillance image of a target environment, displaying the surveillance image, wherein at least one camera is configured in the target environment, and the surveillance image is acquired by the at least one camera; In response to a selection operation on a target area in the surveillance image, a first instruction is issued to a robot located in the target environment, wherein the first instruction is used to instruct the robot to move to a target position in the target environment to perform a task, wherein the target position is obtained by converting the pixel coordinates of the target area into a coordinate system of a map of the target environment.

2. The method according to claim 1, characterized in that Before displaying the monitoring image, the method further includes: Modeling the environment in which the robot is located to obtain a map of the target environment; The camera is matched with the map to obtain external parameters of the camera under the map, wherein the external parameters are used to convert pixel coordinates in the monitoring image into a coordinate system of the map.

3. The method according to claim 2, characterized in that The matching the camera with the map to obtain external parameters of the camera under the map includes: The extrinsic parameters are determined based on the pixel coordinates of the target point in the monitoring image and the position coordinates of the target point in the map.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: In response to a marking operation on a target object in the monitoring image, a target position of the target object in the target environment is recorded, wherein the target position is obtained by converting pixel coordinates of the target object in the monitoring image into a coordinate system of a map of the target environment.

5. The method according to claim 4, characterized in that After recording the target position of the target object in the target environment, the method further includes: Detecting the position of the target object in real time in the monitoring image; When the position of the target object changes, updating the target position based on the position of the target object in the monitoring image; When the target object disappears in the monitoring image, prompt information is output, where the prompt information is used to prompt that the target object disappears.

6. The method according to claim 4 or 5, characterized in that: Also includes: In response to a binding operation on the target object and the target user, the target object is bound to the target user.

7. The method according to any one of claims 1 to 6, characterized in that: The tasks include one or more of the following: Cleaning tasks, pickup or delivery tasks, or goods counting tasks.

8. An electronic device, characterized in that: include: Display screen; one or more processors; Memory; One or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the one or more processors, the electronic device executes the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product runs on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.