Robot control method and system based on artificial intelligence

Abstract

The invention discloses a robot control method and system based on artificial intelligence, and the method comprises the steps: extracting a first environment image of a target environment through different angle images, extracting first environment data of the target environment, constructing a three-dimensional grid image of the target environment according to the first environment data, and obtaining a three-dimensional grid image of the target environment; marking a concealable area in the three-dimensional grid image according to the first environment data, dividing a walkable area, planning a first path based on the three-dimensional grid image, obtaining a second environment image of the concealable area through the camera device, extracting a feature value of the concealable area based on the second environment image to identify a concealing point, and obtaining a second path based on the second environment image; and formulating a disinfection scheme for the hidden point, planning a second path based on the hidden point and the hidden area, and controlling the mechanical arm to position and disinfect the hidden point according to the disinfection scheme. According to the scheme, the problems that the disinfection effect of the disinfection robot is not ideal and the disinfectant fluid utilization rate is low are solved, so that the disinfection effect is improved, and the disinfectant fluid utilization rate is increased.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot control, and in particular relates to a robot control method and system based on artificial intelligence. Background Art

[0002] With the continuous advancement of disinfection robot technology, they have been able to perform disinfection tasks independently and enter potential infection or high-risk areas without human intervention. This automated solution effectively solves the problems of insufficient coverage and infection risks that may exist in traditional manual disinfection.

[0003] However, in closed environments such as hospitals and restaurants that require high disinfection requirements, existing disinfection robots mostly use the method of spraying disinfectant water above the air or horizontally, using atomization or fine water mist to spread the disinfectant. The spraying range is limited, and the disinfection effect in hidden areas is not ideal. In addition, when facing hidden areas with different characteristics, the spraying method of the disinfection robot cannot be customized according to the characteristics of the hidden area, reducing the utilization rate of disinfectant water. Therefore, it is necessary to design an artificial intelligence-based robot control method and system to improve the disinfection effect and increase the utilization rate of disinfectant water. Summary of the invention

[0004] The purpose of this application is to provide a robot control method and system based on artificial intelligence to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, this application provides the following technical solutions: A robot control method based on artificial intelligence comprises the following steps: Step S1: The disinfection robot controls the pan / tilt to move the camera up and down or rotate to perform an all-round scan of the target environment to obtain images at different angles; Step S2: extracting a first environment image of the target environment according to the captured image, and extracting first environment data of the target environment based on the first environment image; constructing a three-dimensional grid model of the target environment according to the first environment data, and identifying a concealable area in the three-dimensional grid image according to the first environment data, and dividing a walkable area; Step S3: controlling the disinfection robot to execute the first path, and in the process, obtaining a second environment image of the concealable area by controlling the camera; Step S4: extracting feature values ​​of the concealable area based on the second environment image, identifying hidden points in the three-dimensional grid image according to the feature values, uploading coordinate data of the hidden points to the robotic arm analysis module, and formulating a disinfection plan for the hidden points; Step S5: Plan a second path based on the hidden points and the concealable areas, and control the disinfection robot to execute. When executing the second path, control the robotic arm to locate and disinfect the hidden points according to the formulated disinfection plan.

[0006] According to the above technical solution, step S2 specifically includes: Step S2.1: The ultrasonic sensor arranged on the disinfection robot collects the position and distance information of the human body to determine whether someone has entered the target environment. When someone enters the target environment, the person's position is obtained through triangulation, and the person is not marked as a hidden area. When no one enters the target environment, no processing is performed.

[0007] Step S2.2: For each image at different angles, feature points are extracted by SIFT algorithm, and a high-dimensional feature descriptor is generated for each feature point; the feature descriptor includes the local shape and texture information of the area around the feature point, and the Euclidean distance between different feature descriptors is calculated and matched according to the different feature descriptors of each feature point in each image at different angles, so as to obtain a matching pair with the smallest Euclidean distance, and establish a corresponding relationship between images at different angles through the matching pairs; the first environmental data in the target environment is extracted, and the position change of the feature point between each image at different angles is identified, and when the position change exceeds the set threshold, it is determined to be a dynamic object; the first environmental data includes the structure of the target environment, the position and features of static objects, and the position and state of dynamic objects.

[0008] Step S2.3: Construct a three-dimensional point cloud in the target environment according to the first environmental data through a three-dimensional reconstruction algorithm, grid the grid point cloud to construct a three-dimensional grid image, extract the grid point cloud of static objects in the first environmental data as concealable areas, and mark the concealable areas; use the walkable area of ​​the disinfection robot in the target environment as blank coordinates in the three-dimensional grid image, and perform self-sensing and avoidance of dynamic objects during the disinfection process, wherein the static objects are objects in the first environmental data excluding humans and dynamic objects.

[0009] According to the above technical solution, step S3 specifically includes: Step S3.1: The first path is planned according to the walkable area through the A* algorithm, and all walkable areas in the target environment can be disinfected by controlling the disinfection robot to execute the first path.

[0010] Step S3.2: When the disinfection robot executes the first path for disinfection, it obtains a second environment image of each concealable area through a camera according to the three-dimensional coordinates of the concealable area in the three-dimensional grid image.

[0011] Step S3.3: Update the concealable areas in the three-dimensional grid image in real time according to the second environmental image, mark the updated concealable areas in green, and mark the unupdated concealable areas in red; wherein, when acquiring the second environmental image, if a dynamic object blocks the concealable area and the second environmental image cannot be acquired, the concealable area is marked in red and numbered.

[0012] According to the above technical solution, step S4 specifically includes: Step S4.1: After graying the second environment image, calculate the brightness value of each pixel in the second environment image, compare the brightness value with a set brightness threshold, and mark the pixels below the brightness threshold in yellow; Step S4.2: Analyze the binarization threshold of the second environment image by the Otsu law, binarize the second environment image according to the binarization threshold, and extract the linear contour of the binarized second environment image; calculate the density of the yellow mark in each linear contour area, and when the density is greater than the density threshold, it is determined to be a hidden point, and the coordinate data of the hidden point is obtained; based on the hidden point linear contour, the coordinate data of the hidden point area is obtained, and the coordinate data is uploaded to the robot arm control terminal; Step S4.3: When the robot control terminal performs the hidden point disinfection task, it transmits infrared rays to the hidden point through the infrared transmitter according to the coordinate data, and the infrared receiver receives the infrared signal reflected from the hidden point and records the time difference between the infrared emission and reception; the depth data of the hidden point is obtained by multiplying the light propagation speed and the time difference, that is, the distance that the disinfection robot needs to spray the hidden point; then a disinfection plan is formulated for the hidden point based on the coordinate data and the depth data, and the disinfection plan includes: the optimal spray column thickness and the optimal spray force of the robot arm.

[0013] According to the above technical solution, in step S4.3, a disinfection plan is formulated for the hidden point according to the coordinate data and the depth data, specifically including: Step S4.3.1: Locate the hidden point according to the coordinate data, construct the shape of the hidden point according to the coordinate information of each hidden point, and calculate the optimal spray column thickness according to the area of ​​the shape. The calculation formula is as follows:

[0014] In the above formula, represents the optimum injection column diameter, represents the adjustment factor used to adjust the proportional relationship between the area and the diameter of the spray column, A represents the area of ​​the hidden point, T represents the desired injection time, v represents the injection speed, Indicates the maximum diameter limit of the spray column.

[0015] Step S4.3.2: Analyze the optimal spray force based on the depth data, and the calculation formula is:

[0016] In the above formula, Indicates the optimum spray intensity. m The adjustment coefficient that indicates the proportional relationship between the depth and the spraying force. d Represents depth data, An index that indicates the effect of depth on jet strength. An index that represents the effect of time on the jet intensity. Indicates the basic spray force.

[0017] Step S4.3.3: The optimal spray column diameter and optimal spray force results calculated in steps S4.3.1 and S4.3.23 are transmitted to the robot control terminal, which is called when the robot control terminal controls the robot.

[0018] According to the above technical solution, step S5 specifically includes: Step S5.1: using the A* algorithm to plan the second path based on the hidden points and the hidden areas marked in red; Step S5.2: Control the disinfection robot to perform the disinfection task according to the second path. When the disinfection robot passes through a hidden point, the robotic arm is controlled to disinfect the hidden point according to the corresponding disinfection plan.

[0019] When passing through a hidden area marked in red, identify whether the hidden area has a number. When the number is identified, follow the analysis steps of the hidden area marked in green to identify the hidden point and formulate a disinfection plan, and send the analysis results to the robot arm control terminal, which will execute the disinfection plan.

[0020] When the number is not recognized, the hidden area is an inaccessible area, and the boundary image of the hidden area is obtained. According to the three-dimensional grid image, multi-angle images of the directly adjacent areas of the hidden area are obtained, and the brightness value of each pixel in the boundary image is calculated. Pixels below the brightness threshold are marked with yellow, and the linear contour of the boundary image is extracted by Otsu's law. The density of the yellow mark in each linear contour area is calculated. When the density is greater than the density threshold, it is determined to be a hidden point, and the coordinate data of the hidden point is obtained. According to the change of the relative position of the hidden point in the multi-angle image, the three-dimensional direction of the hidden point is obtained, and the size of the directly adjacent area corresponding to the three-dimensional direction is identified. The size is determined as the depth data of the hidden point, and the optimal spray column diameter and the optimal spray intensity are obtained according to the coordinate data and the depth data.

[0021] A robot control system based on artificial intelligence includes a target environment image acquisition module, a three-dimensional grid image construction module, a first path disinfection module, a robotic arm analysis module and a robotic arm control terminal.

[0022] The target environment image acquisition module is used for the disinfection robot to perform an all-round scan of the target environment by controlling the camera to move up and down and rotate to obtain images at different angles, and to extract a first environment image of the target environment based on the images at different angles.

[0023] The three-dimensional grid image construction module is used to extract first environmental data of the target environment based on the first environmental image, and construct a three-dimensional grid image of the target environment according to the first environmental data, identify the concealable area in the three-dimensional grid image according to the first environmental data, and divide the walkable area.

[0024] The first path disinfection module is used to plan the first path based on the three-dimensional grid image, and to obtain a second environmental image of the concealable area through a camera device while controlling the disinfection robot to execute the first path.

[0025] The robotic arm analysis module is used to extract characteristic values ​​of concealable areas based on the second environment image, identify hidden points in the three-dimensional grid image according to the characteristic values, upload the coordinate data of the hidden points to the robotic arm analysis terminal, and formulate a disinfection plan for the hidden points.

[0026] The robotic arm control terminal is used to plan a second path based on the hidden point and the concealable area. When executing the second path, the robotic arm control terminal controls the robotic arm to locate and disinfect the hidden point according to the disinfection plan.

[0027] The robotic arm analysis module includes a hidden point feature recognition module and a disinfection plan formulation module.

[0028] Among them, the hidden point feature recognition module is used to calculate the brightness value of each pixel in the second environment image after the grayscale processing of the second environment image, and compare the brightness value with the set brightness threshold, mark the pixels below the brightness threshold with yellow, analyze the binarization threshold of the second environment image through the Otsu law, binarize the second environment image according to the binarization threshold, extract the linear contour of the binarized second environment image, calculate the density of the yellow mark in each linear contour area, and when the density is greater than the density threshold, it is judged as the hidden point, and the coordinate data of the hidden point is obtained.

[0029] The disinfection plan formulation module is used for the robot arm control terminal to transmit infrared rays to the hidden point through the infrared transmitter according to the coordinate data when performing the disinfection of the hidden point, and the infrared receiver receives the infrared signal reflected from the hidden point, records the time difference between the infrared emission and the reception, and obtains the depth data of the hidden point by multiplying the light propagation speed and the time difference, that is, the distance that the disinfection robot needs to spray the hidden point; formulates a disinfection plan for the hidden point according to the coordinate data and the depth data; the disinfection plan includes: the optimal spray column thickness and the optimal spray force of the robot arm.

[0030] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the above-mentioned robot control method based on artificial intelligence.

[0031] A computer-readable storage medium is used to store a computer program. When the computer program is run on a computer, the computer executes the above-mentioned robot control method based on artificial intelligence.

[0032] Compared with the prior art, the present invention has the following beneficial effects: The present invention adopts a combination of a camera device and an ultrasonic sensor, which can perceive the target environment in real time, and accurately identify the concealable area and the walkable area, so as to plan the optimal disinfection path, use a three-dimensional reconstruction algorithm to construct a three-dimensional grid image of the target environment, and update the three-dimensional grid image according to the second environment image obtained in real time, so as to realize dynamic perception and adaptation to the environment, plan the path through the A* algorithm, and the mechanical arm accurately locates and disinfects according to the disinfection scheme, and formulates the disinfection scheme based on the identification of the hidden point through the analysis of its characteristic value, wherein the disinfection scheme is the optimal spray column thickness and the optimal spray force. The solution described in this application solves the problems of unsatisfactory disinfection effect of the disinfection robot and low utilization rate of disinfectant water, thereby improving the disinfection effect and increasing the utilization rate of disinfectant water. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of the robot control method based on artificial intelligence described in the present invention.

[0034] Figure 2 It is a schematic diagram of the robot control system based on artificial intelligence of the present invention.

[0035] Figure 3 is a schematic diagram of the electronic device of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Embodiment 1:

[0038] This embodiment can be applied in scenarios with high disinfection requirements, and is implemented by a robot control method and system based on artificial intelligence provided by this embodiment. Figure 1 As shown, the method specifically comprises the following steps: Step S1: The disinfection robot performs an all-round scan of the target environment by controlling the camera device to move up and down and rotate to obtain images at different angles, and extracts a first environmental image of the target environment based on the images at different angles. Specifically, the camera device is installed on a liftable pan-tilt platform, and sends instructions to the liftable platform to make the platform move up and down and rotate according to a preset trajectory or algorithm. As the platform moves and rotates, the camera device will also adjust its shooting angle and position accordingly, thereby ensuring that every corner and detail of the target environment can be captured.

[0039] Step S2: extracting first environmental data of the target environment based on the first environmental image, constructing a three-dimensional grid image of the target environment according to the first environmental data, identifying concealable areas in the three-dimensional grid image according to the first environmental data, and dividing walkable areas. Specifically including: Step S2.1: The ultrasonic sensor arranged on the disinfection robot collects the position and distance information of the human body to determine whether someone has entered the target environment. When someone enters the target environment, the person's position is obtained through triangulation, and the person is not marked as a hidden area. When no one enters the target environment, no processing is performed.

[0040] Step S2.2: For each image at different angles, feature points are extracted by SIFT algorithm, and a high-dimensional feature descriptor is generated for each feature point; the feature descriptor includes the local shape and texture information of the area around the feature point, and the Euclidean distance between different feature descriptors is calculated and matched according to the different feature descriptors of each feature point in each image at different angles, so as to obtain a matching pair with the smallest Euclidean distance, and establish a corresponding relationship between images at different angles through the matching pairs; the first environmental data in the target environment is extracted, and the position change of the feature point between each image at different angles is identified, and when the position change exceeds the set threshold, it is determined to be a dynamic object; the first environmental data includes the structure of the target environment, the position and features of static objects, and the position and state of dynamic objects.

[0041] Step S2.3: Construct a three-dimensional point cloud in the target environment according to the first environmental data through a three-dimensional reconstruction algorithm, grid the grid point cloud to construct a three-dimensional grid image, extract the grid point cloud of static objects in the first environmental data as concealable areas, and mark the concealable areas; use the walkable area of ​​the disinfection robot in the target environment as blank coordinates in the three-dimensional grid image, and perform self-sensing and avoidance of dynamic objects during the disinfection process, wherein the static objects are objects in the first environmental data excluding humans and dynamic objects.

[0042] Step S3: planning a first path based on the three-dimensional grid image, and controlling the disinfection robot to execute the first path, and obtaining a second environment image of the concealable area through a camera device. Specifically including: Step S3.1: The first path is planned according to the walkable area through the A* algorithm, and all walkable areas in the target environment can be disinfected by controlling the disinfection robot to execute the first path.

[0043] Step S3.2: When the disinfection robot executes the first path for disinfection, it obtains a second environment image of each concealable area through a camera according to the three-dimensional coordinates of the concealable area in the three-dimensional grid image.

[0044] Step S3.3: Update the concealable area in the three-dimensional grid image in real time according to the second environment image, mark the updated concealable area in green, and mark the unupdated concealable area in red; wherein, when acquiring the second environment image, if the second environment image of the concealable area cannot be acquired due to occlusion by a dynamic object, mark the concealable area in red and add a number. The number can be any form of number as long as it can be recognized, such as numbers, letters, etc., mainly to distinguish the concealable area that has not acquired the second environment image due to dynamic object factors.

[0045] Step S4: Extract the characteristic value of the concealable area based on the second environment image, identify the hidden point in the three-dimensional grid image according to the characteristic value, upload the coordinate data of the hidden point to the robot arm analysis terminal, and formulate a disinfection plan for the hidden point. Specifically including: Step S4.1: After graying the second environment image, the brightness value of each pixel in the second environment image is calculated, the brightness value is compared with a set brightness threshold, and pixels below the brightness threshold are marked in yellow.

[0046] Step S4.2: Analyze the binarization threshold of the second environment image through the Otsu law, binarize the second environment image according to the binarization threshold, and extract the linear contour of the binarized second environment image; calculate the density of the yellow mark in each linear contour area, and when the density is greater than the density threshold, it is determined to be a hidden point, and the coordinate data of the hidden point is obtained; based on the linear contour of the hidden point, the coordinate data of the hidden point area is obtained, and the coordinate data is uploaded to the robot arm control terminal.

[0047] Step S4.3: When the robot control terminal performs the hidden point disinfection task, it transmits infrared rays to the hidden point through the infrared transmitter according to the coordinate data, and the infrared receiver receives the infrared signal reflected from the hidden point and records the time difference between the infrared emission and reception; the depth data of the hidden point is obtained by multiplying the light propagation speed and the time difference, that is, the distance that the disinfection robot needs to spray the hidden point; then, a disinfection plan is formulated for the hidden point according to the coordinate data and the depth data, and the disinfection plan includes: the optimal spray column thickness and the optimal spray force of the robot arm. Formulating the disinfection plan specifically includes the following steps: Step S4.3.1: Locate the hidden point according to the coordinate data, construct the shape of the hidden point according to the coordinate information of each hidden point, and calculate the optimal spray column thickness according to the area of ​​the shape. The calculation formula is as follows:

[0048] In the above formula, represents the optimum injection column diameter, represents the adjustment factor used to adjust the proportional relationship between the area and the diameter of the spray column, A represents the area of ​​the hidden point, T represents the desired injection time, v represents the injection speed, Indicates the maximum diameter limit of the spray column.

[0049] Step S4.3.2: Analyze the optimal spray force based on the depth data, and the calculation formula is:

[0050] In the above formula, Indicates the optimum spray intensity. m The adjustment coefficient that indicates the proportional relationship between the depth and the spraying force. d Represents depth data, An index that indicates the effect of depth on jet strength. An index that represents the effect of time on the jet intensity. Indicates the basic spray force.

[0051] Step S4.3.3: The optimal spray column diameter and optimal spray force results calculated in steps S4.3.1 and S4.3.23 are transmitted to the robot control terminal, which is called when the robot control terminal controls the robot.

[0052] Step S5: planning a second path based on the hidden point and the concealable area, and when executing the second path, the robot arm control terminal controls the robot arm to locate and disinfect the hidden point according to the disinfection scheme. Specifically including: Step S5.1: Use the A* algorithm to plan the second path based on the hidden points and the hidden areas marked in red.

[0053] Step S5.2: Control the disinfection robot to perform the disinfection task according to the second path. When the disinfection robot passes through a hidden point, the robotic arm is controlled to disinfect the hidden point according to the corresponding disinfection plan.

[0054] When passing through a hidden area marked in red, identify whether the hidden area has a number. When the number is identified, follow the analysis steps of the hidden area marked in green to identify the hidden point and formulate a disinfection plan, and send the analysis results to the robot arm control terminal, which will execute the disinfection plan.

[0055] When the number is not recognized, the hidden area is an inaccessible area, and the boundary image of the hidden area is obtained. According to the three-dimensional grid image, multi-angle images of the directly adjacent areas of the hidden area are obtained, and the brightness value of each pixel in the boundary image is calculated. Pixels below the brightness threshold are marked with yellow, and the linear contour of the boundary image is extracted by Otsu's law. The density of the yellow mark in each linear contour area is calculated. When the density is greater than the density threshold, it is determined to be a hidden point, and the coordinate data of the hidden point is obtained. According to the change of the relative position of the hidden point in the multi-angle image, the three-dimensional direction of the hidden point is obtained, and the size of the directly adjacent area corresponding to the three-dimensional direction is identified. The size is determined as the depth data of the hidden point, and the optimal spray column diameter and the optimal spray intensity are obtained according to the coordinate data and the depth data.

[0056] Embodiment 2: In this embodiment, a robot control method and system based on artificial intelligence, Figure 2 As shown, Figure 2 As shown, it includes a target environment image acquisition module, a three-dimensional grid image construction module, a first path disinfection module, a robotic arm analysis module and a robotic arm control terminal.

[0057] The target environment image acquisition module is used for the disinfection robot to perform an all-round scan of the target environment by controlling the camera to move up and down and rotate to obtain images at different angles, and to extract a first environment image of the target environment based on the images at different angles.

[0058] The three-dimensional grid image construction module is used to extract first environmental data of the target environment based on the first environmental image, and construct a three-dimensional grid image of the target environment according to the first environmental data, identify the concealable area in the three-dimensional grid image according to the first environmental data, and divide the walkable area.

[0059] The first path disinfection module is used to plan the first path based on the three-dimensional grid image, and to obtain a second environmental image of the concealable area through a camera device while controlling the disinfection robot to execute the first path.

[0060] The robotic arm analysis module is used to extract characteristic values ​​of concealable areas based on the second environment image, identify hidden points in the three-dimensional grid image according to the characteristic values, upload the coordinate data of the hidden points to the robotic arm analysis terminal, and formulate a disinfection plan for the hidden points.

[0061] The robotic arm control terminal is used to plan a second path based on the hidden point and the concealable area. When executing the second path, the robotic arm control terminal controls the robotic arm to locate and disinfect the hidden point according to the disinfection plan.

[0062] In this embodiment, the three-dimensional grid image construction module includes a human body perception module, a first environment data extraction module and a region identification module.

[0063] The human body sensing module senses the position and distance of the human body through an ultrasonic sensor to determine whether a person has entered the target environment. When a person enters the target environment, the person's position is obtained through triangulation, and the person is not marked as a concealed area. When no one enters the target environment, no processing is performed.

[0064] The first environment data extraction module is used to extract feature points in each image at different angles through the SIFT algorithm, generate a high-dimensional feature descriptor for each feature point, the feature descriptor includes the local shape and texture information of the area around the feature point, calculate the Euclidean distance between the feature descriptors according to the different feature descriptors of each feature point in each image at different angles, obtain the matching pair with the smallest Euclidean distance, establish the corresponding relationship between the images at different angles through the matching pair, extract the first environment data in the target environment, identify the position change of the feature point between each image at different angles exceeds a set threshold value and judge it as a dynamic object, the first environment data is the structure of the target environment, the position and characteristics of static objects, and the position and state of dynamic objects; The area identification module is used to construct a three-dimensional point cloud in the target environment according to the first environmental data through a three-dimensional reconstruction algorithm, grid the grid point cloud to construct the three-dimensional grid image, extract the grid point cloud of the static objects in the first environmental data as the concealable area, and identify the concealable area. The walkable area of ​​the disinfection robot in the target environment is the blank coordinates in the three-dimensional grid image. During the disinfection process, the dynamic objects are sensed and avoided by the robot itself, wherein the static objects are obtained by excluding humans and the dynamic objects in the first environmental data.

[0065] In this embodiment, the first path disinfection module includes a path planning module, a second environment image acquisition module and a hidden area update module.

[0066] The path planning module plans a first path according to the walkable area through the A* algorithm, and the first path can be a path for disinfecting all the walkable areas in the target environment.

[0067] The second environment image acquisition module is used to acquire the second environment image of each concealable area through a camera device according to the three-dimensional coordinates of the concealable area in the three-dimensional grid image when the disinfection robot performs the first path disinfection.

[0068] The hidden area updating module is used for real-time updating of the hidden areas in the three-dimensional grid image according to the second environment image, marking the updated hidden areas in green and marking the unupdated hidden areas in red.

[0069] In this embodiment, the robotic arm analysis module includes a hidden point feature recognition module and a disinfection plan formulation module.

[0070] The hidden point feature recognition module is used to calculate the brightness value of each pixel in the second environment image after the second environment image is grayed, compare the brightness value with a set brightness threshold, mark the pixels below the brightness threshold in yellow, analyze the binarization threshold of the second environment image through the Otsu law, binarize the second environment image according to the binarization threshold, extract the linear contour of the binarized second environment image, calculate the density of the yellow mark in each linear contour area, and when the density is greater than the density threshold, it is determined to be a hidden point, and the coordinate data of the hidden point is obtained.

[0071] The disinfection program formulation module is used for the robot control terminal to transmit infrared rays to the hidden point according to the coordinate data through the infrared transmitter when performing the hidden point disinfection, and the infrared receiver receives the infrared signal reflected from the hidden point, records the time difference between the infrared emission and reception, and obtains the depth data of the hidden point by multiplying the light propagation speed and the time difference, that is, the distance that the disinfection robot needs to spray the hidden point, and formulates a disinfection program for the hidden point according to the coordinate data and the depth data. The disinfection program includes: the optimal spray column thickness and the optimal spray force of the robot.

[0072] Embodiment three: An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the above-mentioned robot control method based on artificial intelligence.

[0073] In this embodiment, the electronic device may be a server, and the structure of the electronic device may be as follows: Figure 3 As shown, it includes a memory 2001 , a communication module 2003 and one or more processors 2002 .

[0074] The memory 2001 is used to store computer programs executed by the processor 2002. The memory 2001 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0075] The memory 2001 may be a volatile memory, such as a random-access memory (RAM); the memory 2001 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 2001 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2001 may be a combination of the above memories.

[0076] The processor 2002 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 2002 is configured to implement the above-mentioned audio data processing method when calling the computer program stored in the memory 2001 .

[0077] The communication module 2003 is used to communicate with terminal devices and other servers.

[0078] The specific connection medium between the memory 2001, the communication module 2003 and the processor 2002 is not limited in the embodiment of the present application. Figure 3 In the embodiment, the memory 2001 and the processor 2002 are connected via a bus 2004. The bus 2004 is Figure 3 The arrows in the figure are used to describe the connection between other components, which is only for illustrative purposes and is not intended to be limiting. Bus 2004 can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 3 The diagram only uses one arrow, but does not describe only one bus or one type of bus.

[0079] Embodiment 4: A computer-readable storage medium is used to store a computer program. When the computer program is run on a computer, the computer executes the above-mentioned robot control method based on artificial intelligence.

[0080] The computer readable storage medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0081] Based on the same application concept as the above method embodiment, this embodiment also provides a computer program product, which includes a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the control method according to various exemplary embodiments of the present application described above in this specification. The program product can adopt any combination of one or more readable media. These computer program commands can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the process Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Claims

1. A robot control method based on artificial intelligence, characterized in that: The steps include: Step S1: The disinfection robot controls the pan / tilt platform to enable the camera to move up and down or rotate, and performs an all-round scan of the target environment to obtain images at different angles; Step S2: extracting a first environment image of the target environment according to the captured image, and extracting first environment data of the target environment based on the first environment image; constructing a three-dimensional grid model of the target environment according to the first environment data, and identifying a concealable area in the three-dimensional grid image according to the first environment data, and dividing a walkable area; Step S3: controlling the disinfection robot to execute the first path, and in the process, obtaining a second environment image of the concealable area by controlling the camera; Step S4: extracting feature values ​​of the concealable area based on the second environment image, identifying hidden points in the three-dimensional grid image according to the feature values, uploading coordinate data of the hidden points to the robotic arm analysis module, and formulating a disinfection plan for the hidden points; Step S5: Plan a second path based on the hidden points and the concealable areas, and control the disinfection robot to execute. When executing the second path, control the robotic arm to locate and disinfect the hidden points according to the formulated disinfection plan.

2. The robot control method based on artificial intelligence according to claim 1, characterized in that: The step S2 specifically includes: Step S2.1: The ultrasonic sensor arranged on the disinfection robot collects the position and distance information of the human body to determine whether a person has entered the target environment. When a person has entered the target environment, the person's position is obtained through triangulation, and the person is not marked as a concealable area. When no one has entered the target environment, no processing is performed; Step S2.2: Extract feature points in each image at different angles by SIFT algorithm, and generate a high-dimensional feature descriptor for each feature point; the feature descriptor includes the local shape and texture information of the area around the feature point, and calculate the Euclidean distance between different feature descriptors according to the different feature descriptors of each feature point in each image at different angles, and perform matching to obtain a matching pair with the smallest Euclidean distance, and establish a corresponding relationship between images at different angles through the matching pairs; extract the first environment data in the target environment, identify the position change of the feature point between each image at different angles, and determine it as a dynamic object when the position change exceeds a set threshold; The first environmental data includes the structure of the target environment, the location and characteristics of static objects, and the location and status of dynamic objects; Step S2.3: Construct a three-dimensional point cloud in the target environment according to the first environmental data through a three-dimensional reconstruction algorithm, grid the grid point cloud to construct a three-dimensional grid image, extract the grid point cloud of static objects in the first environmental data as concealable areas, and mark the concealable areas; use the walkable area of ​​the disinfection robot in the target environment as blank coordinates in the three-dimensional grid image, and perform self-sensing and avoidance of dynamic objects during the disinfection process, wherein the static objects are objects in the first environmental data excluding humans and dynamic objects.

3. The robot control method based on artificial intelligence according to claim 2, characterized in that: The step S3 specifically includes: Step S3.1: planning a first path according to the walkable area by using the A* algorithm, and controlling the disinfection robot to execute the first path to disinfect all the walkable areas in the target environment; Step S3.2: When the disinfection robot executes the first path for disinfection, the second environment image of each concealable area is obtained through the camera according to the three-dimensional coordinates of the concealable area in the three-dimensional grid image; Step S3.3: Update the concealable areas in the three-dimensional grid image in real time according to the second environmental image, mark the updated concealable areas in green, and mark the unupdated concealable areas in red; wherein, when acquiring the second environmental image, if a dynamic object blocks the concealable area and the second environmental image cannot be acquired, the concealable area is marked in red and numbered.

4. The robot control method based on artificial intelligence according to claim 3, characterized in that: The step S4 specifically includes: Step S4.1: After graying the second environment image, calculate the brightness value of each pixel in the second environment image, compare the brightness value with a set brightness threshold, and mark the pixels below the brightness threshold in yellow; Step S4.2: Analyze the binarization threshold of the second environment image by the Otsu law, binarize the second environment image according to the binarization threshold, and extract the linear contour of the binarized second environment image; calculate the density of the yellow mark in each linear contour area, and when the density is greater than the density threshold, it is determined to be a hidden point, and the coordinate data of the hidden point is obtained; based on the hidden point linear contour, the coordinate data of the hidden point area is obtained, and the coordinate data is uploaded to the robot arm control terminal; Step S4.3: When the robot control terminal performs the hidden point disinfection task, it transmits infrared rays to the hidden point through the infrared transmitter according to the coordinate data, and the infrared receiver receives the infrared signal reflected from the hidden point and records the time difference between the infrared emission and reception; the depth data of the hidden point is obtained by multiplying the light propagation speed and the time difference, that is, the distance that the disinfection robot needs to spray the hidden point; then a disinfection plan is formulated for the hidden point based on the coordinate data and the depth data, and the disinfection plan includes: the optimal spray column thickness and the optimal spray force of the robot arm.

5. The robot control method based on artificial intelligence according to claim 4, characterized in that: In step S4.3, a disinfection plan is formulated for the hidden point according to the coordinate data and the depth data, specifically including: Step S4.3.1: Locate the hidden point according to the coordinate data, construct the shape of the hidden point according to the coordinate information of each hidden point, and calculate the optimal spray column thickness according to the area of ​​the shape. The calculation formula is as follows: In the above formula, represents the optimum injection column diameter, represents the adjustment factor used to adjust the proportional relationship between the area and the diameter of the spray column, A represents the area of ​​the hidden point, T represents the desired injection time, v represents the injection speed, Indicates the maximum diameter limit of the spray column; Step S4.3.2: Analyze the optimal spray force based on the depth data, and the calculation formula is: In the above formula, Indicates the optimum spray intensity. m The adjustment coefficient that indicates the proportional relationship between the depth and the spraying force. d Represents depth data, An index that indicates the effect of depth on jet strength. An index that represents the effect of time on the jet intensity. Indicates the basic spray force; Step S4.3.3: The optimal spray column diameter and optimal spray force results calculated in steps S4.3.1 and S4.3.23 are transmitted to the robot control terminal, which is called when the robot control terminal controls the robot.

6. The robot control method based on artificial intelligence according to claim 5, characterized in that: The step S5 specifically includes: Step S5.1: using the A* algorithm to plan the second path based on the hidden points and the hidden areas marked in red; Step S5.2: Control the disinfection robot to perform the disinfection task according to the second path. When the disinfection robot passes through the hidden point, control the robotic arm to disinfect the hidden point according to the corresponding disinfection plan; When passing through a hidden area marked in red, identify whether the hidden area has a number. When the number is identified, follow the analysis steps of the hidden area marked in green to identify the hidden point and formulate a disinfection plan, and send the analysis results to the robot arm control terminal, which will execute the disinfection plan; When the number is not recognized, the hidden area is an inaccessible area, and the boundary image of the hidden area is obtained. According to the three-dimensional grid image, multi-angle images of the directly adjacent areas of the hidden area are obtained, and the brightness value of each pixel in the boundary image is calculated. Pixels below the brightness threshold are marked with yellow, and the linear contour of the boundary image is extracted by Otsu's law. The density of the yellow mark in each linear contour area is calculated. When the density is greater than the density threshold, it is determined to be a hidden point, and the coordinate data of the hidden point is obtained. According to the change of the relative position of the hidden point in the multi-angle image, the three-dimensional direction of the hidden point is obtained, and the size of the directly adjacent area corresponding to the three-dimensional direction is identified. The size is determined as the depth data of the hidden point, and the optimal spray column diameter and the optimal spray intensity are obtained according to the coordinate data and the depth data.

7. A robot control system based on artificial intelligence, characterized in that: It includes target environment image acquisition module, 3D grid image construction module, first path disinfection module, robotic arm analysis module and robotic arm control terminal: The target environment image acquisition module is used for the disinfection robot to perform an all-round scan of the target environment by controlling the camera to move up and down and rotate to obtain images at different angles, and to extract a first environment image of the target environment based on the images at different angles; The three-dimensional grid image construction module is used to extract first environmental data of the target environment based on the first environmental image, and construct a three-dimensional grid image of the target environment according to the first environmental data, and identify the concealable area in the three-dimensional grid image according to the first environmental data to divide the walkable area; The first path disinfection module is used to plan the first path based on the three-dimensional grid image, and to control the disinfection robot to execute the first path, and to obtain a second environment image of the concealable area through a camera device; The robotic arm analysis module is used to extract characteristic values ​​of the concealable area based on the second environment image, identify the hidden points in the three-dimensional grid image according to the characteristic values, upload the coordinate data of the hidden points to the robotic arm analysis terminal, and formulate a disinfection plan for the hidden points; The robotic arm control terminal is used to plan a second path based on the hidden point and the concealable area. When executing the second path, the robotic arm control terminal controls the robotic arm to locate and disinfect the hidden point according to the disinfection plan.

8. The robot control system based on artificial intelligence according to claim 7, characterized in that: The robotic arm analysis module includes a hidden point feature recognition module and a disinfection plan formulation module; Among them, the hidden point feature recognition module is used to calculate the brightness value of each pixel in the second environment image after the second environment image is grayed, and compare the brightness value with the set brightness threshold, and mark the pixels below the brightness threshold in yellow, analyze the binarization threshold of the second environment image by Otsu's law, binarize the second environment image according to the binarization threshold, extract the linear contour of the binarized second environment image, calculate the density of the yellow mark in each linear contour area, and when the density is greater than the density threshold, it is judged as the hidden point, and the coordinate data of the hidden point is obtained; The disinfection plan formulation module is used for the robot arm control terminal to transmit infrared rays to the hidden point through the infrared transmitter according to the coordinate data when performing the disinfection of the hidden point, and the infrared receiver receives the infrared signal reflected from the hidden point, records the time difference between the infrared emission and the reception, and obtains the depth data of the hidden point by multiplying the light propagation speed and the time difference, that is, the distance that the disinfection robot needs to spray the hidden point; formulates a disinfection plan for the hidden point according to the coordinate data and the depth data; the disinfection plan includes: the optimal spray column thickness and the optimal spray force of the robot arm.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the electronic device implements the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program. When the computer program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images