Fan control system based on multi-module fusion

Abstract

The invention discloses a fan control system based on multi-module fusion, and relates to the technical field of fan control, in which a processor determines the feature center of a person and the feature center of each obstacle by using an image collected by a camera and an identification model; determining a first position of the person and each obstacle by using the point cloud segmentation model and the distance and angle between each position point and the 3D sensor; determining a failure judgment result; if not, determining an environment map and a target coordinate based on the feature centers of the person and each obstacle and the first position; if so, determining a second position of the person and each obstacle and determining an environment map and a target coordinate by using the point cloud segmentation model and the distance and angle between each position point and the ultrasonic sensor; and determining a planned path of the fan from the original point to the target coordinate by using a path planning algorithm and an environment map, controlling the moving module to move to the target coordinate, and blowing air to the person by the fan at the target coordinate. And flexible and efficient work of the fan is realized.

Description

Technical Field

[0001] The present application relates to the technical field of fan control, and in particular to a fan control system based on multi-module fusion. Background Art

[0002] In large-scale production environments such as factories, due to the vast space and numerous equipment, the use of air conditioning for cooling faces difficulties such as high power consumption, high initial investment and complex maintenance. Therefore, many factories turn to traditional fans to achieve cooling purposes. Traditional fans have the advantages of energy saving and environmental protection, low cost, easy installation, strong mobility and improved ventilation, which can alleviate the discomfort caused by high temperatures to a certain extent. Although its cooling effect is limited by the ambient temperature and range, traditional fans are still an effective choice for cooling large factories.

[0003] However, most fans are installed in fixed locations, which limits their flexibility in use in different areas of the factory or for different work needs. If the cooling area needs to be changed, the fan may need to be reinstalled or moved. Some fans are designed for specific usage scenarios, such as indoors or outdoors, and these fans are not flexible enough for frequent switching between different environments. The cooling effect of general fans is limited by their coverage area. For large-area factory environments, a large number of fans may need to be installed to achieve the desired effect. Fans, especially large fans, are usually not able to directly act on workers due to their fixed position, and their efficiency is low. Summary of the invention

[0004] The purpose of this application is to provide a fan control system based on multi-module fusion to achieve flexible and efficient operation of the fan.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] The present application provides a fan control system based on multi-module fusion, comprising: a camera, a 3D sensor, an ultrasonic sensor, a processor and a mobile module; the camera, the 3D sensor, the ultrasonic sensor, the mobile module and the fan are all connected to the processor; the camera, the 3D sensor, the ultrasonic sensor, the processor and the fan are all arranged on the mobile module;

[0007] The camera is used to collect a panoramic image of the target area under the control of the processor;

[0008] The 3D sensor is used to collect the distance and angle between each position point in the target area and the 3D sensor under the control of the processor;

[0009] The ultrasonic sensor is used to collect the distance and angle between each position point in the target area and the ultrasonic sensor under the control of the processor;

[0010] The processor is configured to:

[0011] Controlling the start of the camera and the 3D sensor;

[0012] Using the panoramic image and the recognition model, determining the pixel points of the person and the pixel points of each obstacle in the target area, and determining the feature center of the person and the feature center of each obstacle;

[0013] Determine the first position of the person and the first position of each obstacle by using the point cloud segmentation model and the distance and angle between each position point and the 3D sensor;

[0014] In the fan coordinate system, the distance between the characteristic center of the character and the first position of the character is calculated to obtain the character distance, and the distance between the characteristic center of each obstacle and the first position of the corresponding obstacle is calculated to obtain the obstacle distance of each obstacle; the fan coordinate system is a coordinate system with the fan control system as the origin;

[0015] Determine the failure judgment result of the camera and the 3D sensor based on the distance between the person and each obstacle; the failure judgment result is failure or not failure;

[0016] If the failure judgment result is not failed, determining the environment map and target coordinates of the target area based on the characteristic center and the first position of the person and the characteristic center and the first position of each obstacle;

[0017] If the failure judgment result is failure, starting the ultrasonic sensor;

[0018] Determine the second position of the person and the second position of each obstacle by using the point cloud segmentation model and the distance and angle between each position point and the ultrasonic sensor;

[0019] Determining an environment map and target coordinates of the target area based on the second position of the person and the second positions of the obstacles;

[0020] Determine a planned path of the fan from an origin to a target coordinate using a path planning algorithm and the environment map;

[0021] The planned path is used to control the moving module to move to the target coordinates, and the fan blows air toward the person at the target coordinates.

[0022] Optionally, collecting the distance and angle between each position point in the target area and the 3D sensor includes:

[0023] The 3D sensor emits a light beam to each position point in the target area and receives a light beam reflected by each position point;

[0024] Determine the distance between each position point in the target area and the 3D sensor according to the emission time of the light beam and the reception time of the light beam reflected by each position point;

[0025] The angle between each position point and the 3D sensor is determined according to the direction of each reflected light beam.

[0026] Optionally, collecting the distance and angle between each position point in the target area and the ultrasonic sensor includes:

[0027] The ultrasonic sensor transmits ultrasonic pulses to each position point in the target area, and receives ultrasonic pulses reflected by each position point;

[0028] Determine the distance between each position point and the ultrasonic sensor according to the emission time of the ultrasonic pulse and the reception time of the ultrasonic pulse reflected by each position point;

[0029] The angle between each position point and the ultrasonic sensor is determined according to the direction of the ultrasonic pulse reflected by each position point.

[0030] Optionally, using the panoramic image and the recognition model, determining the pixel points of the person and the pixel points of each obstacle in the target area, and determining the feature center of the person and the feature center of each obstacle, includes:

[0031] Preprocessing the panoramic images respectively to obtain corresponding preprocessed images; the preprocessing includes: resizing, normalization, and denoising;

[0032] Inputting each preprocessed image into the recognition model respectively to obtain pixel points of the person and pixel points of each obstacle in the target area; the recognition model is obtained by training a convolutional neural network using a recognition training set; the recognition training set includes: a plurality of training images and pixel points of the person and pixel points of each obstacle in each training image;

[0033] The centroid of the pixel points of the character is determined as the characteristic center of the character, and the centroid of the pixel points of each obstacle is determined as the characteristic center of each obstacle.

[0034] Optionally, using the point cloud segmentation model and the distance and angle between each position point and the 3D sensor to determine the first position of the person and the first position of each obstacle includes:

[0035] Convert the distance and angle between each position point and the 3D sensor into a coordinate point of each position point in the fan coordinate system, thereby obtaining a first initial point cloud of the target area;

[0036] Inputting the first initial point cloud into the point cloud segmentation model to obtain a first point cloud of the person and a first point cloud of each obstacle; the point cloud segmentation model is obtained by training a deep learning model using a segmentation training set, wherein the segmentation training set includes: point clouds of multiple training areas and point clouds of the person and point clouds of each obstacle in each training area;

[0037] The centroid of the first point cloud of the character is determined as the first position of the character, and the centroid of the first point cloud of each obstacle is determined as the first position of each obstacle.

[0038] Optionally, determining the failure judgment result of the camera and the 3D sensor based on the person distance and the obstacle distances includes:

[0039] Determine whether the distance to the person or any obstacle is greater than a preset distance;

[0040] If yes, the failure judgment result is failure;

[0041] If not, the failure judgment result is not failed.

[0042] Optionally, determining the environment map and target coordinates of the target area based on the feature center and the first position of the character and the feature center and the first position of each obstacle includes:

[0043] Calculating an average value of coordinates of a feature center and a first position of the character in the fan coordinate system to obtain a first coordinate of the character, and determining the first coordinate of the character as a target coordinate;

[0044] Calculate the average values ​​of the coordinates of the characteristic center and the first position of each obstacle in the fan coordinate system respectively to obtain the first coordinates of each obstacle;

[0045] The environment map is constructed based on the first coordinates of the character and the first coordinates of each obstacle.

[0046] Optionally, using the point cloud segmentation model and the distance and angle between each position point and the ultrasonic sensor to determine the second position of the person and the second position of each obstacle includes:

[0047] Convert the distance and angle between each position point and the ultrasonic sensor into the coordinate point of each position point in the fan coordinate system, thereby obtaining a second initial point cloud of the target area;

[0048] Inputting the second initial point cloud into the point cloud segmentation model to obtain a second point cloud of the person and a second point cloud of each obstacle;

[0049] The centroid of the second point cloud of the character is determined as the second position of the character, and the centroid of the second point cloud of each obstacle is determined as the second position of each obstacle.

[0050] Optionally, determining the environment map and target coordinates of the target area based on the second position of the person and the second positions of each obstacle includes:

[0051] Determine the coordinates of the second position of the character in the fan coordinate system as the second coordinates of the character;

[0052] Determine the coordinates of the second position of each obstacle in the fan coordinate system as the second coordinates of each obstacle, and determine the second coordinates of the character as the target coordinates;

[0053] The environment map is constructed based on the second coordinates of the character and the second coordinates of each obstacle.

[0054] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0055] The present application discloses a fan control system based on multi-module fusion, including: a camera, a 3D sensor, an ultrasonic sensor, a processor and a mobile module; the camera, the 3D sensor, the ultrasonic sensor, the mobile module and the fan are all connected to the processor; the camera, the 3D sensor, the ultrasonic sensor, the processor and the fan are all arranged on the mobile module; the camera is used to collect a panoramic image of a target area under the control of the processor; the 3D sensor is used to collect the distance and angle between each position point in the target area and the 3D sensor under the control of the processor; the ultrasonic sensor is used to collect the distance and angle between each position point in the target area and the ultrasonic sensor under the control of the processor; the processor is used to: control the start of the camera and the 3D sensor; determine the pixel points of the person and the pixel points of each obstacle in the target area by using the panoramic image and the recognition model, and determine the feature center of the person and the feature center of each obstacle; determine the first position of the person and the first position of each obstacle by using the point cloud segmentation model and the distance and angle between each position point and the 3D sensor; calculate the feature center of the person in the fan coordinate system; The distance between the feature center and the first position of the character is calculated to obtain the character distance, and the distance between the feature center of each obstacle and the first position of the corresponding obstacle is calculated respectively to obtain the obstacle distance of each obstacle; the fan coordinate system is a coordinate system with the fan control system as the origin; based on the character distance and the distance of each obstacle, the failure judgment result of the camera and the 3D sensor is determined; the failure judgment result is failure or non-failure; if the failure judgment result is non-failure, the environmental map and target coordinates of the target area are determined based on the feature center and the first position of the character and the feature center and the first position of each obstacle; if the failure judgment result is failure, the ultrasonic sensor is started; the second position of the character and the second position of each obstacle are determined by using the point cloud segmentation model and the distance and angle between each position point and the ultrasonic sensor; the environmental map and target coordinates of the target area are determined based on the second position of the character and the second position of each obstacle; the path planning algorithm and the environmental map are used to determine the planned path of the fan from the origin to the target coordinate; the mobile module is controlled to move to the target coordinate by using the planned path, and the fan blows air to the character at the target coordinate. This application utilizes the coordinated cooperation of a camera, a 3D sensor, an ultrasonic sensor, a processor, and a mobile module to realize the motion blowing of the fan, making the fan work more flexibly and efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1A schematic diagram of the hardware structure of a fan control system based on multi-module fusion provided in one embodiment of the present application;

[0058] Figure 2 A schematic diagram of the electrical connection structure of a fan control system based on multi-module fusion provided in one embodiment of the present application.

[0059] Reference numerals:

[0060] Camera—1, 3D sensor—2, Ultrasonic sensor—3, Processor—4, Mobile module—5, Fan—6. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] The purpose of this application is to provide a fan control system based on multi-module fusion, aiming to achieve flexible and efficient operation of the fan.

[0063] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0064] In an exemplary embodiment, Figure 1 and Figure 2 As shown, the fan control system based on multi-module fusion in this embodiment includes: a camera 1, a 3D sensor 2, an ultrasonic sensor 3, a processor 4 and a mobile module 5; the camera 1, the 3D sensor 2, the ultrasonic sensor 3, the mobile module 5 and the fan 6 are all connected to the processor 4; the camera 1, the 3D sensor 2, the ultrasonic sensor 3, the processor 4 and the fan 6 are all arranged on the mobile module 5.

[0065] The camera is used to collect a panoramic image of the target area under the control of the processor.

[0066] The 3D sensor is used to collect the distance and angle between each position point in the target area and the 3D sensor under the control of the processor.

[0067] The ultrasonic sensor is used to collect the distance and angle between each position point in the target area and the ultrasonic sensor under the control of the processor.

[0068] Processor for:

[0069] Controls the activation of the camera and 3D sensor.

[0070] The panoramic image and the recognition model are used to determine the pixel points of the person and the pixel points of each obstacle in the target area, and to determine the feature center of the person and the feature center of each obstacle.

[0071] The point cloud segmentation model and the distance and angle between each position point and the 3D sensor are used to determine the first position of the person and the first position of each obstacle.

[0072] In the fan coordinate system, the distance between the feature center of the character and the first position of the character is calculated to obtain the character distance, and the distance between the feature center of each obstacle and the first position of the corresponding obstacle is calculated respectively to obtain the obstacle distance of each obstacle; the fan coordinate system is a coordinate system with the fan control system as the origin.

[0073] Based on the distance between the person and each obstacle, the failure judgment result of the camera and the 3D sensor is determined; the failure judgment result is failure or not failure.

[0074] If the failure judgment result is not failed, the environment map and target coordinates of the target area are determined based on the characteristic center and the first position of the person and the characteristic center and the first position of each obstacle.

[0075] If the failure judgment result is failure, the ultrasonic sensor is started.

[0076] The point cloud segmentation model and the distance and angle between each position point and the ultrasonic sensor are used to determine the second position of the person and the second position of each obstacle.

[0077] Based on the second position of the character and the second positions of the obstacles, an environment map and target coordinates of the target area are determined.

[0078] Using the path planning algorithm and the environment map, the planned path of the fan from the origin to the target coordinates is determined.

[0079] The planned path is used to control the mobile module to move to the target coordinates, and the fan blows air to the character at the target coordinates.

[0080] Specifically, the path planning algorithm includes but is not limited to: A* algorithm, Dijkstra algorithm, Rapidly-exploring Random Trees (RRT) algorithm, etc. After the initial path is obtained by path planning, the initial path is smoothed by using cubic spline interpolation method to obtain the planned path.

[0081] As an optional implementation, collecting the distance and angle between each position point in the target area and the 3D sensor includes:

[0082] Step 11: The 3D sensor transmits a light beam to each position point in the target area and receives a light beam reflected by each position point.

[0083] Specifically, the 3D sensor simultaneously emits a light beam to each position point in the target area.

[0084] Step 12: Determine the distance between each position point in the target area and the 3D sensor according to the emission time of the light beam and the reception time of the light beam reflected by each position point.

[0085] Specifically, the calculation formula for the distance between any current position point and the 3D sensor is:

[0086]

[0087] Among them, S 1 is the distance between the current position and the 3D sensor; c is the speed of light; t 1 is the emission time of the light beam; t 2 The reception time of the light beam reflected from the current position.

[0088] Step 13: Determine the angle between each position point and the 3D sensor according to the direction of each reflected light beam.

[0089] As an optional implementation, collecting the distance and angle between each position point in the target area and the ultrasonic sensor includes:

[0090] Step 21: The ultrasonic sensor transmits ultrasonic pulses to each position point in the target area, and receives ultrasonic pulses reflected by each position point.

[0091] Specifically, the ultrasonic sensor simultaneously transmits ultrasonic pulses to each position point in the target area.

[0092] Step 22: Determine the distance between each position point and the ultrasonic sensor according to the emission time of the ultrasonic pulse and the reception time of the ultrasonic pulse reflected by each position point.

[0093] Specifically, the calculation formula for the distance between any current position point and the ultrasonic sensor is:

[0094]

[0095] Among them, S 2 is the distance between the current position and the ultrasonic sensor; v is the propagation speed of the ultrasonic pulse; t 3 is the emission time of the ultrasonic pulse; t 4 It is the reception time of the ultrasonic pulse reflected at the current position.

[0096] Step 23: Determine the angle between each position point and the ultrasonic sensor according to the direction of the ultrasonic pulse reflected by each position point.

[0097] As an optional implementation, the panoramic image and the recognition model are used to determine the pixel points of the person and the pixel points of each obstacle in the target area, and to determine the feature center of the person and the feature center of each obstacle, including:

[0098] Step 31: preprocess the panoramic images respectively to obtain corresponding preprocessed images; the preprocessing includes: resizing, normalization, and denoising.

[0099] Specifically, the expressions for adjusting the size include:

[0100] I resized = resize(I, size).

[0101] Among them, I resized is the resized image; resize(·) is the resizing function; I is the original image; size is the preset size.

[0102] The normalized expressions include:

[0103]

[0104] Among them, I normalized (i, j) is the pixel value of the i-th row and j-th column in the normalized image; I resized (i, j) is the pixel value at the i-th row and j-th column; μ is the mean of the resized image; M is the number of rows of the resized image; N is the number of columns of the resized image; σ is the standard deviation of the resized image.

[0105] Step 32: Input each preprocessed image into the recognition model respectively to obtain the pixel points of the person and the pixel points of each obstacle in the target area; the recognition model is obtained by training the convolutional neural network using the recognition training set; the recognition training set includes: multiple training images and the pixel points of the person and the pixel points of each obstacle in each training image.

[0106] Step 33: Determine the centroid of the pixel points of the person as the characteristic center of the person, and determine the centroid of the pixel points of each obstacle as the characteristic center of each obstacle.

[0107] As an optional implementation, the first position of the person and the first position of each obstacle are determined by using the point cloud segmentation model and the distance and angle between each position point and the 3D sensor, including:

[0108] Step 41: Convert the distance and angle between each position point and the 3D sensor into the coordinate point of each position point in the fan coordinate system, thereby obtaining a first initial point cloud of the target area.

[0109] Specifically, the distance and angle between each position point and the 3D sensor are converted into the coordinate point of each position point in the fan coordinate system by using the coordinate conversion formula, so as to obtain the first initial point cloud of the target area. The coordinate formula includes:

[0110] x=S 1 ×cos(φ 1 )×sin(φ 2 ).

[0111] y=S 1 ×cos(φ 1 )×cos(φ 2 ).

[0112] z=S 1 ×sin(φ 1 ).

[0113] Where x is the x-axis component of the coordinate point in the fan coordinate system; φ 1 is the vertical component of the angle; φ 2 is the horizontal component of the angle; y is the y-axis component of the coordinate point in the fan coordinate system; z is the z-axis component of the coordinate point in the fan coordinate system.

[0114] Step 42: Input the first initial point cloud into the point cloud segmentation model to obtain the first point cloud of the person and the first point cloud of each obstacle; the point cloud segmentation model is obtained by training the deep learning model using the segmentation training set, and the segmentation training set includes: point clouds of multiple training areas and point clouds of the person and point clouds of each obstacle in each training area.

[0115] Step 44: Determine the centroid of the first point cloud of the character as the first position of the character, and determine the centroid of the first point cloud of each obstacle as the first position of each obstacle.

[0116] As an optional implementation, based on the distance between the person and the distance between obstacles, the failure judgment result of the camera and the 3D sensor is determined, including:

[0117] Step 51: Determine whether the distance to the person or any obstacle is greater than a preset distance.

[0118] Step 52: If yes, the failure judgment result is failure.

[0119] Step 53: If not, the failure judgment result is not failed.

[0120] As an optional implementation, determining the environment map and target coordinates of the target area based on the feature center and the first position of the person and the feature center and the first position of each obstacle includes:

[0121] Step 61: Calculate the average value of the coordinates of the feature center and the first position of the character in the fan coordinate system to obtain the first coordinate of the character, and determine the first coordinate of the character as the target coordinate.

[0122] Step 62: Calculate the average values ​​of the coordinates of the characteristic center and the first position of each obstacle in the fan coordinate system respectively to obtain the first coordinates of each obstacle.

[0123] Step 63: construct an environment map based on the first coordinates of the character and the first coordinates of each obstacle.

[0124] As an optional implementation, the second position of the person and the second position of each obstacle are determined by using the point cloud segmentation model and the distance and angle between each position point and the ultrasonic sensor, including:

[0125] Step 71: Convert the distance and angle between each position point and the ultrasonic sensor into the coordinate point of each position point in the fan coordinate system, thereby obtaining a second initial point cloud of the target area.

[0126] Step 72: Input the second initial point cloud into the point cloud segmentation model to obtain a second point cloud of the person and a second point cloud of each obstacle.

[0127] Step 73: Determine the centroid of the second point cloud of the character as the second position of the character, and determine the centroid of the second point cloud of each obstacle as the second position of each obstacle.

[0128] As an optional implementation, determining the environment map and target coordinates of the target area based on the second position of the person and the second positions of the obstacles includes:

[0129] Step 81: Determine the coordinates of the second position of the character in the fan coordinate system as the second coordinates of the character.

[0130] Step 82: The coordinates of the second position of each obstacle in the fan coordinate system are respectively determined as the second coordinates of each obstacle, and the second coordinates of the character are determined as the target coordinates.

[0131] Step 83: Construct an environment map based on the second coordinates of the character and the second coordinates of each obstacle.

[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0133] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the system and its core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A fan control system based on multi-module fusion, characterized in that: The fan control system based on multi-module fusion includes: a camera, a 3D sensor, an ultrasonic sensor, a processor and a mobile module; the camera, the 3D sensor, the ultrasonic sensor, the mobile module and the fan are all connected to the processor; the camera, the 3D sensor, the ultrasonic sensor, the processor and the fan are all arranged on the mobile module; The camera is used to collect a panoramic image of the target area under the control of the processor; The 3D sensor is used to collect the distance and angle between each position point in the target area and the 3D sensor under the control of the processor; The ultrasonic sensor is used to collect the distance and angle between each position point in the target area and the ultrasonic sensor under the control of the processor; The processor is configured to: Controlling the start of the camera and the 3D sensor; Using the panoramic image and the recognition model, determining the pixel points of the person and the pixel points of each obstacle in the target area, and determining the feature center of the person and the feature center of each obstacle; Determine the first position of the person and the first position of each obstacle by using the point cloud segmentation model and the distance and angle between each position point and the 3D sensor; In the fan coordinate system, the distance between the characteristic center of the character and the first position of the character is calculated to obtain the character distance, and the distance between the characteristic center of each obstacle and the first position of the corresponding obstacle is calculated to obtain the obstacle distance of each obstacle; the fan coordinate system is a coordinate system with the fan control system as the origin; Determine the failure judgment result of the camera and the 3D sensor based on the distance between the person and each obstacle; the failure judgment result is failure or not failure; If the failure judgment result is not failed, determining the environment map and target coordinates of the target area based on the characteristic center and the first position of the person and the characteristic center and the first position of each obstacle; If the failure judgment result is failure, starting the ultrasonic sensor; Determine the second position of the person and the second position of each obstacle by using the point cloud segmentation model and the distance and angle between each position point and the ultrasonic sensor; Determining an environment map and target coordinates of the target area based on the second position of the person and the second positions of the obstacles; Determine a planned path of the fan from an origin to a target coordinate using a path planning algorithm and the environment map; The planned path is used to control the moving module to move to the target coordinates, and the fan blows air toward the person at the target coordinates.

2. The fan control system based on multi-module fusion according to claim 1 is characterized in that: Collecting the distance and angle between each position point in the target area and the 3D sensor includes: The 3D sensor emits a light beam to each position point in the target area and receives a light beam reflected by each position point; Determine the distance between each position point in the target area and the 3D sensor according to the emission time of the light beam and the reception time of the light beam reflected by each position point; The angle between each position point and the 3D sensor is determined according to the direction of each reflected light beam.

3. The fan control system based on multi-module fusion according to claim 1, characterized in that: Collecting the distance and angle between each position point in the target area and the ultrasonic sensor includes: The ultrasonic sensor transmits ultrasonic pulses to each position point in the target area, and receives ultrasonic pulses reflected by each position point; Determine the distance between each position point and the ultrasonic sensor according to the emission time of the ultrasonic pulse and the reception time of the ultrasonic pulse reflected by each position point; The angle between each position point and the ultrasonic sensor is determined according to the direction of the ultrasonic pulse reflected by each position point.

4. The fan control system based on multi-module fusion according to claim 1, characterized in that: Using the panoramic image and the recognition model, determining the pixel points of the person and the pixel points of each obstacle in the target area, and determining the feature center of the person and the feature center of each obstacle, including: Preprocessing the panoramic images respectively to obtain corresponding preprocessed images; the preprocessing includes: resizing, normalization, and denoising; Inputting each preprocessed image into the recognition model respectively to obtain pixel points of the person and pixel points of each obstacle in the target area; the recognition model is obtained by training a convolutional neural network using a recognition training set; the recognition training set includes: a plurality of training images and pixel points of the person and pixel points of each obstacle in each training image; The centroid of the pixel points of the character is determined as the characteristic center of the character, and the centroid of the pixel points of each obstacle is determined as the characteristic center of each obstacle.

5. The fan control system based on multi-module fusion according to claim 1, characterized in that: Determining the first position of the person and the first position of each obstacle by using the point cloud segmentation model and the distance and angle between each position point and the 3D sensor, including: Convert the distance and angle between each position point and the 3D sensor into a coordinate point of each position point in the fan coordinate system, thereby obtaining a first initial point cloud of the target area; Inputting the first initial point cloud into the point cloud segmentation model to obtain a first point cloud of the person and a first point cloud of each obstacle; the point cloud segmentation model is obtained by training a deep learning model using a segmentation training set, wherein the segmentation training set includes: point clouds of multiple training areas and point clouds of the person and point clouds of each obstacle in each training area; The centroid of the first point cloud of the character is determined as the first position of the character, and the centroid of the first point cloud of each obstacle is determined as the first position of each obstacle.

6. The fan control system based on multi-module fusion according to claim 1, characterized in that: Determining failure judgment results of the camera and the 3D sensor based on the person distance and the obstacle distances includes: Determine whether the distance to the person or any obstacle is greater than a preset distance; If yes, the failure judgment result is failure; If not, the failure judgment result is not failed.

7. The fan control system based on multi-module fusion according to claim 1, characterized in that: Determining the environment map and target coordinates of the target area based on the feature center and the first position of the person and the feature center and the first position of each obstacle includes: Calculating an average value of coordinates of a feature center and a first position of the character in the fan coordinate system to obtain a first coordinate of the character, and determining the first coordinate of the character as a target coordinate; Calculate the average values ​​of the coordinates of the characteristic center and the first position of each obstacle in the fan coordinate system respectively to obtain the first coordinates of each obstacle; The environment map is constructed based on the first coordinates of the character and the first coordinates of each obstacle.

8. The fan control system based on multi-module fusion according to claim 5, characterized in that: Determining the second position of the person and the second position of each obstacle by using the point cloud segmentation model and the distance and angle between each position point and the ultrasonic sensor, including: Convert the distance and angle between each position point and the ultrasonic sensor into the coordinate point of each position point in the fan coordinate system, thereby obtaining a second initial point cloud of the target area; Inputting the second initial point cloud into the point cloud segmentation model to obtain a second point cloud of the person and a second point cloud of each obstacle; The centroid of the second point cloud of the character is determined as the second position of the character, and the centroid of the second point cloud of each obstacle is determined as the second position of each obstacle.

9. The fan control system based on multi-module fusion according to claim 1, characterized in that: Determining the environment map and target coordinates of the target area based on the second position of the person and the second positions of the obstacles includes: Determine the coordinates of the second position of the character in the fan coordinate system as the second coordinates of the character; Determine the coordinates of the second position of each obstacle in the fan coordinate system as the second coordinates of each obstacle, and determine the second coordinates of the character as the target coordinates; The environment map is constructed based on the second coordinates of the character and the second coordinates of each obstacle.

Images