Automatic navigation control method and system for unmanned sweeper

Abstract

The application relates to the technical field of automatic navigation, and provides an automatic navigation control method and system of an unmanned sweeper. The method comprises the following steps: obtaining a historical monitoring video of a to-be-cleaned area, generating an initial cleaning path of the to-be-cleaned area based on the historical monitoring video, and predicting a cleaning time period corresponding to the initial cleaning path; obtaining multi-source data information of a target area, and constructing a dynamic environment semantic map sequence corresponding to the to-be-cleaned area in the cleaning time period based on the multi-source data information; wherein the to-be-cleaned area is contained in the target area; performing optimization processing on the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path; and controlling the sweeper to perform a cleaning task based on the target cleaning path. The method helps to improve the adaptability of the sweeper in a complex environment.

Description

Technical Field

[0001] This application relates to the field of automatic navigation technology, and in particular to an automatic navigation control method and system for an unmanned sweeper. Background Technology

[0002] With the continuous development of technology, autonomous driving technology is being applied more and more widely in various fields, especially in the field of intelligent cleaning equipment. Autonomous sweepers have become widely used cleaning equipment in homes, shopping malls, office buildings, and other places. Thanks to their automation and intelligence, autonomous sweepers can autonomously plan cleaning paths and efficiently complete cleaning tasks, greatly reducing the labor intensity of manual cleaning and improving cleaning efficiency and quality.

[0003] However, most current autonomous sweeping vehicles rely on simple map building for path planning, which is usually static and lacks adaptability to dynamic environments. This method is not effective in cleaning complex dynamic environments. Summary of the Invention

[0004] This application provides an automatic navigation control method and system for an unmanned sweeping vehicle, in order to solve the problems mentioned in the background art.

[0005] In a first aspect, this application provides an automatic navigation control method for an unmanned sweeper, comprising:

[0006] The system acquires historical surveillance video of the area to be cleaned, generates an initial cleaning path for the area based on the historical surveillance video, and predicts the cleaning time period corresponding to the initial cleaning path.

[0007] Acquire multi-source data information of the target area, and construct a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein, the area to be cleaned is included in the target area;

[0008] The initial cleaning path is optimized based on the dynamic environmental semantic map sequence to obtain the target cleaning path;

[0009] The sweeper is controlled to perform the cleaning task based on the target cleaning path.

[0010] In one possible implementation, generating an initial cleaning path for the area to be cleaned based on the historical surveillance video, and predicting the cleaning time period corresponding to the initial cleaning path, includes:

[0011] Acquire multi-camera surveillance videos of the area to be cleaned within a preset historical time period, and stitch the multi-camera surveillance videos into a panoramic surveillance video;

[0012] Dynamic object removal processing is performed on the panoramic monitoring video to obtain a static scene video corresponding to the area to be cleaned.

[0013] A pollution heat map is generated based on the static scene video;

[0014] An initial cleaning path is generated based on the pollution heat map; the initial cleaning path includes multiple sub-cleanable ground areas, and each sub-cleanable ground area has a corresponding cleaning rate.

[0015] The cleaning time period is generated based on the cleaning rate corresponding to each sub-cleanable ground area.

[0016] In one possible implementation, the step of performing dynamic object removal processing based on the panoramic monitoring video to obtain a static scene video corresponding to the area to be cleaned includes:

[0017] The panoramic surveillance video is processed into frames to obtain a video frame sequence;

[0018] Dynamic object removal processing is performed on each video frame of the video frame sequence in sequence to obtain a first target video frame sequence, and the static scene video is generated based on the first target video frame sequence.

[0019] In one possible implementation, generating the pollution heatmap based on the static scene video includes:

[0020] Obtain the historical moment when the last cleaning of the area to be cleaned was completed, and extract a target video from the static scene video based on the historical moment; wherein, the target video is the video corresponding to the historical moment and the current moment in the static scene video;

[0021] The target video is segmented into frames to obtain a second target video frame sequence;

[0022] For each second target video frame in the second target video frame sequence, the cleanable ground area corresponding to the second target video frame is segmented to obtain multiple sub-cleanable ground areas;

[0023] For each of the sub-cleanable ground areas, the pollution levels corresponding to the sub-cleanable ground areas in each of the second target video frames are fused to obtain the target pollution level corresponding to the sub-cleanable ground areas;

[0024] For each of the sub-cleanable ground areas, the sub-cleanable ground areas are rendered based on the target pollution level corresponding to the sub-cleanable ground area; the rendered sub-cleanable ground areas constitute the pollution heat map.

[0025] In one possible implementation, the multi-source data information includes the operational data information of each dynamic object within the target area during a preset time period, wherein the preset time period is a time period connected to and preceding the current time. The step of constructing a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information includes:

[0026] For each dynamic object, a predicted running trajectory of the dynamic object is generated based on the running data information corresponding to the dynamic object within the cleaning time period, and it is determined whether the predicted running trajectory passes through the area to be cleaned. If it does, the dynamic object is determined to be the target dynamic object.

[0027] For each of the target dynamic objects, the predicted running trajectory of the target dynamic object in the area to be cleaned is segmented to obtain multiple running nodes, and the target time corresponding to each running node is determined, and the semantics corresponding to the target dynamic object are marked at each running node.

[0028] For each target time, a dynamic environmental semantic map corresponding to the target time is generated based on the running node corresponding to the target time;

[0029] Based on the target time corresponding to each dynamic environment semantic map, the dynamic environment semantic maps are arranged in sequence to obtain the dynamic environment semantic map sequence.

[0030] In one possible implementation, optimizing the initial cleaning path based on the dynamic environmental semantic map sequence to obtain the target cleaning path includes:

[0031] Determine the cleaning node corresponding to each target time on the initial cleaning path;

[0032] For each target time, the target dynamic objects in the dynamic environment semantic map corresponding to the target time are expanded, and it is determined whether the cleaning node corresponding to the target time coincides with any target dynamic object after expansion. If they coincide, the cleaning node is adjusted so that the cleaning area does not coincide with any target dynamic object after expansion. The initial cleaning path after the cleaning node adjustment is the target cleaning path.

[0033] Secondly, this application provides an automatic navigation control system for an unmanned sweeper, comprising:

[0034] The generation module is used to acquire historical monitoring videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical monitoring videos, and predict the cleaning time period corresponding to the initial cleaning path.

[0035] A construction module is used to acquire multi-source data information of the target area and construct a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein, the area to be cleaned is included in the target area;

[0036] An optimization processing module is used to optimize the initial cleaning path based on the dynamic environmental semantic map sequence to obtain the target cleaning path;

[0037] The control module controls the sweeper to perform the cleaning task based on the target cleaning path.

[0038] This application provides an automatic navigation control method and system for an unmanned sweeping vehicle. The method includes: acquiring historical monitoring video of the area to be cleaned, generating an initial cleaning path for the area based on the historical monitoring video, and predicting the cleaning time period corresponding to the initial cleaning path; acquiring multi-source data information of a target area, and constructing a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein the area to be cleaned is included in the target area; optimizing the initial cleaning path based on the dynamic environmental semantic map sequence to obtain a target cleaning path; and controlling the sweeping vehicle to perform a cleaning task based on the target cleaning path. The method provided in this embodiment, on the one hand, acquires historical monitoring videos of the area to be cleaned, generates an initial cleaning path for the area based on the historical monitoring videos, and predicts the cleaning time period corresponding to the initial cleaning path. This enables the sweeper to fully perceive the area to be cleaned before cleaning. On the other hand, by acquiring multi-source data information of the target area and constructing a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information, it enables the prediction of the dynamic environment of the area to be cleaned within the future cleaning time period, which helps improve the sweeper's adaptability in complex environments. Furthermore, by optimizing the initial cleaning path based on the dynamic environmental semantic map sequence to obtain the target cleaning path, the sweeper can avoid obstacles as much as possible during the sweeping process, prevent collisions between the sweeper and obstacles, and improve the sweeping efficiency of the sweeper. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1A flowchart illustrating the automatic navigation control method for an unmanned sweeper provided in this application embodiment;

[0041] Figure 2 A schematic block diagram of the structure of the automatic navigation control system for the unmanned sweeper provided in the embodiments of this application;

[0042] Figure 3 A schematic block diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the described order. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0045] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0046] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the relevant listed items and all possible combinations, and includes such combinations.

[0047] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0048] Please see Figure 1 , Figure 1 This is a flowchart illustrating the automatic navigation control method for an unmanned sweeper provided in an embodiment of this application, as shown below. Figure 1 As shown, the automatic navigation control method for the unmanned sweeper provided in this application includes steps S1 to S4.

[0049] Step S1: Obtain historical monitoring videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical monitoring videos, and predict the cleaning time period corresponding to the initial cleaning path.

[0050] Step S2: Obtain multi-source data information of the target area, and construct a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein, the area to be cleaned is included in the target area.

[0051] Step S3: Optimize the initial cleaning path based on the dynamic environment semantic map sequence to obtain the target cleaning path.

[0052] Step S4: Control the sweeper to perform the cleaning task based on the target cleaning path.

[0053] It should be noted that the method provided in this embodiment is used for cleaning indoor spaces such as homes, shopping malls, and office buildings.

[0054] This embodiment specifically includes:

[0055] As described in step S1 above, historical monitoring videos of the area to be cleaned are acquired, and an initial cleaning path for the area to be cleaned is generated based on the historical monitoring videos. The cleaning time period corresponding to the initial cleaning path is then predicted. Specifically, step S1 includes: acquiring multi-camera monitoring videos of the area to be cleaned within a preset historical time period, and stitching the multi-camera monitoring videos into a panoramic monitoring video; performing dynamic object removal processing on the panoramic monitoring video to obtain a static scene video corresponding to the area to be cleaned; generating a pollution heat map based on the static scene video; generating an initial cleaning path based on the pollution heat map; the initial cleaning path includes multiple sub-cleanable ground areas, each of which has a corresponding cleaning rate; and generating the cleaning time period based on the cleaning rate corresponding to each sub-cleanable ground area.

[0056] As described in step S2 above, multi-source data information of the target area is obtained, and a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period is constructed based on the multi-source data information; wherein, the area to be cleaned is included in the target area. Specifically, the multi-source data information includes the operation data information of each dynamic object in the target area within a preset time period. The preset time period is a time period connected to the current time and preceding the current time. Step S2 includes: for each dynamic object, generating a predicted running trajectory of the dynamic object within the cleaning time period based on the operation data information corresponding to the dynamic object, and determining whether the predicted running trajectory passes through the area to be cleaned. If it does, the dynamic object is determined to be a target dynamic object; for each target dynamic object, segmenting a portion of the predicted running trajectory of the target dynamic object within the area to be cleaned to obtain multiple running nodes, determining the target time corresponding to each running node, and marking the semantics corresponding to the target dynamic object at each running node; for each target time, generating a dynamic environment semantic map corresponding to the target time based on the running nodes corresponding to the target time; and arranging each dynamic environment semantic map in sequence based on the target time corresponding to each dynamic environment semantic map to obtain the dynamic environment semantic map sequence.

[0057] As described in step S3 above, the initial cleaning path is optimized based on the dynamic environment semantic map sequence to obtain the target cleaning path. Specifically, step S3 includes: determining the cleaning node corresponding to each target time on the initial cleaning path; for each target time, performing dilation processing on each target dynamic object in the dynamic environment semantic map corresponding to the target time, and determining whether the cleaning node corresponding to the target time overlaps with any target dynamic object after dilation processing. If they overlap, the cleaning node is adjusted so that the cleaning area does not overlap with any target dynamic object after dilation processing; wherein, the initial cleaning path after the cleaning node adjustment is the target cleaning path.

[0058] As described in step S4 above, the sweeper is controlled to perform the cleaning task based on the target cleaning path. Specifically, firstly, a control command corresponding to the target cleaning path is generated, and the sweeper is controlled to perform the cleaning task based on the control command.

[0059] The method provided in this embodiment, on the one hand, acquires historical monitoring videos of the area to be cleaned, generates an initial cleaning path for the area based on the historical monitoring videos, and predicts the cleaning time period corresponding to the initial cleaning path. This enables the sweeper to fully perceive the area to be cleaned before cleaning. On the other hand, by acquiring multi-source data information of the target area and constructing a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information, it enables the prediction of the dynamic environment of the area to be cleaned within the future cleaning time period, which helps improve the sweeper's adaptability in complex environments. Furthermore, by optimizing the initial cleaning path based on the dynamic environmental semantic map sequence to obtain the target cleaning path, the sweeper can avoid obstacles as much as possible during the sweeping process, prevent collisions between the sweeper and obstacles, and improve the sweeping efficiency of the sweeper.

[0060] In some embodiments, generating an initial cleaning path for the area to be cleaned based on the historical surveillance video and predicting the cleaning time period corresponding to the initial cleaning path includes the following steps:

[0061] The system acquires multi-camera surveillance videos of the area to be cleaned within a preset historical time period and stitches the multi-camera surveillance videos into a panoramic surveillance video. The preset historical time period is at least two days prior to the current moment. Specifically, the multi-camera surveillance videos are stitched together based on the SIFT feature matching algorithm to obtain the panoramic surveillance video.

[0062] Dynamic object removal processing is performed on the panoramic monitoring video to obtain a static scene video corresponding to the area to be cleaned.

[0063] A pollution heat map is generated based on the static scene video;

[0064] An initial cleaning path is generated based on the pollution heat map; the initial cleaning path includes multiple sub-cleanable ground areas, and each sub-cleanable ground area has a corresponding cleaning rate.

[0065] The cleaning time period is generated based on the cleaning rate corresponding to each sub-cleanable ground area; specifically, for each sub-cleanable ground area, the ratio of the cleaning length corresponding to the sub-cleanable ground area to the cleaning rate corresponding to the sub-cleanable ground area is determined as the cleaning duration corresponding to the sub-cleanable ground area, and the cleaning durations of each sub-cleanable ground area are added together to obtain the target cleaning duration, and the cleaning time period corresponding to the target cleaning duration is determined by taking the current time as the initial time.

[0066] The method provided in this embodiment has several advantages. First, compared to traditional single-view monitoring, using multi-camera stitched panoramic video can more accurately acquire the overall view of the area to be cleaned, improving the depth of understanding of the area and providing an effective data foundation for subsequent path planning and improved cleaning efficiency. Second, by performing dynamic object removal processing based on the panoramic monitoring video, a static scene video corresponding to the area to be cleaned is obtained. This eliminates interference from dynamic objects in the environment, retaining only the static scene related to the cleaning task, reducing noise factors in the environment, and helping to improve the accuracy of path planning. Third, by generating a pollution heat map based on the static scene video, the pollution level and cleaning needs of different sub-cleanable ground areas can be clearly identified, which helps to improve cleaning quality and efficiency.

[0067] In some embodiments, the step of performing dynamic object removal processing based on the panoramic monitoring video to obtain a static scene video corresponding to the area to be cleaned includes the following steps:

[0068] The panoramic surveillance video is processed into frames to obtain a video frame sequence;

[0069] Dynamic object removal processing is performed on each video frame of the video frame sequence in sequence to obtain a first target video frame sequence, and the static scene video is generated based on the first target video frame sequence; specifically, semantic recognition is performed on each object in each video frame of the video frame sequence, and dynamic objects in the video frame are removed based on the results of semantic recognition.

[0070] The method provided in this embodiment obtains a first target video frame sequence by sequentially performing dynamic object removal processing on each video frame of the video frame sequence. It can accurately identify and remove dynamic objects in each frame. Compared with the traditional whole video processing method, frame-by-frame processing can analyze the changes in the video more meticulously, avoid missing any dynamic objects, and improve the accuracy of static scene videos.

[0071] In some embodiments, generating a pollution heatmap based on the static scene video includes the following steps:

[0072] The historical moment when the last cleaning of the area to be cleaned was completed is obtained, and a target video is extracted from the static scene video based on the historical moment; wherein, the target video is the video in the static scene video corresponding to the historical moment and the current moment; specifically, the historical moment is obtained from the database corresponding to the area to be cleaned.

[0073] The target video is segmented into frames to obtain a second target video frame sequence;

[0074] For each second target video frame in the second target video frame sequence, the cleanable ground area corresponding to the second target video frame is segmented to obtain multiple sub-cleanable ground areas;

[0075] For each of the sub-cleanable ground areas, the pollution levels corresponding to the sub-cleanable ground areas in each of the second target video frames are fused to obtain the target pollution level corresponding to the sub-cleanable ground area; specifically, for each of the second target video frames, the pollution level corresponding to the sub-cleanable ground area in the second target video frame is determined based on the pollution type corresponding to the sub-cleanable ground area in the second target video frame, and the highest pollution level among the pollution levels corresponding to the sub-cleanable ground area is determined as the target pollution level corresponding to the sub-cleanable ground area;

[0076] For each of the aforementioned sub-cleanable ground areas, the sub-cleanable ground areas are rendered based on the target pollution level corresponding to the sub-cleanable ground area; the rendered sub-cleanable ground areas constitute the pollution heat map. Specifically, the rendering color corresponding to the target pollution level is obtained from the database, and the sub-cleanable ground areas are rendered based on the rendering color.

[0077] The method provided in this embodiment, on the one hand, obtains the historical moment when the cleaning of the area to be cleaned was completed last time, and extracts the target video from the static scene video based on the historical moment, effectively capturing the environmental state of the area to be cleaned during a specific time period, which helps to improve the accuracy of the cleaning path. On the other hand, for each of the sub-cleanable ground areas, the pollution level corresponding to the sub-cleanable ground area in each of the second target video frames is fused to obtain the target pollution level corresponding to the sub-cleanable ground area. This can accurately assess the pollution status of each sub-cleanable ground area and assign the most appropriate pollution level to each sub-cleanable ground area, ensuring the effectiveness of the pollution heat map, thereby helping to improve cleaning efficiency.

[0078] In some embodiments, generating an initial cleaning path based on the contamination heat map includes the following steps:

[0079] The shortest travel path for the sweeper is generated based on the current position of the sweeper and the corresponding positions of each of the sub-cleanable ground areas, and the cleaning rate of the path segment corresponding to each sub-cleanable ground area is marked on the shortest travel path; it can be understood that the pollution level of the sub-cleanable ground area is inversely proportional to its corresponding cleaning rate.

[0080] The method provided in this embodiment, on the one hand, marks the corresponding cleaning rate for each sub-cleanable ground area based on the generated shortest travel path, enabling reasonable adjustment of the cleaning rate according to the degree of contamination of each sub-cleanable ground area. On the other hand, the setting that the contamination level of the sub-cleanable ground area is inversely proportional to its corresponding cleaning rate ensures that when the contamination level is high, the sweeper will slow down the cleaning speed to ensure thorough cleaning of the area and avoid omissions, while the cleaning rate can be increased for areas with lighter contamination, thereby saving time and improving efficiency.

[0081] In some embodiments, the multi-source data information includes the operational data information of each dynamic object within the target area during a preset time period. The preset time period is a time period connected to and preceding the current time. The step of constructing a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information includes the following steps:

[0082] For each dynamic object, a predicted running trajectory of the dynamic object is generated based on the corresponding running data information of the dynamic object within the cleaning time period, and it is determined whether the predicted running trajectory passes through the area to be cleaned. If it does, the dynamic object is determined to be the target dynamic object. The running data information includes at least the historical running trajectory and speed of the dynamic object within the preset time period. Specifically, the running data information and the cleaning time period are input into a preset motion trajectory prediction model to obtain the predicted running trajectory. The motion trajectory prediction model is a pre-trained neural network model.

[0083] For each of the target dynamic objects, the predicted running trajectory of the target dynamic object corresponding to the area to be cleaned is segmented to obtain multiple running nodes, and the target time corresponding to each running node is determined, and the semantics corresponding to the target dynamic object are marked at each running node; wherein, the time difference between the target times corresponding to two adjacent moving nodes is no greater than 0.1s;

[0084] For each target time, a dynamic environmental semantic map corresponding to the target time is generated based on the running node corresponding to the target time;

[0085] Based on the target time corresponding to each dynamic environment semantic map, the dynamic environment semantic maps are arranged in sequence to obtain the dynamic environment semantic map sequence.

[0086] The method provided in this embodiment can predict dynamic factors that affect the area to be cleaned in advance, which helps to avoid the sudden appearance of dynamic objects that may interfere with the cleaning path and improves the adaptability of path planning.

[0087] In some embodiments, optimizing the initial cleaning path based on the dynamic environmental semantic map sequence to obtain the target cleaning path includes:

[0088] Each target time is determined to correspond to a cleaning node on the initial cleaning path; wherein, the cleaning node refers to the area occupied by the sweeper at the target time.

[0089] For each target time, the dynamic objects in the dynamic environment semantic map corresponding to the target time are dilated. It is then determined whether the cleaning node corresponding to the target time overlaps with any of the dilated dynamic objects. If they overlap, the cleaning node is adjusted so that the cleaning area does not overlap with any of the dilated dynamic objects. The initial cleaning path after the cleaning node adjustment is the target cleaning path. It should be noted that the method for adjusting the cleaning node is to determine the minimum displacement required for the cleaning node to avoid overlapping with any of the target dynamic objects, and to move the cleaning node based on this minimum displacement.

[0090] The method provided in this embodiment helps to effectively avoid conflicts between the sweeper and moving objects, thereby improving cleaning efficiency.

[0091] Please see Figure 2 , Figure 2 This is a schematic block diagram of the automatic navigation control system 100 for the unmanned sweeper provided in this application embodiment, as shown below. Figure 2 As shown in the embodiment of this application, the automatic navigation control system 100 for the unmanned sweeper includes:

[0092] The generation module 110 is used to acquire historical monitoring videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical monitoring videos, and predict the cleaning time period corresponding to the initial cleaning path.

[0093] The construction module 120 is used to acquire multi-source data information of the target area and construct a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein the area to be cleaned is included in the target area.

[0094] The optimization processing module 130 is used to optimize the initial cleaning path based on the dynamic environment semantic map sequence to obtain the target cleaning path.

[0095] The control module 140 controls the sweeper to perform the cleaning task based on the target cleaning path.

[0096] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device and each module described above can be referred to the process in the aforementioned embodiment of the automatic navigation control method for unmanned sweeping vehicles, and will not be repeated here.

[0097] The automatic navigation control system 100 for the unmanned sweeper provided in the above embodiments can be implemented as a computer program, which can be used in, for example... Figure 3 The terminal device 200 shown is running on it.

[0098] Please see Figure 3 , Figure 3 The present invention provides a schematic block diagram of the structure of a terminal device 200. The terminal device 200 includes a processor 201 and a memory 202, which are connected via a device bus 203. The memory 202 may include a non-volatile storage medium and internal memory.

[0099] The non-volatile storage medium can store a computer program. The computer program includes program instructions, which, when executed by the processor 201, cause the processor 201 to perform any of the above-mentioned automatic navigation control methods for the unmanned sweeper.

[0100] The processor 201 provides computing and control capabilities to support the operation of the entire terminal device 200.

[0101] The internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor 201, the processor 201 can execute any of the above-mentioned automatic navigation control methods for unmanned sweeping vehicles.

[0102] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal device 200 involved in the present application. The specific terminal device 200 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0103] It should be understood that processor 201 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, the general-purpose processor can be a microprocessor or any conventional processor.

[0104] In some embodiments, the processor 201 is configured to run a computer program stored in memory to perform the following steps:

[0105] The system acquires historical surveillance video of the area to be cleaned, generates an initial cleaning path for the area based on the historical surveillance video, and predicts the cleaning time period corresponding to the initial cleaning path.

[0106] Acquire multi-source data information of the target area, and construct a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein, the area to be cleaned is included in the target area;

[0107] The initial cleaning path is optimized based on the dynamic environmental semantic map sequence to obtain the target cleaning path;

[0108] The sweeper is controlled to perform the cleaning task based on the target cleaning path.

[0109] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the terminal device 200 described above can be referred to the corresponding process of the automatic navigation control method of the aforementioned unmanned sweeper, and will not be repeated here.

[0110] This application also provides a computer-readable storage medium storing a computer program that, when executed by one or more processors, enables the one or more processors to implement the automatic navigation control method for the unmanned sweeper provided in this application.

[0111] The computer-readable storage medium can be an internal storage unit of the terminal device 200 in the aforementioned embodiments, such as a hard disk or memory of the terminal device 200. The computer-readable storage medium can also be an external storage device of the terminal device 200, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided with the terminal device 200.

[0112] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An automatic navigation control method for an unmanned sweeping vehicle, characterized in that, include: The system acquires historical surveillance video of the area to be cleaned, generates an initial cleaning path for the area based on the historical surveillance video, and predicts the cleaning time period corresponding to the initial cleaning path. Acquire multi-source data information of the target area, and construct a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein, the area to be cleaned is included in the target area; The initial cleaning path is optimized based on the dynamic environmental semantic map sequence to obtain the target cleaning path; The sweeper is controlled to perform the sweeping task based on the target sweeping path; The multi-source data information includes the operational data information of each dynamic object within the target area during a preset time period. The preset time period is a time period connected to and preceding the current time. The step of constructing a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information includes: For each dynamic object, a predicted running trajectory of the dynamic object is generated based on the running data information corresponding to the dynamic object within the cleaning time period, and it is determined whether the predicted running trajectory passes through the area to be cleaned. If it does, the dynamic object is determined to be the target dynamic object. For each of the target dynamic objects, the predicted running trajectory of the target dynamic object in the area to be cleaned is segmented to obtain multiple running nodes, and the target time corresponding to each running node is determined, and the semantics corresponding to the target dynamic object are marked at each running node. For each target time, a dynamic environmental semantic map corresponding to the target time is generated based on the running node corresponding to the target time; Based on the target time corresponding to each dynamic environment semantic map, the dynamic environment semantic maps are arranged in sequence to obtain the dynamic environment semantic map sequence.

2. The automatic navigation control method for the unmanned sweeper according to claim 1, characterized in that, The step of generating an initial cleaning path for the area to be cleaned based on the historical surveillance video, and predicting the cleaning time period corresponding to the initial cleaning path, includes: Acquire multi-camera surveillance videos of the area to be cleaned within a preset historical time period, and stitch the multi-camera surveillance videos into a panoramic surveillance video; Dynamic object removal processing is performed on the panoramic monitoring video to obtain a static scene video corresponding to the area to be cleaned. A pollution heat map is generated based on the static scene video; An initial cleaning path is generated based on the pollution heat map; the initial cleaning path includes multiple sub-cleanable ground areas, and each sub-cleanable ground area has a corresponding cleaning rate. The cleaning time period is generated based on the cleaning rate corresponding to each sub-cleanable ground area.

3. The automatic navigation control method for the unmanned sweeper according to claim 2, characterized in that, The process of removing dynamic objects based on the panoramic monitoring video to obtain a static scene video corresponding to the area to be cleaned includes: The panoramic surveillance video is processed into frames to obtain a video frame sequence; Dynamic object removal processing is performed on each video frame of the video frame sequence in sequence to obtain a first target video frame sequence, and the static scene video is generated based on the first target video frame sequence.

4. The automatic navigation control method for the unmanned sweeper according to claim 2, characterized in that, The generation of the pollution heat map based on the static scene video includes: Obtain the historical moment when the last cleaning of the area to be cleaned was completed, and extract a target video from the static scene video based on the historical moment; wherein, the target video is the video corresponding to the historical moment and the current moment in the static scene video; The target video is segmented into frames to obtain a second target video frame sequence; For each second target video frame in the second target video frame sequence, the cleanable ground area corresponding to the second target video frame is segmented to obtain multiple sub-cleanable ground areas; For each of the sub-cleanable ground areas, the pollution levels corresponding to the sub-cleanable ground areas in each of the second target video frames are fused to obtain the target pollution level corresponding to the sub-cleanable ground areas; For each of the sub-cleanable ground areas, the sub-cleanable ground areas are rendered based on the target pollution level corresponding to the sub-cleanable ground area; the rendered sub-cleanable ground areas constitute the pollution heat map.

5. The automatic navigation control method for the unmanned sweeper according to claim 1, characterized in that, The optimization of the initial cleaning path based on the dynamic environmental semantic map sequence to obtain the target cleaning path includes: Determine the cleaning node corresponding to each target time on the initial cleaning path; For each target time, the target dynamic objects in the dynamic environment semantic map corresponding to the target time are expanded, and it is determined whether the cleaning node corresponding to the target time coincides with any target dynamic object after expansion. If they coincide, the cleaning node is adjusted so that the cleaning area does not coincide with any target dynamic object after expansion. The initial cleaning path after the cleaning node adjustment is the target cleaning path.

6. An automatic navigation control system for an unmanned sweeper, characterized in that, include: The generation module is used to acquire historical monitoring videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical monitoring videos, and predict the cleaning time period corresponding to the initial cleaning path. A construction module is used to acquire multi-source data information of the target area and construct a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein, the area to be cleaned is included in the target area; An optimization processing module is used to optimize the initial cleaning path based on the dynamic environmental semantic map sequence to obtain the target cleaning path; The control module controls the sweeper to perform the cleaning task based on the target cleaning path; The multi-source data information includes the operational data information of each dynamic object within the target area during a preset time period. The preset time period is a time period connected to and preceding the current time. The step of constructing a dynamic environmental semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information includes: For each dynamic object, a predicted running trajectory of the dynamic object is generated based on the running data information corresponding to the dynamic object within the cleaning time period, and it is determined whether the predicted running trajectory passes through the area to be cleaned. If it does, the dynamic object is determined to be the target dynamic object. For each of the target dynamic objects, the predicted running trajectory of the target dynamic object in the area to be cleaned is segmented to obtain multiple running nodes, and the target time corresponding to each running node is determined, and the semantics corresponding to the target dynamic object are marked at each running node. For each target time, a dynamic environmental semantic map corresponding to the target time is generated based on the running node corresponding to the target time; Based on the target time corresponding to each dynamic environment semantic map, the dynamic environment semantic maps are arranged in sequence to obtain the dynamic environment semantic map sequence.

Images