Automatic navigation control method and system of unmanned sweeper

Abstract

The invention relates to the technical field of automatic navigation, and provides an automatic navigation control method and system of an unmanned sweeper. The method comprises the steps of obtaining a historical monitoring video of a to-be-cleaned area, generating an initial cleaning path of the to-be-cleaned area based on the historical monitoring video, and predicting a cleaning time period corresponding to the initial cleaning path; multi-source data information of a target area is acquired, and a dynamic environment semantic map sequence corresponding to the area to be cleaned in the cleaning time period is constructed based on the multi-source data information; wherein the to-be-cleaned area is included in the target area; performing optimization processing on the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path; and controlling the sweeper to execute a sweeping task based on the target sweeping path. The method is helpful for improving the adaptive capacity of the sweeper in a complex environment.

Description

Technical Field

[0001] The present application relates to the field of automatic navigation technology, and in particular to an automatic navigation control method and system for an unmanned sweeper. Background Art

[0002] With the continuous development of science and technology, the application of autonomous driving technology is becoming increasingly widespread in various fields, especially in the field of intelligent cleaning equipment. Autonomous sweepers have become a widely used cleaning equipment in homes, shopping malls, office buildings, and other places. With their automated and intelligent features, autonomous sweepers can independently plan cleaning paths and efficiently complete cleaning tasks, greatly reducing the labor intensity of manual cleaning and improving cleaning efficiency and quality.

[0003] However, the path planning technology of most unmanned sweepers currently relies mainly on simple map construction. Its path planning is usually relatively static and lacks the ability to adapt to dynamic environments. This method has poor cleaning effects in complex dynamic environments. Summary of the Invention

[0004] The present application provides an automatic navigation control method and system for an unmanned sweeping vehicle to solve the problems raised by the above background technology.

[0005] In a first aspect, the present application provides an automatic navigation control method for an unmanned sweeping vehicle, comprising: Obtain historical surveillance videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical surveillance videos, and predict a cleaning time period corresponding to the initial cleaning path; Acquire multi-source data information of a target area, and construct a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein the area to be cleaned is included in the target area; Optimizing the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path; The sweeper is controlled to perform a cleaning task based on the target cleaning path.

[0006] In one possible implementation, generating an initial cleaning path for the area to be cleaned based on the historical surveillance video and predicting a cleaning time period corresponding to the initial cleaning path includes: Acquire multi-camera surveillance videos of the area to be cleaned within a preset historical time period, and stitch the multi-camera surveillance videos into a panoramic surveillance video; Performing dynamic object removal processing based on the panoramic surveillance video to obtain a static scene video corresponding to the area to be cleaned; generating a pollution heat map based on the static scene video; generating an initial cleaning path based on the pollution heat map; the initial cleaning path includes a plurality of sub-cleanable ground areas, each of the sub-cleanable ground areas being set with a corresponding cleaning rate; The cleaning time period is generated based on the cleaning rates corresponding to the sub-cleanable floor areas.

[0007] In one possible implementation, performing dynamic object removal processing based on the panoramic surveillance video to obtain a static scene video corresponding to the area to be cleaned includes: Performing frame processing on the panoramic surveillance video to obtain a video frame sequence; Dynamic object removal processing is performed on each video frame of the video frame sequence in sequence to obtain a first target video frame sequence, and the static scene video is generated based on the first target video frame sequence.

[0008] In a possible implementation, generating a pollution heat map based on the static scene video includes: Obtaining the historical moment when the cleaning of the area to be cleaned was last completed, and intercepting a target video in the static scene video based on the historical moment; wherein the target video is a video corresponding to the historical moment and the current moment in the static scene video; Performing frame processing on the target video to obtain a second target video frame sequence; For each second target video frame in the second target video frame sequence, segmenting the cleanable ground area corresponding to the second target video frame to obtain a plurality of sub-cleanable ground areas; For each of the sub-cleanable ground areas, the pollution levels corresponding to the sub-cleanable ground areas in each of the second target video frames are fused to obtain a target pollution level corresponding to the sub-cleanable ground area; For each of the sub-cleanable floor areas, the sub-cleanable floor area is rendered based on the target pollution level corresponding to the sub-cleanable floor area; and each of the rendered sub-cleanable floor areas constitutes the pollution heat map.

[0009] In one possible implementation, the multi-source data information includes operating data information of each dynamic object in the target area within a preset time period, where the preset time period is a time period connected to and before the current moment. Constructing a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information includes: For each dynamic object, a predicted running trajectory of the dynamic object within the cleaning time period is generated based on the running data information corresponding to the dynamic object, and it is determined whether the predicted running trajectory passes through the area to be cleaned. If so, the dynamic object is determined to be a target dynamic object; For each target dynamic object, segment the corresponding portion of the predicted running trajectory of the target dynamic object in the area to be cleaned to obtain multiple running nodes, determine the target time corresponding to each running node, and mark the semantics corresponding to the target dynamic object at each running node; For each target moment, generating a dynamic environment semantic map corresponding to the target moment based on the running node corresponding to the target moment; The dynamic environment semantic maps are arranged in sequence based on the target moments corresponding to the dynamic environment semantic maps to obtain the dynamic environment semantic map sequence.

[0010] In a possible implementation, the optimizing the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path includes: Determine the cleaning nodes corresponding to the target moments on the initial cleaning path respectively; For each target moment, each target dynamic object in the dynamic environment semantic map corresponding to the target moment is expanded, and it is determined whether the cleaning node corresponding to the target moment coincides with any target dynamic object after the expansion process. If so, the cleaning node is adjusted so that the cleaning area does not coincide with any target dynamic object after the expansion process; wherein, the initial cleaning path after the cleaning node adjustment is the target cleaning path.

[0011] In a second aspect, the present application provides an automatic navigation control system for an unmanned sweeping vehicle, comprising: A generation module is used to obtain historical monitoring videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical monitoring videos, and predict a cleaning time period corresponding to the initial cleaning path; A construction module is used to obtain multi-source data information of a target area, and construct a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein the area to be cleaned is included in the target area; An optimization processing module, configured to optimize the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path; The control module controls the sweeper to perform a cleaning task based on the target cleaning path.

[0012] The present application provides an automatic navigation control method and system for an unmanned sweeper. The method comprises: obtaining historical surveillance videos of an area to be cleaned, generating an initial cleaning path for the area to be cleaned based on the historical surveillance videos, and predicting a cleaning time period corresponding to the initial cleaning path; obtaining multi-source data information of a target area, and constructing a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein the area to be cleaned is included in the target area; optimizing the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path; and controlling the sweeper to perform a cleaning task based on the target cleaning path. The method provided in this embodiment, on the one hand, by acquiring historical monitoring videos of the area to be cleaned, generating an initial cleaning path of the area to be cleaned based on the historical monitoring videos, and predicting the cleaning time period corresponding to the initial cleaning path, can enable the sweeper to fully perceive the area to be cleaned before cleaning; on the other hand, by acquiring multi-source data information of the target area, and constructing a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information, it realizes the prediction of the dynamic environment corresponding to the area to be cleaned within the future cleaning time period, which helps to improve the adaptability of the sweeper in complex environments; on the other hand, by optimizing the initial cleaning path based on the dynamic environment semantic map sequence to obtain the target cleaning path, the sweeper can avoid obstacles as much as possible during the sweeping process, prevent the sweeper from colliding with obstacles, and improve the sweeping efficiency of the sweeper. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] Figure 1 A flow chart of an automatic navigation control method for an unmanned sweeping vehicle provided in an embodiment of the present application; Figure 2 A schematic block diagram of the structure of the automatic navigation control system of the unmanned sweeping vehicle provided in an embodiment of the present application; Figure 3 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0016] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0017] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0018] It should be further understood that the term "and / or" used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0019] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] See also Figure 1 , Figure 1 A flow chart of the automatic navigation control method of the unmanned sweeping vehicle provided in the embodiment of the present application is shown as follows: Figure 1 As shown, the automatic navigation control method of the unmanned sweeping vehicle provided in the embodiment of the present application includes steps S1 to S4.

[0021] Step S1: Obtain historical surveillance videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical surveillance videos, and predict a cleaning time period corresponding to the initial cleaning path.

[0022] Step S2: Acquire multi-source data information of the target area, and construct a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein the area to be cleaned is included in the target area.

[0023] Step S3: Optimize the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path.

[0024] Step S4: controlling the sweeper to perform the cleaning task based on the target cleaning path.

[0025] It should be noted that the method provided in this embodiment is used to clean indoor spaces such as homes, shopping malls, and office buildings.

[0026] This embodiment specifically includes: As described in step S1 above, historical surveillance videos of the area to be cleaned are obtained, and an initial cleaning path for the area to be cleaned is generated based on the historical surveillance videos, and a cleaning time period corresponding to the initial cleaning path is predicted. Specifically, step S1 includes: obtaining multi-camera surveillance videos of the area to be cleaned within a preset historical time period, and splicing the multi-camera surveillance videos into a panoramic surveillance video; performing dynamic object removal processing based on the panoramic surveillance video to obtain a static scene video corresponding to the area to be cleaned; generating a pollution heat map based on the static scene video; generating an initial cleaning path based on the pollution heat map; the initial cleaning path includes multiple sub-cleanable ground areas, each of which is provided with a corresponding cleaning rate; and generating the cleaning time period based on the cleaning rate corresponding to each sub-cleanable ground area.

[0027] As described in step S2 above, multi-source data information of the target area is obtained, and a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period is constructed based on the multi-source data information; wherein the area to be cleaned is included in the target area. Specifically, the multi-source data information includes the operation data information of each dynamic object in the target area within a preset time period, and the preset time period is a time period connected to the current moment and before the current moment. Step S2 includes: for each dynamic object, based on the operation data information corresponding to the dynamic object, generating a predicted operation trajectory corresponding to the dynamic object within the cleaning time period, and judging whether the predicted operation trajectory passes through the area to be cleaned, and if so, determining that the dynamic object is a target dynamic object; for each target dynamic object, segmenting the corresponding part of the predicted operation trajectory of the target dynamic object in the area to be cleaned to obtain multiple operation nodes, and determining the target moment corresponding to each operation node, and marking the semantics corresponding to the target dynamic object at each operation node; for each target moment, generating a dynamic environment semantic map corresponding to the target moment based on the operation node corresponding to the target moment; arranging each dynamic environment semantic map in sequence based on the target moment corresponding to each dynamic environment semantic map to obtain the dynamic environment semantic map sequence.

[0028] As described in step S3 above, the initial cleaning path is optimized based on the dynamic environment semantic map sequence to obtain the target cleaning path. Specifically, step S3 includes: respectively determining the cleaning nodes corresponding to each target moment on the initial cleaning path; for each target moment, performing expansion processing on each target dynamic object in the dynamic environment semantic map corresponding to the target moment, and determining whether the cleaning node corresponding to the target moment coincides with any target dynamic object after the expansion processing; if so, adjusting the cleaning node so that the cleaning area does not coincide with any target dynamic object after the expansion processing; wherein, the initial cleaning path after the cleaning node adjustment is the target cleaning path.

[0029] As described in step S4 above, the sweeper is controlled to perform the cleaning task based on the target cleaning path. Specifically, first, a control instruction corresponding to the target cleaning path is generated, and the sweeper is controlled to perform the cleaning task based on the control instruction.

[0030] The method provided in this embodiment, on the one hand, by acquiring historical monitoring videos of the area to be cleaned, generating an initial cleaning path of the area to be cleaned based on the historical monitoring videos, and predicting the cleaning time period corresponding to the initial cleaning path, can enable the sweeper to fully perceive the area to be cleaned before cleaning; on the other hand, by acquiring multi-source data information of the target area, and constructing a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information, it realizes the prediction of the dynamic environment corresponding to the area to be cleaned within the future cleaning time period, which helps to improve the adaptability of the sweeper in complex environments; on the other hand, by optimizing the initial cleaning path based on the dynamic environment semantic map sequence to obtain the target cleaning path, the sweeper can avoid obstacles as much as possible during the sweeping process, prevent the sweeper from colliding with obstacles, and improve the sweeping efficiency of the sweeper.

[0031] In some embodiments, generating an initial cleaning path for the area to be cleaned based on the historical surveillance video and predicting a cleaning time period corresponding to the initial cleaning path includes the following steps: Acquire multi-camera surveillance videos of the area to be cleaned within a preset historical time period, and stitch the multi-camera surveillance videos into a panoramic surveillance video; wherein the preset historical time period is at least two days before the current time, and specifically, stitch the multi-camera surveillance videos based on the SIFT feature matching algorithm to obtain the panoramic surveillance video; Performing dynamic object removal processing based on the panoramic surveillance video to obtain a static scene video corresponding to the area to be cleaned; generating a pollution heat map based on the static scene video; generating an initial cleaning path based on the pollution heat map; the initial cleaning path includes a plurality of sub-cleanable ground areas, each of the sub-cleanable ground areas being set with a corresponding cleaning rate; The cleaning time period is generated based on the cleaning rate corresponding to each sub-cleanable ground area; specifically, for each sub-cleanable ground area, the ratio of the cleaning length corresponding to the sub-cleanable ground area to the cleaning rate corresponding to the sub-cleanable ground area is determined as the cleaning time period corresponding to the sub-cleanable ground area, and each cleaning time period is added together to obtain the target cleaning time period, and the current moment is used as the initial moment to determine the cleaning time period corresponding to the target cleaning time period.

[0032] The method provided in this embodiment, on the one hand, compared with the traditional single-perspective monitoring method, the use of multi-camera stitching panoramic video can more accurately obtain the overall picture of the area to be cleaned, improve the depth of cognition of the area to be cleaned, and thus provide an effective data basis for subsequent path planning and improvement of cleaning efficiency. On the other hand, by performing dynamic object removal processing based on the panoramic monitoring video, a static scene video corresponding to the area to be cleaned is obtained, and the interference of dynamic objects in the environment is eliminated. Only static scenes related to the cleaning task are retained, which reduces the noise factors in the environment and helps to improve the accuracy of path planning. On the other hand, by generating a pollution heat map based on the static scene video, the pollution degree and cleaning requirements of different sub-cleanable ground areas can be clearly identified, which helps to improve the cleaning quality and efficiency.

[0033] In some embodiments, the process of removing dynamic objects based on the panoramic surveillance video to obtain a static scene video corresponding to the area to be cleaned includes the following steps: Performing frame processing on the panoramic surveillance video to obtain a video frame sequence; Dynamic object removal processing is performed on each video frame of the video frame sequence in sequence to obtain a first target video frame sequence, and the static scene video is generated based on the first target video frame sequence; specifically, for each video frame of the video frame sequence, semantic recognition is performed on each object in the video frame, and dynamic objects in the video frame are removed based on the results of the semantic recognition.

[0034] The method provided in this embodiment obtains a first target video frame sequence by sequentially performing dynamic object removal processing on each video frame of the video frame sequence, and can accurately identify and remove dynamic objects in each frame. Compared with traditional overall video processing methods, frame-by-frame processing can more carefully analyze changes in the video, avoid missing any dynamic objects, and improve the accuracy of static scene videos.

[0035] In some embodiments, generating a pollution heat map based on the static scene video comprises the following steps: Obtaining the historical moment when the cleaning of the area to be cleaned was last completed, and intercepting a target video in the static scene video based on the historical moment; wherein the target video is a video corresponding to the historical moment and the current moment in the static scene video; specifically, obtaining the historical moment in a database corresponding to the area to be cleaned; Performing frame processing on the target video to obtain a second target video frame sequence; For each second target video frame in the second target video frame sequence, segmenting the cleanable ground area corresponding to the second target video frame to obtain a plurality of sub-cleanable ground areas; For each of the sub-sweepable ground areas, the pollution levels corresponding to the sub-sweepable ground areas in each of the second target video frames are fused to obtain a target pollution level corresponding to the sub-sweepable ground area; specifically, for each of the second target video frames, the pollution level corresponding to the sub-sweepable ground area in the second target video frame is determined based on the pollution type corresponding to the sub-sweepable ground area in the second target video frame, and the highest pollution level among the pollution levels corresponding to the sub-sweepable ground area is determined as the target pollution level corresponding to the sub-sweepable ground area; For each of the sub-cleanable floor areas, the sub-cleanable floor area is rendered based on the target pollution level corresponding to the sub-cleanable floor area; the rendered sub-cleanable floor areas constitute the pollution heat map. Specifically, the rendering color corresponding to the target pollution level is obtained from a database, and the sub-cleanable floor area is rendered based on the rendering color.

[0036] The method provided in this embodiment, on the one hand, obtains the historical moment when the cleaning of the area to be cleaned was completed for the last time, and intercepts the target video in the static scene video based on the historical moment, effectively capturing the environmental status of the area to be cleaned in a specific time period, which helps to improve the accuracy of the cleaning path; on the other hand, for each of the sub-cleanable ground areas, the pollution levels corresponding to the sub-cleanable ground areas in each of the second target video frames are fused to obtain the target pollution level corresponding to the sub-cleanable ground area, which can accurately evaluate the pollution situation of each sub-cleanable ground area and assign the most appropriate pollution level to each sub-cleanable ground area, thereby ensuring the effectiveness of the pollution heat map, and thus helping to improve the cleaning efficiency.

[0037] In some embodiments, generating an initial cleaning path based on the pollution heat map includes the following steps: The shortest driving path of the sweeper is generated based on the current position of the sweeper and the position corresponding to each of the sub-cleanable ground areas, and the cleaning rate of the path segment corresponding to each sub-cleanable ground area is marked on the shortest driving path; it can be understood that the pollution level of the sub-cleanable ground area is inversely proportional to its corresponding cleaning rate.

[0038] The method provided in this embodiment, on the one hand, marks the corresponding cleaning rate for the path segment of each sub-sweepable ground area on the basis of generating the shortest driving path, so that the cleaning rate can be reasonably adjusted according to the pollution degree of each sub-sweepable ground area. On the other hand, the pollution level of the sub-sweepable ground area and its corresponding cleaning rate are set inversely proportional. When the pollution level is high, the sweeper will slow down the cleaning speed to ensure that the area is thoroughly cleaned to avoid omissions, while the cleaning rate can be increased for areas with less pollution, thereby saving time and improving efficiency.

[0039] In some embodiments, the multi-source data information includes operating data information of each dynamic object in the target area within a preset time period, where the preset time period is a time period connected to and before the current moment. Constructing a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information includes the following steps: For each dynamic object, a predicted running trajectory corresponding to the dynamic object within the cleaning time period is generated based on the running data information corresponding to the dynamic object, and it is determined whether the predicted running trajectory passes through the area to be cleaned. If so, the dynamic object is determined to be a target dynamic object; wherein the running data information at least includes the historical running trajectory and speed of the dynamic object within the preset time period. Specifically, the running data information and the cleaning time period are input into a preset motion trajectory prediction model to obtain the predicted running trajectory, and the motion trajectory prediction model is a pre-trained neural network model; For each target dynamic object, the corresponding portion of the predicted running trajectory of the target dynamic object in the area to be cleaned is segmented to obtain multiple running nodes, and the target time corresponding to each running node is determined, and the semantics corresponding to the target dynamic object is marked at each running node; wherein the time difference between the target times corresponding to two adjacent motion nodes is no more than 0.1s; For each target moment, generating a dynamic environment semantic map corresponding to the target moment based on the running node corresponding to the target moment; The dynamic environment semantic maps are arranged in sequence based on the target moments corresponding to the dynamic environment semantic maps to obtain the dynamic environment semantic map sequence.

[0040] The method provided in this embodiment can predict in advance the dynamic factors that affect the area to be cleaned, which helps to avoid the sudden appearance of dynamic objects that may interfere with the cleaning path and improves the adaptability of path planning.

[0041] In some embodiments, the optimizing the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path includes: Determine the cleaning nodes corresponding to the initial cleaning path at each target time; wherein the cleaning nodes refer to the areas occupied by the sweeper at the target time; For each target moment, each target dynamic object in the dynamic environment semantic map corresponding to the target moment is expanded, and it is determined whether the cleaning node corresponding to the target moment coincides with any target dynamic object after the expansion process. If so, the cleaning node is adjusted so that the cleaning area does not coincide with any target dynamic object after the expansion process; wherein, the initial cleaning path after the cleaning node adjustment is the target cleaning path. It should be noted that the method for adjusting the cleaning node is to determine the minimum displacement at which the cleaning node does not coincide with each target dynamic object, and to move the cleaning node based on the minimum displacement.

[0042] The method provided in this embodiment helps to effectively avoid collisions between the sweeper and dynamic objects, thereby improving cleaning efficiency.

[0043] See also Figure 2 , Figure 2 The schematic block diagram of the structure of the automatic navigation control system 100 of the unmanned sweeping vehicle provided in the embodiment of the present application is as follows: Figure 2 As shown, the automatic navigation control system 100 of the unmanned sweeping vehicle provided in the embodiment of the present application includes: The generation module 110 is used to obtain historical monitoring videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical monitoring videos, and predict a cleaning time period corresponding to the initial cleaning path.

[0044] The construction module 120 is used to obtain multi-source data information of the target area and construct a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein the area to be cleaned is included in the target area.

[0045] The optimization processing module 130 is used to optimize the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path.

[0046] The control module 140 controls the sweeper to perform the cleaning task based on the target cleaning path.

[0047] It should be noted that, those skilled in the art will clearly understand that for the sake of convenience and brevity in description, the specific working processes of the above-described devices and modules can refer to the processes in the aforementioned embodiment of the automatic navigation control method for the unmanned sweeping vehicle, and will not be repeated here.

[0048] The automatic navigation control system 100 of the unmanned sweeping vehicle provided in the above embodiment can be implemented in the form of a computer program. The computer program can be used in Figure 3 The system runs on the terminal device 200 shown.

[0049] See also Figure 3 , Figure 3 This is a schematic block diagram of the structure of a terminal device 200 provided in an embodiment of the present application. The terminal device 200 includes a processor 201 and a memory 202. The processor 201 and the memory 202 are connected via a device bus 203, wherein the memory 202 may include a non-volatile storage medium and an internal memory.

[0050] The non-volatile storage medium can store a computer program. The computer program includes program instructions, and when the program instructions are executed by the processor 201, the processor 201 can execute any of the above-mentioned automatic navigation control methods for the unmanned sweeping vehicle.

[0051] The processor 201 is used to provide computing and control capabilities to support the operation of the entire terminal device 200.

[0052] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor 201, the processor 201 can execute any of the above-mentioned automatic navigation control methods for the unmanned sweeping vehicle.

[0053] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the terminal device 200 involved in the solution of the present application. The specific terminal device 200 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0054] It should be understood that the processor 201 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0055] In some embodiments, the processor 201 is configured to execute a computer program stored in the memory to implement the following steps: Obtain historical surveillance videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical surveillance videos, and predict a cleaning time period corresponding to the initial cleaning path; Acquire multi-source data information of a target area, and construct a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein the area to be cleaned is included in the target area; Optimizing the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path; The sweeper is controlled to perform a cleaning task based on the target cleaning path.

[0056] It should be noted that, those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the terminal device 200 described above can refer to the corresponding process of the automatic navigation control method of the aforementioned unmanned sweeping vehicle, and will not be repeated here.

[0057] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the one or more processors implement the automatic navigation control method of the unmanned sweeping vehicle provided in the embodiment of the present application.

[0058] The computer-readable storage medium may be an internal storage unit of the terminal device 200 in the aforementioned embodiment, such as a hard disk or memory of the terminal device 200. The computer-readable storage medium may also be an external storage device of the terminal device 200, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped with the terminal device 200.

[0059] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An automatic navigation control method for an unmanned sweeping vehicle, characterized in that: include: Obtain historical surveillance videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical surveillance videos, and predict a cleaning time period corresponding to the initial cleaning path; Acquire multi-source data information of a target area, and construct a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein the area to be cleaned is included in the target area; Optimizing the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path; The sweeper is controlled to perform a cleaning task based on the target cleaning path.

2. The automatic navigation control method of the unmanned sweeping vehicle according to claim 1, characterized in that: Generating an initial cleaning path for the area to be cleaned based on the historical surveillance video and predicting a cleaning time period corresponding to the initial cleaning path includes: Acquire multi-camera surveillance videos of the area to be cleaned within a preset historical time period, and stitch the multi-camera surveillance videos into a panoramic surveillance video; Performing dynamic object removal processing based on the panoramic surveillance video to obtain a static scene video corresponding to the area to be cleaned; generating a pollution heat map based on the static scene video; generating an initial cleaning path based on the pollution heat map; the initial cleaning path includes a plurality of sub-cleanable ground areas, each of the sub-cleanable ground areas being set with a corresponding cleaning rate; The cleaning time period is generated based on the cleaning rates corresponding to the sub-cleanable floor areas.

3. The automatic navigation control method of the unmanned sweeping vehicle according to claim 2, characterized in that: The process of removing dynamic objects based on the panoramic surveillance video to obtain a static scene video corresponding to the area to be cleaned includes: Performing frame processing on the panoramic surveillance video to obtain a video frame sequence; Dynamic object removal processing is performed on each video frame of the video frame sequence in sequence to obtain a first target video frame sequence, and the static scene video is generated based on the first target video frame sequence.

4. The automatic navigation control method of the unmanned sweeping vehicle according to claim 2, characterized in that: Generating a pollution heat map based on the static scene video includes: Obtaining the historical moment when the cleaning of the area to be cleaned was last completed, and intercepting a target video in the static scene video based on the historical moment; wherein the target video is a video corresponding to the historical moment and the current moment in the static scene video; Performing frame processing on the target video to obtain a second target video frame sequence; For each second target video frame in the second target video frame sequence, segmenting the cleanable ground area corresponding to the second target video frame to obtain a plurality of sub-cleanable ground areas; For each of the sub-cleanable ground areas, the pollution levels corresponding to the sub-cleanable ground areas in each of the second target video frames are fused to obtain a target pollution level corresponding to the sub-cleanable ground area; For each of the sub-cleanable floor areas, the sub-cleanable floor area is rendered based on the target pollution level corresponding to the sub-cleanable floor area; and each of the rendered sub-cleanable floor areas constitutes the pollution heat map.

5. The automatic navigation control method of the unmanned sweeping vehicle according to claim 1, characterized in that: The multi-source data information includes operation data information of each dynamic object in the target area within a preset time period, where the preset time period is a time period connected to and before the current moment. The dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period is constructed based on the multi-source data information, including: For each dynamic object, a predicted running trajectory of the dynamic object within the cleaning time period is generated based on the running data information corresponding to the dynamic object, and it is determined whether the predicted running trajectory passes through the area to be cleaned. If so, the dynamic object is determined to be a target dynamic object; For each target dynamic object, segment the corresponding portion of the predicted running trajectory of the target dynamic object in the area to be cleaned to obtain multiple running nodes, determine the target time corresponding to each running node, and mark the semantics corresponding to the target dynamic object at each running node; For each target moment, generating a dynamic environment semantic map corresponding to the target moment based on the running node corresponding to the target moment; The dynamic environment semantic maps are arranged in sequence based on the target moments corresponding to the dynamic environment semantic maps to obtain the dynamic environment semantic map sequence.

6. The automatic navigation control method of the unmanned sweeping vehicle according to claim 5, characterized in that: The optimizing the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path includes: Determine the cleaning nodes corresponding to the target moments on the initial cleaning path respectively; For each target moment, each target dynamic object in the dynamic environment semantic map corresponding to the target moment is expanded, and it is determined whether the cleaning node corresponding to the target moment coincides with any target dynamic object after the expansion process. If so, the cleaning node is adjusted so that the cleaning area does not coincide with any target dynamic object after the expansion process; wherein, the initial cleaning path after the cleaning node adjustment is the target cleaning path.

7. An automatic navigation control system for an unmanned sweeping vehicle, characterized in that: include: A generation module is used to obtain historical monitoring videos of the area to be cleaned, generate an initial cleaning path for the area to be cleaned based on the historical monitoring videos, and predict a cleaning time period corresponding to the initial cleaning path; A construction module is used to obtain multi-source data information of a target area, and construct a dynamic environment semantic map sequence corresponding to the area to be cleaned within the cleaning time period based on the multi-source data information; wherein the area to be cleaned is included in the target area; An optimization processing module, configured to optimize the initial cleaning path based on the dynamic environment semantic map sequence to obtain a target cleaning path; The control module controls the sweeper to perform a cleaning task based on the target cleaning path.

Images