An image processing method, apparatus, electronic device, and storage medium

Abstract

The application relates to an image processing method and device, electronic equipment and storage medium, comprising: acquiring a plurality of first subgraphs corresponding to a target region; in the case that at least one group of repeated subgraph groups exist in the plurality of first subgraphs and the first number of repeated subgraphs included in the repeated subgraph groups is greater than a preset repeated subgraph number, deleting a second number of repeated subgraphs satisfying a preset condition from the repeated subgraph groups based on a quality evaluation parameter of each repeated subgraph in the repeated subgraph groups; and the quality evaluation parameter of each repeated subgraph is determined based on the number of normal image frames and the number of abnormal image frames during construction of each repeated subgraph. The application introduces the quality evaluation parameter of the subgraph, and when it is necessary to delete the repeated subgraphs corresponding to the same region, the repeated subgraph with poor quality is deleted according to the quality evaluation parameter of the repeated subgraph, so that the overall quality of the subgraph is improved, and the quality of the overall map constructed based on the subgraph is ensured.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the popularization of smart technology, cleaning robots and various mobile robots are entering households. Indoor mobile robots are generally equipped with LiDAR and use simultaneous localization and mapping (SLAM) technology for localization and mapping. The mainstream SLAM technology employs a front-end and back-end approach: the front-end uses LiDAR to construct sub-maps, and the back-end adjusts the poses (positions and orientations) between these sub-maps and then stitches them together to form a complete map. As the mobile robot repeatedly moves within the same area (e.g., using a bow sweeper), multiple duplicate sub-maps will appear, increasing the workload of constraint calculations in the back-end. Therefore, to reduce the processing pressure on the back-end, duplicate sub-maps need to be removed.

[0003] Current techniques typically delete the oldest subgraph and retain only the most recent subgraphs. However, deleting subgraphs chronologically cannot guarantee the quality of the overall map built upon them, and may even lead to a deterioration in the overall map quality. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention discloses an image processing method that determines the quality evaluation parameters of a sub-image based on the number of normal and abnormal image frames during its construction. When it is necessary to selectively delete duplicate sub-images corresponding to the same region, a certain number of duplicate sub-images with poor quality are deleted according to the quality evaluation parameters of the duplicate sub-images, thereby improving the overall quality of the sub-images and ensuring the quality of the overall map constructed based on the sub-images.

[0005] To achieve the above objectives, the present invention provides an image processing method, characterized in that it includes:

[0006] Multiple first sub-images corresponding to the target region are obtained; each first sub-image is constructed by the target autonomous mobile device based on multiple image frames acquired;

[0007] If at least one set of repeating subgraphs exists among the plurality of first subgraphs and the first number of repeating subgraphs included in the repeating subgraph set is greater than a preset number of repeating subgraphs, a second number of repeating subgraphs that meet the preset conditions are deleted from the repeating subgraph set based on the quality evaluation parameters of each repeating subgraph in the repeating subgraph set; the quality evaluation parameters of each repeating subgraph are determined based on the number of normal image frames and the number of abnormal image frames of each repeating subgraph during the construction period, and the second number is the difference between the first number and the preset number of repeating subgraphs.

[0008] In an optional embodiment, the target region includes at least one target sub-region, and after obtaining the plurality of first sub-images corresponding to the target region, the method further includes:

[0009] If multiple second subgraphs in the plurality of first subgraphs all cover the same target sub-region, the plurality of second subgraphs are determined as a group of repeating subgraphs.

[0010] In an optional embodiment, before deleting a second number of repeating subgraphs that meet a preset condition from the repeating subgraph group based on the quality evaluation parameters of each repeating subgraph in the repeating subgraph group, the method further includes:

[0011] Obtain the number of normal image frames and the number of abnormal image frames for each repeating subgraph during construction;

[0012] The sum of the number of normal image frames and the number of abnormal image frames corresponding to each repeated sub-image is determined as the total number of image frames corresponding to each repeated sub-image.

[0013] The ratio of the number of normal image frames corresponding to each repeated subgraph to the total number of image frames corresponding to each repeated subgraph is determined as the quality evaluation parameter of each repeated subgraph.

[0014] In an optional embodiment, before obtaining the number of normal image frames and the number of abnormal image frames for each repeating subgraph during construction, the method further includes:

[0015] Acquire the motion state of the target autonomous mobile device when acquiring multiple image frames corresponding to each of the repeating sub-graphs;

[0016] When the motion state indicates that the target autonomous mobile device has an abnormal motion phenomenon, the image frame corresponding to the motion state is determined as the abnormal image frame;

[0017] If the motion state indicates that the target autonomous mobile device does not exhibit any abnormal motion, the image frame corresponding to the motion state is determined as the normal image frame.

[0018] In an optional embodiment, the step of deleting a second number of repeating subgraphs that meet preset conditions from the repeating subgraph group based on the quality evaluation parameters of each repeating subgraph in the repeating subgraph group includes:

[0019] If the first number of repeating subgraphs included in the repeating subgraph group is greater than the preset number of repeating subgraphs, the difference between the first number and the preset number of repeating subgraphs is determined as the second number.

[0020] Delete the second number of duplicate subgraphs; the quality evaluation parameters of the second number of duplicate subgraphs are all less than the quality evaluation parameters of the duplicate subgraphs that have not been deleted in the duplicate subgraph group.

[0021] In an optional embodiment, deleting the second number of repeating subgraphs includes:

[0022] The quality evaluation parameters of each repeated subgraph in the repeated subgraph group are arranged in descending or ascending order to obtain a quality evaluation parameter sequence.

[0023] The second number of quality evaluation parameters are determined based on the quality evaluation parameter sequence; all of the second number of quality evaluation parameters are less than the remaining quality evaluation parameters in the quality evaluation parameter sequence.

[0024] Delete the duplicate subgraphs corresponding to the second number of quality evaluation parameters.

[0025] In an optional embodiment, the method further includes:

[0026] If at least one set of repeating subgraphs exists in the plurality of first subgraphs and the first number of repeating subgraphs included in the repeating subgraph set is less than or equal to the preset number of repeating subgraphs, the first number of repeating subgraphs in the repeating subgraph set shall be retained.

[0027] The present invention also provides an image processing apparatus, characterized in that it comprises:

[0028] The sub-image acquisition module is used to acquire multiple first sub-images corresponding to the target area; each first sub-image is constructed by the target autonomous mobile device based on multiple image frames acquired.

[0029] A duplicate subgraph deletion module is configured to, when at least one set of duplicate subgraph groups exists among the plurality of first subgraphs and the first number of duplicate subgraphs included in the duplicate subgraph group is greater than a preset number of duplicate subgraphs, delete a second number of duplicate subgraphs that meet preset conditions from the duplicate subgraph group based on the quality evaluation parameters of each duplicate subgraph in the duplicate subgraph group; the quality evaluation parameters of each duplicate subgraph are determined based on the number of normal image frames and the number of abnormal image frames of each duplicate subgraph during the construction period, and the second number is the difference between the first number and the preset number of duplicate subgraphs.

[0030] The present invention also provides an electronic device, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image processing method as described above.

[0031] The present invention also provides a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the image processing method as described above.

[0032] Implementing the embodiments of the present invention has the following beneficial effects:

[0033] The image processing method disclosed in this invention determines the quality evaluation parameters of a sub-map based on the number of normal and abnormal image frames during its construction. When it is necessary to selectively delete duplicate sub-maps corresponding to the same region, a certain number of duplicate sub-maps with poor quality are deleted according to the quality evaluation parameters of the duplicate sub-maps, thereby improving the overall quality of the sub-maps and ensuring the quality of the overall map constructed based on the sub-maps. Attached Figure Description

[0034] To more clearly illustrate the image processing method, apparatus, electronic device, and storage medium of the present invention, the accompanying drawings required for the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of the present invention;

[0036] Figure 2 A flowchart illustrating a method for determining quality evaluation parameters provided in an embodiment of the present invention;

[0037] Figure 3 A flowchart illustrating an image frame type determination method provided in an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of real-time sensor data and image frame sampling provided in an embodiment of the present invention;

[0039] Figure 5 This is a flowchart illustrating an image processing method provided in an embodiment of the present invention;

[0040] Figure 6 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present invention;

[0041] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0044] Please refer to Figure 1 The diagram illustrates a flowchart of an image processing method according to an embodiment of the present invention. This specification provides the operational steps of the method described in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed in the order shown in the embodiments or drawings. Specifically, as shown... Figure 1 As shown, the image processing method can be applied to a target autonomous mobile device or a cloud server, and the method includes:

[0045] S101, acquire multiple first sub-images corresponding to the target area; each first sub-image is constructed by the target autonomous mobile device based on multiple image frames acquired;

[0046] In the embodiments described in this specification, the target autonomous mobile device may be a mobile robot, such as a cleaning robot;

[0047] In the embodiments described in this specification, when the target autonomous mobile device moves within the target area, image frames are acquired by the acquisition end of the target autonomous mobile device. The acquisition end can be a lidar sensor or other visual information acquisition device, and correspondingly, the image frame can be a laser frame or other visual image frame.

[0048] In the embodiments of this specification, the acquisition end of the target autonomous mobile device sends multiple acquired image frames to the image processing module or cloud server of the target autonomous mobile device, and the image processing module or the cloud server constructs a first sub-image corresponding to the target region based on the multiple image frames.

[0049] In this embodiment of the specification, the target region includes at least one target sub-region, and after obtaining the plurality of first sub-images corresponding to the target region, the method further includes:

[0050] If multiple second subgraphs in the plurality of first subgraphs all cover the same target sub-region, the plurality of second subgraphs are determined as a group of repeating subgraphs.

[0051] In the embodiments described in this specification, the target sub-region is a region with an area larger than a preset area, and the preset area can be set according to the actual application scenario of the target autonomous mobile device;

[0052] In this embodiment of the specification, the repetition of sub-images is determined by judging which sub-images in the multiple first sub-images corresponding to the target area cover the same target sub-area;

[0053] For example, if the target region corresponds to 20 sub-graphs, where 3 sub-graphs cover the same target sub-region S1 and 5 sub-graphs cover the same target sub-region S2, then the 3 sub-graphs covering the target sub-region S1 are determined as a group of repeating sub-graphs, and the 5 sub-graphs covering the target sub-region S2 are determined as another group of repeating sub-graphs.

[0054] Based on the above technical means, in the embodiments of this specification, after obtaining multiple subgraphs, it is determined which subgraphs are duplicated, and then it is further determined whether some duplicate subgraphs need to be deleted. If so, according to the quality evaluation parameters of the duplicate subgraphs, some duplicate subgraphs with poor quality are deleted, thereby improving the overall quality of the subgraphs and ensuring the quality of the overall map constructed based on the subgraphs.

[0055] S103, if at least one set of repeating subgraph groups exists among the plurality of first subgraphs and the first number of repeating subgraphs included in the repeating subgraph group is greater than a preset number of repeating subgraphs, then based on the quality evaluation parameters of each repeating subgraph in the repeating subgraph group, a second number of repeating subgraphs that meet the preset conditions are deleted from the repeating subgraph group; the quality evaluation parameters of each repeating subgraph are determined based on the number of normal image frames and the number of abnormal image frames of each repeating subgraph during construction, and the second number is the difference between the first number and the preset number of repeating subgraphs;

[0056] Please refer to Figure 2The diagram shows a flowchart of a method for determining quality evaluation parameters according to an embodiment of the present invention. In this embodiment, before deleting a second number of repeating subgraphs that meet preset conditions from the repeating subgraph group based on the quality evaluation parameters of each repeating subgraph in the repeating subgraph group, the method further includes:

[0057] S201, obtain the number of normal image frames and the number of abnormal image frames for each repeating subgraph during construction;

[0058] In the embodiments of this specification, each image frame can be determined as a normal image frame or an abnormal image frame based on the motion state of the target autonomous mobile device when acquiring each image frame, and then the number of normal image frames and the number of abnormal image frames corresponding to each repeated sub-image can be further determined.

[0059] S203, the sum of the number of normal image frames and the number of abnormal image frames corresponding to each repeated sub-image is determined as the total number of image frames corresponding to each repeated sub-image;

[0060] In the embodiments of this specification, the formula for calculating the total number of image frames corresponding to each repeating sub-image is N. total =N normal +N exception , where N normal N represents the number of normal image frames corresponding to each repeating sub-image. exception The number of abnormal image frames corresponding to each repeating subgraph;

[0061] S205, the ratio of the number of normal image frames corresponding to each repeated sub-image to the total number of image frames corresponding to each repeated sub-image is determined as the quality evaluation parameter of each repeated sub-image;

[0062] In the embodiments of this specification, the calculation formula for the quality evaluation parameters is as follows: Where, N normal N represents the number of normal image frames corresponding to each repeating sub-image. total The total number of image frames corresponding to each repeating sub-image; the quality evaluation parameter σ ranges from 0 < σ ≤ 1, and the larger σ is, the better the quality of the sub-image;

[0063] Furthermore, the quality evaluation parameters can also be calculated using the ratio of the number of normal image frames corresponding to each repeating sub-image to the number of abnormal image frames corresponding to each repeating sub-image, i.e. Where, N normal N represents the number of normal image frames corresponding to each repeating sub-image. exception The number of abnormal image frames corresponding to each repeating subgraph;

[0064] Based on the above technical means, the embodiments of this specification determine the quality evaluation parameters of a subgraph based on the number of normal image frames and the number of abnormal image frames during the construction period. Furthermore, when it is necessary to delete some duplicate subgraphs, the lower quality duplicate subgraphs are deleted according to the quality evaluation parameters of the duplicate subgraphs, thereby improving the overall quality of the subgraphs and ensuring the quality of the overall map constructed based on the subgraphs.

[0065] Please refer to Figure 3 The diagram shows a flowchart of an image frame type determination method provided by an embodiment of the present invention. In this embodiment, before obtaining the number of normal image frames and the number of abnormal image frames in each repeating sub-image during construction, the method further includes:

[0066] S301, Obtain the motion state of the target autonomous mobile device when acquiring multiple image frames corresponding to each of the repeating sub-images;

[0067] In the embodiments of this specification, based on the motion state of the target autonomous mobile device when acquiring each image frame, the type of each image frame can be determined, and the type of the image frame may include normal image frames and abnormal image frames;

[0068] S303, when the motion state indicates that the target autonomous mobile device has an abnormal motion phenomenon, the image frame corresponding to the motion state is determined as the abnormal image frame.

[0069] In the embodiments of this specification, the abnormal motion phenomenon may include slipping, collision, tilting, excessive rotation, radar anomaly, etc. If the target autonomous mobile device exhibits the abnormal motion phenomenon when acquiring a certain image frame, then the image frame is determined to be an abnormal image frame.

[0070] Specifically, it can be determined whether the abnormal motion phenomenon occurs when the target autonomous mobile device acquires a certain image frame based on the motion data of the target autonomous mobile device when acquiring a certain image frame, the characteristics of the image frame acquired by the target autonomous mobile device, or the radar status data of the target autonomous mobile device when acquiring a certain image frame.

[0071] The target autonomous mobile device may have multiple sensors; please refer to [reference needed]. Figure 4 The diagram illustrates a real-time sensor data and image frame sampling method provided by an embodiment of the present invention. The image frame Scan is being collected. i At the same time, it can count Scan i-1 To Scan i The corresponding sensor data d0, d1, d2, ..., d within the time period n-1 d nIf d0, d1, d2, ..., d n-1 d n If there are anomalous data indicating slippage, collision, tilting, or excessive rotation, it indicates that Scan... i For example, whether the target autonomous mobile device (AMO) slipped while acquiring an image frame can be determined by whether its actual forward distance matches the forward distance detected by laser positioning; whether a collision occurred while acquiring an image frame can be determined by the output of the collision switch of the target AMO; whether the target AMO tilted while acquiring an image frame can be determined by the z-axis acceleration acquired by the three-axis accelerometer of the target AMO, and if the z-axis acceleration is not equal to the preset acceleration, then the target AMO tilted while acquiring that image frame, and that image frame is an abnormal image frame; and whether the target AMO rotated too fast while acquiring an image frame can be determined by the angular velocity acquired by the angular velocity sensor of the target AMO, and if the angular velocity is greater than the preset angular velocity, then the target AMO rotated too fast while acquiring that image frame, and that image frame is an abnormal image frame.

[0072] Furthermore, the presence or absence of an abnormal image frame can be determined based on the number of point clouds in a specific image frame acquired by the target autonomous mobile device. If the number of point clouds in the image frame is less than a preset number, it indicates that the radar of the target autonomous mobile device is blocked, and the image frame is an abnormal image frame. Alternatively, the presence or absence of an abnormal image frame can be determined based on the radar status data of the target autonomous mobile device when acquiring a specific image frame. If the radar status data indicates that the radar of the target autonomous mobile device has experienced abnormal phenomena such as rotational speed failure (e.g., the radar is stuck by a foreign object) or unstable power supply, then the image frame is determined to be an abnormal image frame.

[0073] S305, if the motion state characterizes that the target autonomous mobile device does not exhibit any abnormal motion, the image frame corresponding to the motion state is determined as the normal image frame.

[0074] In the embodiments of this specification, if the target autonomous mobile device does not exhibit the abnormal motion phenomenon when acquiring a certain image frame, then the image frame is determined to be a normal image frame.

[0075] Based on the above technical means, the embodiments of this specification can determine the type of each image frame according to the motion state of the target autonomous mobile device when acquiring each image frame, and further determine the number of normal image frames and the number of abnormal image frames corresponding to each repeated sub-image. Based on the number of normal image frames and the number of abnormal image frames corresponding to each repeated sub-image, the quality evaluation parameters of each repeated sub-image are determined. When it is necessary to delete some repeated sub-images, the poor quality repeated sub-images can be deleted according to the quality evaluation parameters of the repeated sub-images, thereby improving the overall quality of the sub-images and ensuring the quality of the overall map constructed based on the sub-images.

[0076] In the embodiments of this specification, the step of deleting a second number of repeating subgraphs that meet preset conditions from the repeating subgraph group based on the quality evaluation parameters of each repeating subgraph in the repeating subgraph group includes:

[0077] If the first number of repeating subgraphs included in the repeating subgraph group is greater than the preset number of repeating subgraphs, the difference between the first number and the preset number of repeating subgraphs is determined as the second number.

[0078] Delete the second number of duplicate subgraphs; the quality evaluation parameters of the second number of duplicate subgraphs are all less than the quality evaluation parameters of the duplicate subgraphs that have not been deleted in the duplicate subgraph group;

[0079] In the embodiments of this specification, if there is at least one set of repeating subgraph groups in the plurality of first subgraphs and the first number of repeating subgraphs included in the repeating subgraph group is greater than the preset number of repeating subgraphs, then it is necessary to delete the part of the repeating subgraphs that exceeds the preset number of repeating subgraphs, that is, it is necessary to delete the second number of repeating subgraphs.

[0080] In the embodiments of this specification, the preset condition refers to the fact that the quality evaluation parameters of the duplicate subgraphs that need to be deleted in the duplicate subgraph group are all less than the quality evaluation parameters of the duplicate subgraphs that have not been deleted in the duplicate subgraph group.

[0081] Based on the above technical means, in the embodiments of this specification, when the number of duplicate subgraphs is greater than the preset number of duplicate subgraphs, the lower quality duplicate subgraphs are deleted according to the quality evaluation parameters of the duplicate subgraphs, thereby improving the overall quality of the subgraphs and ensuring the quality of the overall map constructed based on the subgraphs.

[0082] In the embodiments of this specification, deleting the second number of duplicate subgraphs includes:

[0083] The quality evaluation parameters of each repeated subgraph in the repeated subgraph group are arranged in descending or ascending order to obtain a quality evaluation parameter sequence.

[0084] The second number of quality evaluation parameters are determined based on the quality evaluation parameter sequence; all of the second number of quality evaluation parameters are less than the remaining quality evaluation parameters in the quality evaluation parameter sequence.

[0085] Delete the duplicate subgraphs corresponding to the second number of quality evaluation parameters;

[0086] In the embodiments of this specification, if the quality evaluation parameters of each repeated subgraph in the repeated subgraph group are arranged in descending order, then the repeated subgraphs corresponding to the second number of quality evaluation parameters that are later in the sequence of quality evaluation parameters are deleted; if the quality evaluation parameters of each repeated subgraph in the repeated subgraph group are arranged in ascending order, then the repeated subgraphs corresponding to the second number of quality evaluation parameters that are earlier in the sequence of quality evaluation parameters are deleted.

[0087] Based on the above technical means, in the embodiments of this specification, when the number of repeated subgraphs is greater than the preset number of repeated subgraphs, the repeated subgraphs with smaller quality evaluation parameters are deleted according to the quality evaluation parameters of the repeated subgraphs, thereby improving the overall quality of the subgraphs and ensuring the quality of the overall map constructed based on the subgraphs.

[0088] In the embodiments described in this specification, the method further includes:

[0089] If at least one set of repeating subgraphs exists in the plurality of first subgraphs and the first number of repeating subgraphs included in the repeating subgraphs is less than or equal to the preset number of repeating subgraphs, the first number of repeating subgraphs in the repeating subgraphs are retained.

[0090] In the embodiments of this specification, if there is at least one set of repeating subgraph groups in the plurality of first subgraphs and the first number of repeating subgraphs included in the repeating subgraph group is less than or equal to the preset number of repeating subgraphs, then it is not necessary to delete the repeating subgraphs in the repeating subgraph group.

[0091] In the embodiments described in this specification, the preset number of repeating subgraphs can be set according to the actual application of the target autonomous mobile device;

[0092] Based on the above technical means, in the embodiments of this specification, when the number of repeated subgraphs is less than or equal to the preset number of repeated subgraphs, it is not necessary to delete repeated subgraphs; when the number of repeated subgraphs is greater than the preset number of repeated subgraphs, the repeated subgraphs with smaller quality evaluation parameters are deleted according to the quality evaluation parameters of the repeated subgraphs, thereby improving the overall quality of the subgraphs and ensuring the quality of the overall map constructed based on the subgraphs.

[0093] Figure 5This is a flowchart illustrating an image processing method provided in an embodiment of the present invention. S501: Obtain multiple first sub-images corresponding to a target region; each first sub-image is constructed by the target autonomous mobile device based on multiple acquired image frames. S503: Determine whether there is at least one group of repeating sub-images among the multiple first sub-images; if yes, proceed to S505; otherwise, proceed to S509. S505: Determine whether the first number of repeating sub-images included in the repeating sub-image group is greater than a preset number of repeating sub-images; if yes, proceed to S507; otherwise, proceed to S511. S507: Based on the quality evaluation parameters of each repeating sub-image in the repeating sub-image group, delete a second number of repeating sub-images that meet preset conditions from the repeating sub-image group; the quality evaluation parameters of each repeating sub-image are determined based on the number of normal image frames and the number of abnormal image frames during the construction of each repeating sub-image, and the second number is the difference between the first number and the preset number of repeating sub-images. S509: Retain the multiple first sub-images. S511: Retain the first number of repeating sub-images in the repeating sub-image group.

[0094] Specifically, when the target autonomous mobile device moves within the target area, the acquisition end captures image frames and sends them to the image processing module or cloud server of the target autonomous mobile device. The image processing module or cloud server constructs multiple sub-images based on the image frames. If there is at least one set of repeated sub-image groups among the multiple sub-images, and the number of repeated sub-images included in the repeated sub-image group is greater than the preset number of repeated sub-images, then according to the quality evaluation parameter of each repeated sub-image in the repeated sub-image group, a certain number of repeated sub-images with smaller quality evaluation parameters are deleted from the repeated sub-image group. If there is no repeated sub-image group among the multiple sub-images, then no sub-images need to be deleted. If there is at least one set of repeated sub-image groups among the multiple sub-images, and the number of repeated sub-images included in the repeated sub-image group is less than or equal to the preset number of repeated sub-images, then no repeated sub-images in the repeated sub-image group need to be deleted.

[0095] For example, if a target autonomous mobile device constructs 20 sub-maps while moving within a target area, with 12 sub-maps being repeated (forming a first repeated sub-map group) and another 5 sub-maps being repeated (forming a second repeated sub-map group), and the preset number of repeated sub-maps is 10, then for the first repeated sub-map group, since the number of repeated sub-maps is greater than the preset number, 2 repeated sub-maps need to be deleted from the first repeated sub-map group based on the quality evaluation parameters of the 12 repeated sub-maps. The quality evaluation parameters of the 2 deleted repeated sub-maps are both lower than the quality evaluation parameters of the other 10 repeated sub-maps in the first repeated sub-map group that were not deleted. For the second repeated sub-map group, since the number of repeated sub-maps is less than the preset number, no repeated sub-maps need to be deleted from the second repeated sub-map group.

[0096] As can be seen from the above embodiments of the image processing method provided by the present invention, the embodiments of the present invention acquire multiple first sub-images corresponding to a target area; each first sub-image is constructed by the target autonomous mobile device based on multiple image frames acquired; when there is at least one set of duplicate sub-image groups among the multiple first sub-images and the first number of duplicate sub-images included in the duplicate sub-image group is greater than a preset number of duplicate sub-images, a second number of duplicate sub-images that meet the preset conditions are deleted from the duplicate sub-image group based on the quality evaluation parameters of each duplicate sub-image in the duplicate sub-image group; the quality evaluation parameters of each duplicate sub-image are determined based on the number of normal image frames and the number of abnormal image frames during the construction of each duplicate sub-image, and the second number is the difference between the first number and the preset number of duplicate sub-images. The technical solution provided by the embodiments of this specification determines the quality evaluation parameters of a sub-image based on the number of normal image frames and the number of abnormal image frames during the construction of the sub-image. When it is necessary to selectively delete duplicate sub-images corresponding to the same area, a certain number of duplicate sub-images with poor quality are deleted according to the quality evaluation parameters of the duplicate sub-images, thereby improving the overall quality of the sub-images and ensuring the quality of the overall map constructed based on the sub-images.

[0097] This invention also provides an image processing apparatus, such as... Figure 6 As shown, this is a schematic diagram of the structure of an image processing device provided in an embodiment of the present invention; specifically, the device includes:

[0098] The sub-image acquisition module 610 is used to acquire multiple first sub-images corresponding to the target area; each first sub-image is constructed by the target autonomous mobile device based on multiple image frames acquired;

[0099] The duplicate subgraph deletion module 620 is configured to delete a second number of duplicate subgraphs that meet preset conditions from the duplicate subgraph group when at least one duplicate subgraph group exists among the plurality of first subgraphs and the first number of duplicate subgraphs included in the duplicate subgraph group is greater than a preset number of duplicate subgraphs, based on the quality evaluation parameters of each duplicate subgraph in the duplicate subgraph group; the quality evaluation parameters of each duplicate subgraph are determined based on the number of normal image frames and the number of abnormal image frames of each duplicate subgraph during the construction period, and the second number is the difference between the first number and the preset number of duplicate subgraphs.

[0100] In this embodiment of the specification, the target region includes at least one target sub-region, and the device further includes:

[0101] The repeating subgraph group determination module is used to determine the multiple second subgraphs as a repeating subgraph group when multiple second subgraphs in the multiple first subgraphs all cover the same target sub-region;

[0102] The image frame count acquisition module is used to acquire the number of normal image frames and the number of abnormal image frames for each repeating subgraph during construction.

[0103] The total image frame count determination module is used to determine the sum of the number of normal image frames and the number of abnormal image frames corresponding to each repeated sub-image as the total image frame count corresponding to each repeated sub-image.

[0104] The quality evaluation parameter determination module is used to determine the ratio of the number of normal image frames corresponding to each repeated sub-image to the total number of image frames corresponding to each repeated sub-image as the quality evaluation parameter of each repeated sub-image.

[0105] The motion state acquisition module is used to acquire the motion state of the target autonomous mobile device when acquiring multiple image frames corresponding to each of the repeating sub-images;

[0106] An abnormal image frame determination module is used to determine the image frame corresponding to the motion state as the abnormal image frame when the motion state characterizes the abnormal motion phenomenon of the target autonomous mobile device.

[0107] The normal image frame determination module is used to determine the image frame corresponding to the motion state as the normal image frame when the motion state characterizes that the target autonomous mobile device does not exhibit any abnormal motion phenomena.

[0108] A repeating subgraph retention module is used to retain a first number of repeating subgraphs in the repeating subgraph group when there is at least one repeating subgraph group in the plurality of first subgraphs and the first number of repeating subgraphs included in the repeating subgraph group is less than or equal to the preset number of repeating subgraphs.

[0109] In this embodiment of the specification, the repeating subgraph deletion module 620 includes:

[0110] The unit for determining the number of duplicate subgraphs to be deleted is used to determine the difference between the first number and the preset number of duplicate subgraphs as the second number when the first number of duplicate subgraphs included in the duplicate subgraph group is greater than the preset number of duplicate subgraphs.

[0111] The duplicate subgraph deletion unit is used to delete the second number of duplicate subgraphs; the quality evaluation parameters of the second number of duplicate subgraphs are all less than the quality evaluation parameters of the duplicate subgraphs that have not been deleted in the duplicate subgraph group.

[0112] In the embodiments of this specification, the repeating subgraph deletion unit includes:

[0113] The sorting subunit is used to arrange the quality evaluation parameters of each repeated subgraph in the repeated subgraph group in descending or ascending order to obtain a sequence of quality evaluation parameters.

[0114] A quality evaluation parameter determination subunit is used to determine a second number of quality evaluation parameters based on the quality evaluation parameter sequence; the second number of quality evaluation parameters are all less than the remaining quality evaluation parameters in the quality evaluation parameter sequence.

[0115] The repeating subgraph deletion subunit is used to delete the repeating subgraphs corresponding to the second number of quality evaluation parameters.

[0116] This invention also provides an electronic device, which includes a processor and a memory; the memory stores one or more instructions, which are adapted to be loaded and executed by the processor to implement the image processing method as described in the above method embodiments.

[0117] Memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the device, etc. Furthermore, memory can include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory may also include a memory controller to provide the processor with access to the memory.

[0118] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The internal structure of the electronic device may include, but is not limited to, a processor 701, a memory 702, and a communication interface 703. The processor 701, memory 702, and communication interface 703 within the electronic device can be connected via a bus or other means, as shown in the embodiments of this specification. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0119] The processor 701 (or CPU, Central Processing Unit) is the computing and control core of the electronic device. The communication interface 703 is used for communication between the memory 702 and the processor 701. The memory 702 is used to store programs and data. It is understood that the memory here can be a high-speed RAM storage device, or a non-volatile memory device, such as at least one disk storage device; optionally, it can also be at least one storage device located remotely from the aforementioned processor. The memory provides storage space, which stores the operating system of the electronic device, and may include, but is not limited to, Windows (an operating system), Linux (an operating system), etc. This invention does not limit this; furthermore, the storage space also stores computer programs (including program code) suitable for loading and execution by the processor. In the embodiments of this specification, the processor 701 loads and executes the computer program stored in the memory 702 to implement the image processing method provided in the above method embodiments.

[0120] This invention also provides a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction, at least one program, code set, or instruction set related to implementing an image processing method in the method embodiments. The at least one instruction, the at least one program, the code set, or the instruction set can be loaded and executed by the processor of the electronic device to implement the image processing method provided in the above method embodiments.

[0121] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0122] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments, while other embodiments fall within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than those shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0123] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0124] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0125] The above description is merely a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. An image processing method, characterized in that, include: Multiple first sub-images corresponding to the target region are obtained; each first sub-image is constructed by the target autonomous mobile device based on multiple image frames acquired; If at least one set of repeating subgraphs exists among the plurality of first subgraphs and the first number of repeating subgraphs included in the repeating subgraph set is greater than a preset number of repeating subgraphs, the number of normal image frames and the number of abnormal image frames for each repeating subgraph during construction are obtained; the sum of the number of normal image frames and the number of abnormal image frames corresponding to each repeating subgraph is determined as the total number of image frames corresponding to each repeating subgraph; the ratio of the number of normal image frames corresponding to each repeating subgraph to the total number of image frames corresponding to each repeating subgraph is determined as the quality evaluation parameter for each repeating subgraph. Based on the quality evaluation parameters of each repeated subgraph in the repeated subgraph group, a second number of repeated subgraphs that meet the preset conditions are deleted from the repeated subgraph group; the quality evaluation parameters of each repeated subgraph are determined based on the number of normal image frames and the number of abnormal image frames of each repeated subgraph during the construction period, and the second number is the difference between the first number and the preset number of repeated subgraphs.

2. The image processing method according to claim 1, characterized in that, The target region includes at least one target sub-region. After obtaining multiple first sub-images corresponding to the target region, the method further includes: If multiple second subgraphs in the plurality of first subgraphs all cover the same target sub-region, the plurality of second subgraphs are determined as a group of repeating subgraphs.

3. The image processing method according to claim 1, characterized in that, Before obtaining the number of normal image frames and the number of abnormal image frames for each repeating subgraph during construction, the method further includes: Acquire the motion state of the target autonomous mobile device when acquiring multiple image frames corresponding to each of the repeating sub-graphs; When the motion state indicates that the target autonomous mobile device has an abnormal motion phenomenon, the image frame corresponding to the motion state is determined as the abnormal image frame. If the motion state indicates that the target autonomous mobile device does not exhibit any abnormal motion, the image frame corresponding to the motion state is determined as the normal image frame.

4. The image processing method according to claim 1, characterized in that, The step of deleting a second number of repeating subgraphs that meet preset conditions from the repeating subgraph group based on the quality evaluation parameters of each repeating subgraph in the repeating subgraph group includes: If the first number of repeating subgraphs included in the repeating subgraph group is greater than the preset number of repeating subgraphs, the difference between the first number and the preset number of repeating subgraphs is determined as the second number. Delete the second number of duplicate subgraphs; the quality evaluation parameters of the second number of duplicate subgraphs are all less than the quality evaluation parameters of the duplicate subgraphs that have not been deleted in the duplicate subgraph group.

5. The image processing method according to claim 4, characterized in that, The deletion of the second number of duplicate subgraphs includes: The quality evaluation parameters of each repeated subgraph in the repeated subgraph group are arranged in descending or ascending order to obtain a quality evaluation parameter sequence. The second number of quality evaluation parameters are determined based on the quality evaluation parameter sequence; all of the second number of quality evaluation parameters are less than the remaining quality evaluation parameters in the quality evaluation parameter sequence. Delete the duplicate subgraphs corresponding to the second number of quality evaluation parameters.

6. The image processing method according to claim 1, characterized in that, The method further includes: If at least one set of repeating subgraphs exists in the plurality of first subgraphs and the first number of repeating subgraphs included in the repeating subgraph set is less than or equal to the preset number of repeating subgraphs, the first number of repeating subgraphs in the repeating subgraph set shall be retained.

7. An image processing apparatus, characterized in that, include: The sub-image acquisition module is used to acquire multiple first sub-images corresponding to the target area; Each first sub-image is constructed by the target autonomous mobile device based on multiple image frames acquired; A duplicate subgraph deletion module is configured to, when at least one set of duplicate subgraph groups exists among the plurality of first subgraphs and the first number of duplicate subgraphs included in the duplicate subgraph group is greater than a preset number of duplicate subgraphs, obtain the number of normal image frames and the number of abnormal image frames for each duplicate subgraph during construction; determine the sum of the number of normal image frames and the number of abnormal image frames corresponding to each duplicate subgraph as the total number of image frames corresponding to each duplicate subgraph; and determine the ratio of the number of normal image frames corresponding to each duplicate subgraph to the total number of image frames corresponding to each duplicate subgraph as the quality evaluation parameter for each duplicate subgraph. Based on the quality evaluation parameters of each repeated subgraph in the repeated subgraph group, a second number of repeated subgraphs that meet the preset conditions are deleted from the repeated subgraph group; the quality evaluation parameters of each repeated subgraph are determined based on the number of normal image frames and the number of abnormal image frames of each repeated subgraph during the construction period, and the second number is the difference between the first number and the preset number of repeated subgraphs.

8. An electronic device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program segment, the at least one instruction or the at least one program segment being loaded and executed by the processor to achieve the purpose of claim 1.

6. The image processing method described in any one of the above.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to achieve the purpose of claim 1.

6. The image processing method described in any one of the above.

Images