Method and device for confirming key frame and related equipment

Abstract

The invention provides a method and device for confirming a key frame and related equipment, and the method comprises the steps: obtaining a first scanning position and a first scanning angle range corresponding to a current frame; acquiring a second scanning angle range corresponding to the historical key frame of the first scanning position; and when the angle difference between the first scanning angle range and the second scanning angle range is greater than or equal to a first preset angle threshold value, determining that the current frame is a key frame associated with the first scanning position. In the method, the consumption of computing resources mainly focuses on the calculation of the angle difference between the first scanning angle range and the second scanning angle range, and the process is low in calculation complexity and small in calculation amount, so that the number of consumed computing resources is small. When the method is used for confirming whether the current frame is the key frame or not, the relation between the current frame and the multiple key frames associated with the position where the current frame is located needs to be considered, and therefore the number of the key frames can be prevented from being excessively increased.

Description

Technical Field

[0001] The present application relates to the technical field of synchronous positioning and map building, and in particular to a method, device and related equipment for confirming key frames. Background Art

[0002] Simultaneous Localization and Mapping (SLAM) is a technology that serves automatic machines (including autonomous mobile robots and unmanned vehicles), through which automatic machines can achieve self-positioning and environmental mapping. In SLAM technology, the selection of key frames is particularly important. Key frames refer to the frames where the key actions of the object's movement or change are located. For example, when an obstacle suddenly appears in the area where the automatic machine is located, the key frame can be the frame in which the automatic machine first scans the obstacle; when an obstacle suddenly decreases in the area where the automatic machine is located, the key frame can be the frame in which the automatic machine first scans the obstacle and disappears.

[0003] When the automatic machine performs self-positioning and environmental mapping in the working area, it needs to obtain the circular keyframes corresponding to each position in the working area. Since the scanning shooting angle range of the automatic machine when passing each position is limited, the circular keyframes corresponding to each position need to be merged and generated based on the keyframes shot by the automatic machine when passing through the position multiple times. However, since the corresponding scanning angle range of the automatic machine is uncontrollable each time it passes through each position in the working area, that is, the image shot by the automatic machine at a certain position may be highly overlapped with the image shot by the automatic machine at the same position before, and the image shot by the automatic machine at a certain position may be a non-keyframe, so it is very important to confirm whether the image shot by the automatic machine each time is a keyframe.

[0004] The current method for confirming a key frame includes determining whether the current frame is a key frame by judging whether the posture difference of the frame, the change in the Euclidean distance between different coordinate systems, and the change in the Euler angle between different coordinate systems are greater than respective preset thresholds. Since the calculation of the posture difference involves the point cloud matching algorithm and the inertial measurement unit (IMU) integration method, both algorithms have high computational complexity, and the point cloud matching algorithm consumes a lot of computing resources in large-scale point cloud data, the current method for confirming a key frame is prone to low execution efficiency and high CPU occupancy when used. Summary of the invention

[0005] The present application provides a method, apparatus and related equipment for confirming key frames, which can not only reduce the consumption of computing resources, but also consider the relationship between the current frame and multiple key frames associated with the position when confirming whether the current frame is a key frame, thereby preventing the number of key frames from growing excessively.

[0006] In a first aspect, the present application provides a method for confirming a key frame, the method comprising: obtaining a first scanning position and a first scanning angle range corresponding to a first current frame; obtaining a second scanning angle range corresponding to a historical key frame of the first scanning position, the historical key frame being a plurality of key frames associated with the scanning position; and determining that the first current frame is a key frame associated with the first scanning position when an angle difference between the first scanning angle range and the second scanning angle range that does not overlap is greater than or equal to a first preset angle threshold.

[0007] In a possible implementation of the first aspect, when the angle difference between the first scanning angle range and the second scanning angle range that does not overlap is greater than or equal to a first preset angle threshold, determining the first current frame as a key frame associated with the first scanning position includes: merging the first scanning angle range with the second scanning angle range to obtain a third scanning angle range; subtracting the number of angles included in the second scanning angle range from the number of angles included in the third scanning angle range to obtain a first angle number; and when the first angle number is greater than or equal to the first preset angle threshold, determining the first current frame as a key frame associated with the first scanning position.

[0008] In a possible implementation of the first aspect, obtaining a second scanning angle range corresponding to a historical key frame of a first scanning position includes: querying a first table according to the first scanning position, the first table including multiple scanning positions and scanning angle ranges corresponding to multiple key frames associated with each scanning position; when the multiple scanning positions include the first scanning position, obtaining the second scanning angle range corresponding to the historical key frame of the first scanning position.

[0009] In a possible implementation of the first aspect, after determining that the first current frame is a key frame associated with the first scanning position, the method also includes: merging the first scanning angle range with the second scanning angle range to obtain a third scanning angle range; and updating the scanning angle range corresponding to the first scanning position in the first table to the third scanning angle range.

[0010] In a possible implementation of the first aspect, the method also includes: obtaining a second scanning position and a fourth scanning angle range corresponding to a second current frame; determining whether the second scanning position has a historical key frame; and when it is determined that the second scanning position does not have a historical key frame, determining that the second current frame is a key frame associated with the second scanning position.

[0011] In a possible implementation of the first aspect, determining whether the second scanning position has a historical key frame includes: querying the first table according to the second scanning position; when the multiple scanning positions do not include the second scanning position, determining that the second current frame is a key frame associated with the second scanning position, and updating the second scanning position, the fourth scanning angle range, and the correspondence between the second scanning position and the fourth scanning angle into the first table.

[0012] In a possible implementation of the first aspect, after obtaining the second scanning angle range corresponding to the historical key frame of the first scanning position, the method also includes: determining whether the number of angles included in the second scanning angle range is greater than or equal to a second preset angle threshold; when it is determined that the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold, confirming that no new key frame is needed for the first scanning position, and confirming the first current frame as a non-key frame.

[0013] In a possible implementation of the first aspect, the first table is a hash table. Before obtaining the first scanning position and the first scanning angle range corresponding to the first current frame, the method also includes: obtaining multiple scanning positions in the working area and multiple key frames associated with each of the multiple scanning positions; calculating a merge of scanning angle ranges corresponding to multiple key frames associated with each of the multiple scanning positions; and constructing a hash table, the hash table including multiple key-value pairs, the key of each key-value pair corresponding to the scanning position, and the value of each key-value pair being a merge of scanning angle ranges corresponding to the multiple key frames associated with the scanning position.

[0014] In a second aspect, the present application provides a device for confirming a key frame, the device comprising: a first acquisition unit, used to obtain a first scanning position and a first scanning angle range corresponding to a first current frame; a second acquisition unit, used to obtain a second scanning angle range corresponding to a historical key frame of the first scanning position, the historical key frame being a plurality of key frames associated with the scanning position; a confirmation unit, used to determine that the first current frame is a key frame associated with the first scanning position when the angle difference between the first scanning angle range and the second scanning angle range that does not overlap is greater than or equal to a first preset angle threshold.

[0015] In a possible implementation of the second aspect, the confirmation unit is specifically used to: merge the first scanning angle range with the second scanning angle range to obtain a third scanning angle range; subtract the number of angles included in the second scanning angle range from the number of angles included in the third scanning angle range to obtain a first angle number; and determine that the first current frame is a key frame associated with the first scanning position when the first angle number is greater than or equal to a first preset angle threshold.

[0016] In a possible implementation of the second aspect, the second acquisition unit is specifically used to: query a first table based on the first scanning position, the first table including multiple scanning positions and scanning angle ranges corresponding to multiple key frames associated with each scanning position; when the multiple scanning positions include the first scanning position, obtain a second scanning angle range corresponding to a historical key frame of the first scanning position.

[0017] In a possible implementation of the second aspect, after determining that the first current frame is a key frame associated with the first scanning position, the confirmation unit is also used to: merge the first scanning angle range with the second scanning angle range to obtain a third scanning angle range; and update the scanning angle range corresponding to the first scanning position in the first table to the third scanning angle range.

[0018] In a possible implementation of the second aspect, the first acquisition unit is further used to obtain the second scanning position and the fourth scanning angle range corresponding to the second current frame; the second acquisition unit is further used to determine whether the second scanning position has a historical key frame; when it is determined that the second scanning position does not have a historical key frame, the second current frame is determined to be a key frame associated with the second scanning position.

[0019] In a possible implementation of the second aspect, the second acquisition unit is specifically used to: query the first table according to the second scanning position; when the multiple scanning positions do not include the second scanning position, determine that the second current frame is a key frame associated with the second scanning position, and update the second scanning position, the fourth scanning angle range, and the correspondence between the second scanning position and the fourth scanning angle into the first table.

[0020] In a possible implementation of the second aspect, the second acquisition unit is also used to: determine whether the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold; when it is determined that the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold, confirm that no new key frame is needed for the first scanning position, and confirm the first current frame as a non-key frame.

[0021] In a possible implementation manner of the second aspect, the first table is a hash table, and before obtaining the first scanning position and the first scanning angle range corresponding to the first current frame, the first acquisition unit is further used to: obtain multiple scanning positions in the working area and multiple key frames associated with each of the multiple scanning positions; calculate a merge of scanning angle ranges corresponding to multiple key frames associated with each of the multiple scanning positions;

[0022] A hash table is constructed, the hash table includes multiple key-value pairs, the key of each key-value pair corresponds to a scanning position, and the value of each key-value pair is a combination of scanning angle ranges corresponding to multiple key frames associated with the scanning position.

[0023] In a third aspect, the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing computer program instructions, the processor implementing the method in the first aspect and any possible implementation manner of the first aspect by running the computer program instructions.

[0024] In a fourth aspect, the present application provides a lawn mower, which includes the device in the second aspect and any possible implementation manner of the second aspect or the electronic device in the third aspect.

[0025] In a fifth aspect, the present application provides a computer program product comprising instructions. A computer program product is a software or program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one electronic device, the at least one electronic device executes the method in the first aspect and any possible implementation of the first aspect.

[0026] In a sixth aspect, the present application provides a computer-readable storage medium, comprising computer instructions, which, when executed by a processor, implement the method in the first aspect and any possible implementation manner of the first aspect.

[0027] The above-mentioned second, third, fourth, fifth and sixth aspects all have various possible designs similar to the first aspect and any possible implementation method of the first aspect, and can produce corresponding technical effects, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 is a schematic diagram of a method for confirming a key frame provided by the present application;

[0030] Figure 2 is a structural diagram of a current frame provided by this application;

[0031] Figure 3 is an architectural diagram of a device including a key frame confirmation device provided by the present application;

[0032] Figure 4 It is a structural schematic diagram of an electronic device provided by the present application;

[0033] Figure 5 It is a structural schematic diagram of a lawn mower provided in the present application. DETAILED DESCRIPTION

[0034] The technical solutions involved in this application are clearly and completely described below in conjunction with the accompanying drawings. The embodiments described below are only part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0035] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the term "comprising" and any variation thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0036] It should be noted that the terms of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" of the present application are also intended to include plural forms, unless the context clearly indicates other meanings.

[0037] In order to solve the problem that the current method for confirming key frames is prone to low execution efficiency and high CPU occupancy when in use, the present application provides a method, device and related equipment for confirming key frames, wherein the method for confirming key frames includes: obtaining a first scanning position and a first scanning angle range corresponding to the current frame; obtaining a second scanning angle range corresponding to the historical key frame of the first scanning position; when the angle difference between the first scanning angle range and the second scanning angle range is greater than or equal to the first preset angle threshold, determining that the current frame is a key frame associated with the first scanning position. In the above method for confirming key frames, the consumption of computing resources is mainly concentrated on calculating the angle difference between the first scanning angle range and the second scanning angle range. The computational complexity and amount of computation of this process are low, so the number of computing resources consumed is small. And when using the above method to confirm whether the current frame is a key frame, it is necessary to consider the relationship between the current frame and the multiple key frames associated with the position of the current frame, so the number of key frames can also be prevented from growing excessively.

[0038] Combine the following Figure 1 To introduce the method for confirming key frames provided by the present application. For the convenience of description, the embodiments provided by the present application are all described in detail by taking the automatic machine as an autonomous mobile robot. Figure 1 , Figure 1 is a schematic diagram of a method for confirming a key frame provided by the present application, such as Figure 1As shown, the method for confirming a key frame includes:

[0039] S101: The device for confirming the key frame obtains a first scanning position and a first scanning angle range corresponding to the current frame.

[0040] The above-mentioned current frame is an image frame taken by the autonomous mobile robot at a certain position at the current moment. One current frame corresponds to the position of the autonomous mobile robot at the current moment (the first scanning position) and the angle range (the first scanning angle range) of the autonomous mobile robot for scanning and shooting at this position.

[0041] The first scanning position is a certain position of the autonomous mobile robot in the working area. The working area may be an area that the autonomous mobile robot needs to pass through regularly. For example, a lawn mower will pass through a grass area regularly; a sweeping robot will pass through a room area regularly. The first scanning angle range is an interval between 0 degrees and 360 degrees for the autonomous mobile robot to scan and shoot at the position. For example, the first scanning angle range may be (0 degrees to 60 degrees). It is explained here that since the corresponding scanning angle range of the autonomous mobile robot is uncontrollable each time it passes through each position in the working area, the angle range of the autonomous mobile robot to scan and shoot at different times will also change. For example, the scanning angle range corresponding to the first time the autonomous mobile robot comes to a certain position is (45 degrees to 135 degrees), and the corresponding scanning angle range when the autonomous mobile robot comes to the position again may become (180 degrees to 270 degrees).

[0042] Optionally, a certain position of the autonomous mobile robot in the working area can be represented by coordinates, that is, a coordinate system is created for the area, and each position in the area can uniquely correspond to a coordinate on the coordinate system.

[0043] The establishment of the above-mentioned coordinate system can be autonomously created by the autonomous mobile robot or manually created. The autonomous mobile robot can create the coordinate system of the area by using a variety of sensors. For example, the autonomous mobile robot can use a lidar to emit a laser beam and receive reflected light to construct a point cloud map of the surrounding environment. The autonomous mobile robot determines its position relative to these feature points by analyzing the feature points in the point cloud map, thereby establishing the coordinate system of the area; the autonomous mobile robot can identify and analyze landmarks or features in the environment through a camera, and then use a visual algorithm to connect the collected information with its own location, thereby completing the establishment of the coordinate system. The manually created coordinate system can be that the staff directly sets the coordinates of the area where they are located in the system of the autonomous mobile robot. For example, when using a lawn mower to mow a lawn, the staff can set some landmarks or features in the lawn area, and then set the coordinate positions of these landmarks or features in the regional coordinate system in the lawn mower system. The lawn mower identifies these landmarks and features through sensors to determine its own position coordinates; when using a sweeper to clean a room, the staff can set the coordinates of several key positions in the room (places that need to be cleaned frequently) in the sweeper system, and the sweeper locates and moves according to these preset coordinates.

[0044] It is noted here that the above-mentioned coordinate system establishment process is only an example provided by this application. The specific establishment method of the coordinate system should be appropriately selected in combination with the application scenario and technical conditions. This application does not limit the creation method of the coordinate system.

[0045] The following is the form of coordinates and Figure 2 To introduce the structure of the current frame.

[0046] See also Figure 2 , Figure 2 is a structural diagram of a current frame provided by this application, Figure 2The coordinate system includes a landmark as the origin. In the use scenario of a lawn mower, the landmark can be the starting point of the charging base station; in the use scenario of a sweeper, the coordinate can be the location of the charging pile. A coordinate system is created with the east direction as the X-axis and the north direction as the Y-axis. The coordinate system includes two coordinate points corresponding to the current frame, where the scanning angle range corresponding to the current frame associated with the coordinate (5, 15) is (270 degrees to 315 degrees); the scanning angle range corresponding to the current frame associated with the coordinate (10, 5) is (90 degrees to 180 degrees). According to the definitions of the current frame, the first scanning position, and the first scanning angle range, the current frame corresponding to the coordinates (5, 15) indicates that when the autonomous mobile robot arrives at the coordinates (5, 15), it scans and photographs the surrounding environment, and the final retained scanning angle range is a frame of image from 270 degrees to 315 degrees; similarly, the current frame corresponding to the coordinates (10, 5) indicates that when the autonomous mobile robot arrives at the coordinates (10, 5), it scans and photographs the surrounding environment, and the final retained scanning angle range is a frame of image from 90 degrees to 180 degrees. Here, Figure 2 The coordinate system, coordinates, and scanning angle ranges provided in the present application are merely examples provided in the present application and should not be considered as specific limitations.

[0047] Optionally, the specific process of the device for confirming the key frame obtaining the first scanning position and the first scanning angle range corresponding to the current frame includes: the autonomous mobile robot is at a certain position in the working area, and after taking a frame of image, it sends information including the coordinates corresponding to the shooting position in the coordinate system and the scanning angle range of the camera during shooting to the device for confirming the key frame.

[0048] The above-mentioned autonomous mobile robot sends the coordinates and scanning angle range to the device for confirming the key frame in a wireless or wired manner, wherein the wired manner includes Ethernet, Universal Serial Bus (USB) connection, serial communication, etc.; the wireless manner includes wireless network (Wireless Fidelity, Wi-Fi), Bluetooth, wireless sensor network, etc. The present application does not limit the specific manner in which the autonomous mobile robot sends information to the device for confirming the key frame, and the present application does not limit the manner in which the autonomous mobile robot sends information to the device for confirming the key frame.

[0049] Optionally, the device for confirming the key frame may also be a part of the autonomous mobile robot, that is, after the autonomous mobile robot is at a certain position and shoots a frame of image, the autonomous mobile robot determines by itself whether the frame of image is a key frame.

[0050] S102: The key frame confirmation device obtains a second scanning angle range corresponding to a historical key frame at the first scanning position.

[0051] The above-mentioned historical key frames are multiple key frames associated with the scanning position. The second scanning angle range corresponding to the multiple key frames associated with the first scanning position is a combination of the scanning angle ranges corresponding to the multiple key frames shot by the autonomous mobile robot at different times (which can be continuous or discontinuous times) at the position of the current frame (i.e., one or more intervals between 0 degrees and 360 degrees). For example, the autonomous mobile robot shoots key frame 1 at position 1, and its scanning angle range is 30 degrees to 90 degrees. At the next moment, the autonomous mobile robot shoots key frame 2 at position 1 again, and its scanning angle range is 240 degrees to 270 degrees. Then the second scanning angle range corresponding to the multiple key frames associated with position 1 is the combination of the scanning angle range corresponding to key frame 1 and the scanning angle range corresponding to key frame 2, which is (30 degrees to 90 degrees, 240 degrees to 270 degrees).

[0052] It is explained here that after obtaining the first scanning position and the first scanning angle range corresponding to the current frame, the device for confirming the key frame will also determine whether there is a corresponding historical key frame at the first scanning position. When it is determined that there is no historical key frame at the first scanning position, the current frame is determined to be the key frame associated with the first scanning position. It should be understood here that since the method provided by the present application compares the current frame of a certain position with all the key frames before the position, when there is no historical key frame at a certain position, it means that the autonomous mobile robot has come to the position in the working area for the first time, and the pictures it scans and takes can be merged to generate a circular key frame.

[0053] Optionally, after obtaining the second scanning angle range corresponding to the historical key frame of the first scanning position, the device for confirming the key frame will also determine whether the number of angles included in the second scanning angle range is greater than or equal to a second preset angle threshold. When it is determined that the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold, it is confirmed that no new key frame is needed for the first scanning position, and the current frame is confirmed as a non-key frame.

[0054] The above-mentioned second preset angle threshold can be obtained through an algorithm or through a simulation experiment. The present application does not limit the specific method of obtaining the second preset angle threshold.

[0055] The number of angles included in the above scanning angle range is the angle size included in the angle range for scanning and shooting by the autonomous mobile robot. For example, the angle range corresponding to the second scanning angle range is (60 degrees to 120 degrees, 300 degrees to 330 degrees), then the number of angles included in the second scanning angle range = 120 degrees - 60 degrees + 330 degrees - 300 degrees = 90 degrees.

[0056] It should be understood that the device for confirming the key frame determines whether the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold. When the number of angles included in the second scanning angle range is large enough, for example, when the number of angles included in the second scanning angle range has reached 340 degrees or 360 degrees, it means that the multiple key frames taken by the autonomous mobile robot at this position can be merged into a circular key frame for subsequent self-positioning and environmental mapping of the autonomous robot. By setting the second preset angle threshold to determine the size of the number of angles included in the second scanning angle range, the number of key frames associated with the autonomous mobile robot at this position will not be too large, thereby reducing memory consumption.

[0057] Optionally, the device for confirming the key frame can also query the first table according to the first scanning position; when the multiple scanning positions in the first table include the first scanning position, obtain the second scanning angle range corresponding to the first scanning position in the first table. By setting a table to store the scanning angle ranges corresponding to multiple key frames associated with different positions of the autonomous mobile robot in the working area, when the autonomous mobile robot comes to a certain position in the working area, the scanning angle ranges corresponding to multiple key frames associated with the autonomous mobile robot at the position can be quickly obtained through the table, and then the scanning angle ranges corresponding to the multiple key frames are compared with the scanning angle ranges corresponding to the current frame collected by the autonomous mobile robot at this time at the position, so as to determine whether the current frame is the key frame associated with the autonomous mobile robot at the position.

[0058] The first table is a table stored inside the device for confirming key frames, in which a plurality of scanning positions and their corresponding scanning angle ranges are recorded, wherein the plurality of scanning positions may be coordinates corresponding to different positions of the autonomous mobile robot in the working area, and the plurality of scanning positions include the first scanning position; the scanning angle range corresponding to a single scanning position is a combination of the scanning angle ranges corresponding to all key frames collected by the autonomous mobile robot at the scanning position within a period of time, and the scanning angle ranges corresponding to the plurality of scanning positions include the second scanning angle range. It should be understood here that before the device for confirming key frames determines that the current frame is a key frame, the scanning angle range corresponding to the first scanning position in the first table (i.e., the second scanning angle range) has no connection with the first scanning angle range.

[0059] See Table 1, which is a possible form of a first table provided in this application.

[0060] Table 1: First table

[0061] coordinate Scanning angle range (5，15) 0 to 60 degrees, 200 to 300 degrees (5，16) 345 degrees to 45 degrees, 220 degrees to 280 degrees …… …… (10，5) 0 to 30 degrees

[0062] Each coordinate in the above Table 1 uniquely corresponds to a scanning angle range. For example, the scanning angle range corresponding to the coordinate (5, 15) is (0 to 60 degrees, 200 to 300 degrees), and the scanning angle range corresponding to the coordinate (5, 16) is (345 to 45 degrees, 220 to 280 degrees). It is explained here that, since the scanning angle range of the autonomous mobile robot is different at different times, the scanning angle range corresponding to the current frame collected by the autonomous mobile robot may be different from the scanning angle range corresponding to the coordinate corresponding to the current frame in the first table. For example, the scanning angle range corresponding to the above coordinate (5, 15) in the first table is (0 to 60 degrees, 200 to 300 degrees), and the scanning angle range corresponding to the current frame captured by the autonomous mobile robot at the coordinate (5, 15) may be (10 to 50 degrees).

[0063] Optionally, the first table may be a hash table, which is a data structure that enables fast data search. The following describes how the device for confirming key frames creates the first table by taking the first table as a hash table.

[0064] The process of creating a first table by a device for confirming a key frame includes: the device for confirming a key frame first constructs a hash table with the coordinates as a key and the scanning angle range corresponding to the key frame associated with the coordinates as a value; when the autonomous mobile robot arrives at a certain position in the working area, the device for confirming the key frame first determines whether the coordinates are recorded in the hash table; if it is determined that the coordinates are not recorded in the hash table, an element in the hash table is created with the coordinates of the position as a key and the scanning angle range of the autonomous mobile robot at the position as a value, and the element is inserted into the hash table; if it is determined that the coordinates are recorded in the hash table, Figure 1 S101 to S103 in the above steps, and when it is determined that the current frame captured at the coordinates of the autonomous mobile robot is a key frame, the scanning angle range corresponding to the current frame is merged with the scanning angle range corresponding to the coordinates in the hash table, and the scanning angle range corresponding to the coordinates in the hash table is updated to the merged scanning angle range.

[0065] It should be understood here that by merging the scanning angle ranges corresponding to all key frames at the same scanning position within a period of time, when selecting a key frame at the scanning position, it is necessary to consider the geometric relationship between the current frame and all previous key frames (in the following step S103, the device for confirming the key frame determines whether the angle difference between the first scanning angle range and the second scanning angle range that does not overlap is greater than or equal to the first preset angle threshold). When the method is used for a long time, the number of key frames can be effectively prevented from growing excessively, so that the CPU occupancy rate will not be too high.

[0066] S103: The device for confirming the key frame determines whether the angle difference between the first scanning angle range and the second scanning angle range that does not overlap is greater than or equal to the first preset angle threshold; when it is determined that the angle difference is greater than or equal to the first preset angle threshold, the device for confirming the key frame determines that the current frame is a key frame associated with the first scanning position; when it is determined that the angle difference is less than the first preset angle threshold, the device for confirming the key frame determines that the current frame is a non-key frame.

[0067] The angle difference value in which the first scanning angle range does not overlap with the second scanning angle range refers to the angle number corresponding to the part of the first scanning angle range that does not belong to the second scanning angle range. For example, the first scanning angle range is (60 degrees to 120 degrees); the second scanning angle range is (90 degrees to 180 degrees, 200 degrees to 300 degrees), then the angle difference value in which the first scanning angle range does not overlap with the second scanning angle range is the angle number corresponding to (60 degrees to 90 degrees), which is 30 degrees.

[0068] The above-mentioned first preset angle threshold can be obtained through an algorithm or through a simulation experiment. The present application does not limit the specific method of obtaining the first preset angle threshold.

[0069] Optionally, the device for confirming a key frame may also determine whether the angle difference between the first scanning angle range and the second scanning angle range that do not overlap is greater than or equal to a first preset angle threshold through the following steps.

[0070] (1) The device for confirming the key frame merges the first scanning angle range with the second scanning angle range to obtain a third scanning angle range.

[0071] The merging of the above scanning angle ranges refers to the merging of the angle ranges for scanning and photographing by the autonomous mobile robot, which is specifically described below by taking the first scanning angle range as (a1, a2) and the second scanning angle range as (b1, b2) as an example.

[0072] When a2 is greater than or equal to b1 and a2 is less than or equal to b2, or b2 is greater than or equal to a1 and b2 is less than or equal to a2, the first scanning angle range is merged with the second scanning angle range, and the angle range corresponding to the obtained third scanning angle range is 1 interval (min(a1, b1), max(a2, b2)), that is, when a2 is greater than or equal to b1 and a2 is less than or equal to b2, the third scanning angle range is (a1, b2); when b2 is greater than or equal to a1 and b2 is less than or equal to a2, the third scanning angle range is (b1, a2).

[0073] When a2 is less than b1, or b2 is less than a1, the first scanning angle range is merged with the second scanning angle range, and the angle range corresponding to the obtained third scanning angle range is 2 intervals ((min(a1, b1), min(a2, b2)), (max(a1, b1), max(a2, b2))), that is, when a2 is less than b1, the third scanning angle range is ((a1, a2), (b1, b2)); when b2 is less than a1, the third scanning angle range is ((b1, b2), (a1, a2)).

[0074] It is noted here that the sizes of the first scanning angle range and the second scanning angle range and the merging process of the third scanning angle range are merely examples provided in the present application and should not be regarded as specific limitations. For example, the first scanning angle range can also be ((a1, a2), (a3, a4)), and the second scanning angle range can also be ((b1, b2), (b3, b4), (b5, b6)).

[0075] (2) The device for confirming the key frame subtracts the number of angles included in the second scanning angle range from the number of angles included in the third scanning angle range to obtain a first angle number.

[0076] The following is a specific description using an example where the third scanning angle range is (a1, b2) and the second scanning angle range is (b1, b2).

[0077] The number of angles included in the third scanning angle range is b2-a1, the number of angles included in the second scanning angle range is b2-b1, and the number of angles included in the third scanning angle range minus the number of angles included in the second scanning angle range is (b2-a1)-(b2-b1).

[0078] It is noted here that the sizes of the angles included in the third scanning angle range and the second scanning angle range and the process of subtracting the angles included in the third scanning angle range from the angles included in the second scanning angle range are merely examples provided by the present application and should not be regarded as specific limitations.

[0079] (3) The device for confirming the key frame determines whether the first angle number is greater than or equal to a first preset angle threshold. When it is determined that the first angle number is greater than or equal to the first preset angle threshold, the current frame is determined to be a key frame associated with the first scanning position; when it is determined that the first angle number is less than the first preset angle threshold, the current frame is determined to be a non-key frame.

[0080] By merging the scanning angle range corresponding to the current frame with the scanning angle range corresponding to the current frame's position in the table, and then subtracting the angle number contained in the scanning angle range corresponding to the current frame's position in the table from the number of angles contained in the merged scanning angle range, the angle difference value (the above-mentioned first angle number) in which the first scanning angle range corresponding to the current frame does not overlap with the second scanning angle range is obtained. When the first angle number is too large, it means that there is a large difference between the image taken by the autonomous mobile robot at the current frame's position and the image taken at the position before, and therefore the current frame is confirmed to be a key frame.

[0081] Optionally, after the device for confirming the key frame determines that the current frame is a key frame, the device for confirming the key frame will also update the second scanning angle range corresponding to the first scanning position in the first table to the third scanning angle range. By merging the scanning angle ranges corresponding to all key frames at the same position within a period of time, when the device for confirming the key frame selects a key frame at the position, it is necessary to consider the geometric relationship between the current frame and all previous key frames. When the method is used for a long time, the number of key frames can be effectively prevented from growing excessively, so that the CPU occupancy rate will not be too high.

[0082] In summary, the present application provides a method for confirming a key frame, which method obtains the scanning position and the first scanning angle range corresponding to the current frame through a device for confirming the key frame; then obtains the second scanning angle range corresponding to the historical key frame of the scanning position; when the angle difference between the first scanning angle range and the second scanning angle range that do not overlap is greater than or equal to a first preset angle threshold, the current frame is determined to be a key frame associated with the scanning position. In the above method for confirming a key frame, the consumption of computing resources is mainly concentrated on calculating the angle difference between the first scanning angle range and the second scanning angle range that do not overlap. The computational complexity and amount of computation of this process are low, and therefore the number of computing resources consumed is small. And when using the above method to confirm whether the current frame is a key frame, it is necessary to consider the relationship between the current frame and multiple key frames associated with the position, so the number of key frames can also be prevented from growing excessively.

[0083] See also Figure 3 , Figure 3 is an architectural diagram of a device including a key frame confirmation device provided by the present application, such as Figure 3 As shown, the autonomous mobile robot 300 includes a device 310 for confirming a key frame and a scanning device 320, wherein the scanning device 320 is used to obtain a current frame and send a scanning position and a scanning angle range corresponding to the current frame to the device 310 for confirming a key frame; the device 310 for confirming a key frame is used to determine whether the current frame is a key frame based on the scanning position and the scanning angle range corresponding to the current frame.

[0084] The key frame confirmation device 310 and the scanning device 320 can be deployed on a computing device, which includes a virtual machine, a container or a server. Among them, a virtual machine is a virtualization technology implemented at the computer software level, which can enable a single physical computer to create multiple virtual operating systems and program application environments, and each virtual machine can independently run its operating system and application program; a container is a lightweight software packaging method for packaging an application program and the application environment of the application program, so that the application program can run in the same way in different environments. Unlike a virtual machine, a container does not contain a complete operating system, but shares the operating system of the physical computer where it is located, which makes the container more lightweight; a server, that is, a general-purpose physical server, includes an ARM server or an X86 server.

[0085] The key frame confirmation device 310 and the scanning device 320 may also be deployed in a computing device cluster, which includes a plurality of the above-mentioned computing devices.

[0086] The key frame confirmation device 310 and the scanning device 320 can also be deployed on terminal devices, including computer terminal devices, mobile terminal devices, network terminal devices, sales terminals (Point Of Sale, POS) devices, industrial control terminal devices and virtual terminal devices, among which computer terminal devices include personal computers, laptops and tablet computers; mobile terminal devices include smart phones, smart watches and portable music players; network terminal devices include routers, switches and modems; sales terminal devices include cash registers, card terminals and self-service payment terminals; industrial control terminal devices include sensors, industrial robots and industrial control panels; virtual terminal devices include virtual reality (VR) devices.

[0087] The key frame confirmation device 310 can be further divided into a plurality of unit modules, such as Figure 3 As shown, the key frame confirmation device 310 also includes a first acquisition unit 301, a second acquisition unit 302 and a confirmation unit 303. It should be understood that Figure 3 The number and names of the unit modules included in the device 310 for confirming a key frame are only examples provided in the present application. The device 310 for confirming a key frame may also include more or fewer unit modules, and the names of the unit modules are not limited to Figure 3 The name of the unit module in, for example, the device 310 for confirming the key frame may further include a calculation unit between the second acquisition unit 302 and the confirmation unit 303, the calculation unit being used to calculate the angle difference between the first scanning angle range and the second scanning angle range that does not overlap, and send the calculated result to the confirmation unit 303; Figure 3The first acquisition unit 301 and the second acquisition unit 302 in the embodiment can be combined into an acquisition unit. The above examples are only for illustration and should not be regarded as specific limitations.

[0088] The above-mentioned first acquisition unit 301, second acquisition unit 302 and confirmation unit 303 can be implemented by software or by hardware. The software and hardware implementation methods of the first acquisition unit 301 are introduced below. The software and hardware implementation methods of the second acquisition unit 302 and the confirmation unit 303 can refer to the software and hardware implementation methods of the first acquisition unit 301.

[0089] When the first acquisition unit 301 is implemented by software, the first acquisition unit 301 may be a code running on the above-mentioned computing device or terminal device, that is, the first acquisition unit 301 may be a code running on a personal computer, a smart phone or a server, and the number of computing devices or terminal devices may be one or more, that is, the first acquisition unit 301 may also be a code running on a computing device cluster.

[0090] When the first acquisition unit 301 is implemented by hardware, the first acquisition unit 301 can be implemented by at least one computing device or terminal device. Alternatively, the first acquisition unit 301 can also be implemented by a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), wherein the PLD includes one or more of a complex programmable logical device (CPLD), a field programmable gate array (FPGA), and a generic array logic (GAL).

[0091] The following takes the autonomous mobile robot 300 as a lawn mower and the working area as a lawn as an example to explain the functions of the first acquisition unit 301, the second acquisition unit 302 and the confirmation unit 303 in the key frame confirmation device 310. The above-mentioned lawn mower has a walking device 5, a cutting device, a control device 6, and a scanning device 320, wherein the walking device 5 is used to drive the lawn mower to walk in the working area, and the cutting device is used to cut the grass on the lawn. It is located at the bottom of the lawn mower. Figure 5 Not shown in the figure, the scanning device 320 is used to obtain the environmental information of the working area, and the control device 6 is used to control the lawn mower to execute the corresponding instructions to perform the operation. The control device 6 has the above-mentioned key frame confirmation device 310.

[0092] The first acquisition unit 301 is used to acquire a first scanning position and a first scanning angle range corresponding to a current frame.

[0093] For an introduction to the current frame, the first scanning position, and the first scanning angle range, see Figure 1 The relevant description of step S101 in the present application will not be repeated here.

[0094] The specific steps for the first acquisition unit 301 to acquire the first scanning position and the first scanning angle range corresponding to the current frame include: after the lawn mower reaches a certain position in the lawn, the image at the position is scanned and photographed by the scanning device 320; the scanning device 320 sends information including the current position of the lawn mower (first scanning position) and the scanning angle range of the scan (first scanning angle range) to the device 310 for confirming the key frame; the first acquisition unit 301 receives and parses the information, thereby acquiring the current position of the lawn mower and the scanning angle range of the scan, and sends the parsed information to the second acquisition unit 302.

[0095] The scanning device 320 may send information including the location of the lawn mower and the scanning angle range of the scanned image to the device 310 for confirming key frames in a wireless or wired manner, wherein the wired manner includes Ethernet, Universal Serial Bus (USB) connection, serial communication, etc.; the wireless manner includes wireless network (Wireless Fidelity, Wi-Fi), Bluetooth, wireless sensor network, etc. The present application does not limit the specific manner in which the scanning device 320 sends information to the device 310 for confirming key frames, and the present application does not limit the manner in which the scanning device 320 sends information to the device 310 for confirming key frames.

[0096] The second acquisition unit 302 is used to acquire a second scanning angle range corresponding to the historical key frame of the first scanning position.

[0097] For an introduction to the above historical keyframes and the second scanning angle range, see Figure 1 The relevant description of step S102 in the present application will not be repeated here.

[0098] The specific steps for the second acquisition unit 302 to acquire the second scanning angle range corresponding to the historical key frame of the first scanning position include: the second acquisition unit 302 queries the first table according to the first scanning position, and when the second acquisition unit 302 queries the position record identical to the first scanning position in the first table, acquires the scanning angle range corresponding to the position in the first table, and sends the first scanning angle range and the scanning angle range (second scanning angle range) corresponding to the current position of the lawn mower in the first table to the confirmation unit 303; when the second acquisition unit 302 does not query the position record identical to the first scanning position in the first table, the first scanning position and the first scanning angle range are combined into a new element and inserted into the first table.

[0099] For an introduction to the first table, see Figure 1 The relevant description of step S102 in the present application will not be repeated here.

[0100] Optionally, after acquiring the first scanning position and the first scanning angle range corresponding to the current frame, the second acquisition unit 302 will also determine whether there is a corresponding historical key frame at the first scanning position of the lawn mower. When it is determined that there is no historical key frame at the first scanning position of the lawn mower, the second acquisition unit 302 determines that the current frame is the key frame associated with the lawn mower at the first scanning position. It should be understood here that since the method provided by the present application compares the current frame of a certain position with all the key frames before the position, when there is no corresponding historical key frame at a certain position, it means that the lawn mower has come to the position for the first time, and the pictures taken by its scanning can be merged to generate a circular key frame.

[0101] Optionally, after acquiring the second scanning angle range corresponding to the historical key frame at the first scanning position, the second acquisition unit 302 will also determine whether the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold. When it is determined that the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold, it is confirmed that the lawn mower does not need to add a new key frame at the first scanning position, and the current frame is confirmed as a non-key frame. It should be understood that the second acquisition unit 302 determines whether the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold. When the number of angles included in the second scanning angle range is large enough, it means that the multiple key frames taken by the lawn mower at this position can be merged into a circular key frame for the self-positioning and environment mapping of the lawn mower. By setting the second preset angle threshold to determine the size of the number of angles included in the second scanning angle range, the number of key frames associated with the lawn mower at this position will not be too many, thereby reducing the memory consumption of the entire device.

[0102] The above-mentioned second preset angle threshold can be obtained through an algorithm or through a simulation experiment. The present application does not limit the specific method of obtaining the second preset angle threshold.

[0103] The number of angles included in the above scanning angle range is the angle size included in the angle range of scanning and photographing by the scanning device 320.

[0104] Optionally, the first table can be a hash table, which is a data structure that can realize fast data search. When the first table is a hash table, the process of the device 310 for confirming the key frame creating the first table can include: the second acquisition unit 302 first constructs a hash table with the coordinates as the key and the scanning angle range as the value. When the lawn mower arrives at a certain position in the working area, the second acquisition unit 302 first determines whether the coordinates of the position are recorded in the hash table. When it is determined that the coordinates of the position are not recorded in the hash table, the coordinates of the position are used as the key and the scanning angle range of the position by the scanning device 320 is used as the value to create an element in the hash table, and the element is inserted into the hash table; when it is determined that the coordinates are recorded in the hash table, the scanning angle range corresponding to the coordinates in the hash table is obtained, and the scanning angle range of the scanning device 320 at the position and the corresponding scanning angle range in the hash table are sent to the confirmation unit 303.

[0105] The confirmation unit 303 is used to determine whether the angle difference between the first scanning angle range and the second scanning angle range that does not overlap is greater than or equal to a first preset angle threshold.

[0106] The confirmation unit 303 determines whether the angle difference between the first scanning angle range and the second scanning angle range that does not overlap is greater than or equal to the first preset angle threshold. Specifically, the confirmation unit 303 merges the first scanning angle range with the second scanning angle range to obtain a third scanning angle range; the confirmation unit 303 subtracts the angle number included in the second scanning angle range from the angle number included in the third scanning angle range to obtain the first angle number; the confirmation unit 303 determines whether the first angle number is greater than or equal to the first preset angle threshold, and when it is determined that the first angle number is greater than or equal to the first preset angle threshold, determines that the current frame is a key frame associated with the first scanning position; when it is determined that the first angle number is less than the first preset angle threshold, determines that the current frame is a non-key frame.

[0107] The specific operation logic of the above confirmation unit 303 is as follows:

[0108] ① The confirmation unit 303 obtains the scanning interval corresponding to each scanning angle range.

[0109] For each scanning angle range, the confirmation unit 303 uses one or more scanning intervals to represent. When the scanning angle range corresponds to one scanning interval, the form of the scanning interval is (θ-start, θ-end), where θ-start represents the starting angle of the scanning interval; θ-end represents the ending angle of the scanning interval. For example, the scanning interval corresponding to the first scanning angle range is (30 degrees to 90 degrees). When the scanning angle range corresponds to multiple scanning intervals, the form of the scanning interval is ((θ-start-1, θ-end-1), (θ-start-2, θ-end-2), ..., (θ-start-n, θ-end-n)), where θ-start-1 represents the starting angle of the first scanning interval, and θ-end-1 represents the ending angle of the first scanning interval. The symbols in the subsequent intervals are similar, and no further details are given here. For example, the scanning interval corresponding to the second scanning angle range can be ((30 degrees to 90 degrees), (100 degrees to 150 degrees)).

[0110] ② The confirmation unit 303 merges the scanning interval corresponding to the first scanning angle range and the scanning interval corresponding to the second scanning angle range according to the defined merging operation to obtain a third scanning angle range.

[0111] For example, the scanning interval corresponding to the first scanning angle range is (θ1-start, θ1-end), and the scanning interval corresponding to the second scanning angle range is (θ2-start, θ2-end).

[0112] When θ1-start is greater than θ2-end, the expression of the merged scanning interval is ((θ2-start, θ2-end), (θ1-start, θ1-end)). For example, the scanning interval corresponding to the first scanning angle range is (30 degrees to 90 degrees), and the scanning interval corresponding to the second scanning angle range is (10 degrees to 20 degrees). Then the scanning interval corresponding to the first scanning angle range and the scanning interval corresponding to the second scanning angle range after merging is ((10 degrees to 20 degrees), (30 degrees to 90 degrees)).

[0113] When θ2-start is greater than θ1-end, the expression of the merged scanning interval is ((θ1-start, θ1-end), (θ2-start, θ2-end)). For example, the scanning interval corresponding to the first scanning angle range is (30 degrees to 90 degrees), and the scanning interval corresponding to the second scanning angle range is (100 degrees to 120 degrees). Then the scanning interval corresponding to the first scanning angle range and the scanning interval corresponding to the second scanning angle range after merging is ((30 degrees to 90 degrees), (100 degrees to 120 degrees)).

[0114] When θ1-end is greater than or equal to θ2-start or θ2-end is greater than or equal to θ1-start, the expression of the merged scanning interval is (min(θ1-start, θ2-start), max(θ1-end, θ2-end)). For example, the scanning interval corresponding to the first scanning angle range is (30 degrees to 90 degrees), and the scanning interval corresponding to the second scanning angle range is (60 degrees to 120 degrees). Then the scanning interval corresponding to the first scanning angle range and the scanning interval corresponding to the second scanning angle range after merging is (30 degrees to 120 degrees).

[0115] It should be understood here that since the first scanning angle range and the second scanning angle range can both correspond to one or more scanning intervals, the merging of the scanning interval corresponding to the first scanning angle range and the scanning interval corresponding to the second scanning angle range may include one or more merging. Exemplarily, taking the scanning interval corresponding to the first scanning angle range as (50 degrees to 170 degrees), and the scanning interval corresponding to the second scanning angle range as ((0 degrees to 60 degrees), (80 degrees to 200 degrees), (300 degrees to 360 degrees)) as an example, when the scanning interval corresponding to the first scanning angle range and the scanning interval corresponding to the second scanning angle range are merged, the scanning interval (0 degrees to 60 degrees) is first merged with (50 degrees to 170 degrees) to obtain the scanning interval (0 degrees to 17 0 degrees); then merge the scanning interval (0 degrees to 170 degrees) and (80 degrees to 200 degrees) to obtain the scanning interval (0 degrees to 200 degrees); since 200 degrees is less than 300 degrees, the scanning intervals (0 degrees to 200 degrees) and (300 degrees to 360 degrees) cannot be merged into one scanning interval; finally, the scanning interval corresponding to the first scanning angle range and the scanning interval corresponding to the second scanning angle range are merged to obtain the scanning interval ((0 degrees to 200 degrees), (300 degrees to 360 degrees)).

[0116] ③ The confirmation unit 303 calculates the difference between the scanning intervals before and after the merging.

[0117] After merging the scanning interval corresponding to the first scanning angle range and the scanning interval corresponding to the second scanning angle range, the confirmation unit 303 determines whether the angle difference between the first scanning angle range and the second scanning angle range that does not overlap is greater than or equal to the first preset angle threshold by comparing the difference between the scanning intervals corresponding to the third scanning angle range and the second scanning angle range.

[0118] The following takes the merged scanning interval as ((θ1_start_1, θ1_end_1), (θ2_start_1, θ2_end_1, …, (θm_start_1, θm_end_1)) and the scanning interval corresponding to the second scanning angle range as ((θ1_start_2, θ1_end_2), (θ2_start_2, θ2_end_2), …, (θn_start_2, θn_end_2)) as an example to specifically illustrate how the confirmation unit 303 calculates the difference before and after the scanning interval merger.

[0119] The confirmation unit 303 calculates the number of angles included in the second scanning angle range before merging by using formula one.

[0120]

[0121] The above-mentioned Total_angle_before represents the number of angles contained in the scanning interval corresponding to the second scanning angle range; θi_end_2 represents the end angle of the i-th sub-interval in the scanning interval corresponding to the second scanning angle range; θi_start_2 represents the starting angle of the i-th sub-interval in the scanning interval corresponding to the second scanning angle range, and the value range of i in Formula 1 is 1 to n.

[0122] The confirmation unit 303 calculates the number of angles included in the combined third scanning angle range through Formula 2.

[0123]

[0124] The above-mentioned Total_angle_after represents the number of angles contained in the scanning interval corresponding to the third scanning angle range; θi_end_1 represents the end angle of the i-th sub-interval in the scanning interval after the merger; θi_start_1 represents the starting angle of the i-th sub-interval in the scanning interval after the merger, and the value range of i in Formula 2 is 1 to m.

[0125] The confirmation unit 303 calculates the difference before and after the merger using Formula 3.

[0126] difference=Total_angle_after-Total_angle_before formula three

[0127] The above difference is the difference between the scanning intervals before and after the merging. For Total_angle_after and Total_angle_before, please refer to the relevant descriptions in the above formulas 1 and 2, which will not be described in detail here.

[0128] ④ The confirmation unit 303 compares the difference between the scanning interval corresponding to the third scanning angle range and the scanning interval corresponding to the second scanning angle range with the first preset angle threshold to determine whether the current frame is a key frame associated with the first scanning position.

[0129] The above-mentioned first preset angle threshold can be obtained through manual experiments or through a computer algorithm, and the present application does not limit the acquisition method.

[0130] The following is a specific introduction using the first preset angle threshold as 30 degrees.

[0131] When the difference obtained by the confirmation unit 303 through the above formula three is 60 degrees, since 60 degrees is greater than 30 degrees, it means that there is a large difference between the scanning angle range corresponding to the current frame and the scanning angle range recorded in the first table, that is, the lawn mower has collected new image information at the position of the current frame, and therefore the current frame is determined to be a key frame.

[0132] When the difference obtained by the confirmation unit 303 through the above formula three is 20 degrees, since 20 degrees is less than 30 degrees, it means that the difference between the scanning angle range corresponding to the current frame and the scanning angle range recorded in the first table is small, that is, the lawn mower has not collected or only collected a small part of the new image information at the position of the current frame, and therefore the current frame is determined to be a non-key frame.

[0133] Optionally, after the confirmation unit 303 determines that the current frame is a key frame, the confirmation unit 303 also updates the second scanning angle range corresponding to the first scanning position in the first table to a third scanning angle range. By merging the scanning angle ranges corresponding to all key frames at the same position within a period of time, when the confirmation unit 303 selects a key frame at the position, it is necessary to consider the geometric relationship between the scanning angle range corresponding to the current frame and the scanning angle ranges corresponding to all previous key frames, which can effectively prevent the number of key frames from growing excessively.

[0134] In summary, the present application provides a device for confirming a key frame, wherein the device 310 for confirming a key frame obtains the scanning position and the first scanning angle range corresponding to the current frame through the first acquisition unit 301; the second acquisition unit 302 obtains the second scanning angle range corresponding to the historical key frame of the scanning position; the confirmation unit 303 determines that the current frame is a key frame associated with the scanning position when the angle difference between the first scanning angle range and the second scanning angle range is greater than or equal to the first preset angle threshold. In the above-mentioned device 310 for confirming a key frame, the consumption of computing resources is mainly concentrated in the confirmation unit 303 calculating the angle difference between the first scanning angle range and the second scanning angle range. The computational complexity of the process is low and the amount of computation is small, so the execution efficiency of the entire method is high and the CPU occupancy rate is low. And when using the above-mentioned device to confirm whether the current frame is a key frame, it is necessary to consider the relationship between the current frame and multiple key frames associated with the position, so the number of key frames can also be prevented from growing excessively.

[0135] See also Figure 4 , Figure 4 4 is a schematic diagram of the structure of an electronic device provided by the present application. The electronic device 400 includes a processor 401, a memory 402, a communication interface 403 and a bus 404. The processor 401, the memory 402 and the communication interface 403 can be interconnected through an internal bus 404, or can communicate through wireless transmission or other means.

[0136] Processor 401 may be composed of at least one general-purpose processor, such as a central processing unit (CPU), or a combination of a CPU and a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Processor 401 is used to execute various types of digital storage instructions. Processor 401 may implement the following by executing corresponding instructions: Figure 1 The method shown.

[0137] The memory 402 may be a volatile memory, such as a random access memory (RAM), a dynamic RAM (DRAM), a static RAM (SRAM), a synchronous dynamic RAM (SDRAM), a double data rate RAM (DDR), a cache, etc. The memory 402 may also include a combination of the above types. The memory 402 may include programs and data. The processor 401 may execute program codes to execute Figure 1 The method shown.

[0138] The communication interface 403 can be used for data interaction between the electronic device 400 and the autonomous mobile robot. For example, the electronic device 400 can receive information sent by a scanning device through the communication interface 403, including the current location of the autonomous mobile robot and the scanning angle range corresponding to the scanning and shooting. The specific communication process is not specifically limited in this application.

[0139] The bus 404 is a Peripheral Component Interconnect Express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus 404 is divided into an address bus, a data bus, a control bus, etc.

[0140] The bus 404 includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity, various buses are labeled as the bus 404 in the figure.

[0141] Need to explain, Figure 4 This is only one possible implementation of the embodiment of the present application. In actual application scenarios, the electronic device 400 may also include more or fewer components, which is not limited here.

[0142] See also Figure 5 , Figure 5Schematic diagram of the structure of a lawn mower provided by the present application. The lawn mower provided by the present application includes a walking device 5, a scanning device 320, a cutting device ( Figure 5 The walking device 5 is used to drive the lawn mower to walk in the working area, the cutting device is used to cut the grass of the lawn, and the scanning device 320 is used to obtain the environmental information of the working area. The control device 6 has the above Figure 3 The key frame confirmation device 310 or the above Figure 4 The electronic device 400 can achieve the above Figure 1 How to confirm key frames in the lawn mower Figure 1 The method for confirming key frames in can be found above Figure 3 The relevant contents in the present application will not be elaborated in detail here.

[0143] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes a method for confirming a key frame.

[0144] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a high-density digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute a method for confirming a key frame.

[0145] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes a plurality of computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0146] The above is a detailed introduction to the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for confirming a key frame, characterized in that: The method comprises: Obtaining a first scanning position and a first scanning angle range corresponding to a first current frame; Acquire a second scanning angle range corresponding to a historical key frame of the first scanning position, wherein the historical key frame is a plurality of key frames associated with the scanning position; When the angle difference between the first scanning angle range and the second scanning angle range that does not overlap is greater than or equal to a first preset angle threshold, the first current frame is determined to be a key frame associated with the first scanning position.

2. The method according to claim 1, characterized in that: When the angle difference between the first scanning angle range and the second scanning angle range that does not overlap is greater than or equal to a first preset angle threshold, determining that the first current frame is a key frame associated with the first scanning position includes: Combining the first scanning angle range with the second scanning angle range to obtain a third scanning angle range; Subtract the number of angles included in the second scanning angle range from the number of angles included in the third scanning angle range to obtain a first angle number; When the first angle is greater than or equal to a first preset angle threshold, the first current frame is determined to be a key frame associated with the first scanning position.

3. The method according to claim 1 or 2, characterized in that: The obtaining of the second scanning angle range corresponding to the historical key frame of the first scanning position includes: querying a first table according to the first scanning position, wherein the first table includes a plurality of scanning positions and scanning angle ranges corresponding to a plurality of key frames associated with each scanning position; In a case where the plurality of scanning positions include the first scanning position, a second scanning angle range corresponding to a historical key frame of the first scanning position is acquired.

4. The method according to claim 3, characterized in that After determining that the first current frame is a key frame associated with the first scanning position, the method further includes: Combining the first scanning angle range with the second scanning angle range to obtain a third scanning angle range; The scanning angle range corresponding to the first scanning position in the first table is updated to the third scanning angle range.

5. The method according to claim 3, characterized in that: The method further comprises: Obtaining a second scanning position and a fourth scanning angle range corresponding to a second current frame; Determining whether the second scanning position has the historical key frame; When it is determined that the second scanning position does not have the historical key frame, the second current frame is determined to be a key frame associated with the second scanning position.

6. The method according to claim 5, characterized in that The determining whether the second scanning position has the historical key frame includes: querying the first table according to the second scanning position; When the multiple scanning positions do not include the second scanning position, the second current frame is determined to be a key frame associated with the second scanning position, and the second scanning position, the fourth scanning angle range, and the correspondence between the second scanning position and the fourth scanning angle are updated to the first table.

7. The method according to claim 6, characterized in that After acquiring the second scanning angle range corresponding to the historical key frame of the first scanning position, the method further includes: Determine whether the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold. When it is determined that the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold, confirm that no new key frame is needed for the first scanning position, and confirm the first current frame as a non-key frame.

8. The method according to claim 7, characterized in that The first table is a hash table. Before obtaining the first scanning position and the first scanning angle range corresponding to the first current frame, the method further includes: Acquire multiple scanning positions in the working area and multiple key frames associated with each of them; Calculating a merge of scanning angle ranges corresponding to a plurality of key frames associated with each scanning position in the plurality of scanning positions; The hash table is constructed, wherein the hash table includes a plurality of key-value pairs, the key of each key-value pair corresponds to a scanning position, and the value of each key-value pair is a combination of scanning angle ranges corresponding to a plurality of key frames associated with the scanning position.

9. A device for confirming a key frame, characterized in that: The device comprises: A first acquisition unit, used to acquire a first scanning position and a first scanning angle range corresponding to a first current frame; A second acquisition unit, configured to acquire a second scanning angle range corresponding to a historical key frame of the first scanning position, wherein the historical key frame is a plurality of key frames associated with the scanning position; A confirmation unit is used to determine that the first current frame is a key frame associated with the first scanning position when the angle difference between the first scanning angle range and the second scanning angle range is greater than or equal to a first preset angle threshold.

10. The device according to claim 9, characterized in that The confirmation unit is specifically used for: Combining the first scanning angle range with the second scanning angle range to obtain a third scanning angle range; Subtract the number of angles included in the second scanning angle range from the number of angles included in the third scanning angle range to obtain a first angle number; When the first angle is greater than or equal to a first preset angle threshold, the first current frame is determined to be a key frame associated with the first scanning position.

11. The device according to claim 9 or 10, characterized in that The second acquisition unit is specifically used for: querying a first table according to the first scanning position, wherein the first table includes a plurality of scanning positions and scanning angle ranges corresponding to a plurality of key frames associated with each scanning position; In a case where the plurality of scanning positions include the first scanning position, a second scanning angle range corresponding to a historical key frame of the first scanning position is acquired.

12. The device according to claim 11, characterized in that After determining that the first current frame is a key frame associated with the first scanning position, the confirmation unit is further used to: Combining the first scanning angle range with the second scanning angle range to obtain a third scanning angle range; The scanning angle range corresponding to the first scanning position in the first table is updated to the third scanning angle range.

13. The device according to claim 11, characterized in that The first acquisition unit is further used to acquire a second scanning position and a fourth scanning angle range corresponding to the second current frame; The second acquisition unit is further used to determine whether the second scanning position has the historical key frame; When it is determined that the second scanning position does not have the historical key frame, the second current frame is determined to be a key frame associated with the second scanning position.

14. The device according to claim 13, characterized in that The second acquisition unit is specifically used for: querying the first table according to the second scanning position; When the multiple scanning positions do not include the second scanning position, the second current frame is determined to be a key frame associated with the second scanning position, and the second scanning position, the fourth scanning angle range, and the correspondence between the second scanning position and the fourth scanning angle are updated to the first table.

15. The device according to claim 14, characterized in that The second acquisition unit is further used for: Determine whether the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold. When it is determined that the number of angles included in the second scanning angle range is greater than or equal to the second preset angle threshold, confirm that no new key frame is needed for the first scanning position, and confirm the first current frame as a non-key frame.

16. The device according to claim 15, characterized in that The first table is a hash table. Before obtaining the first scanning position and the first scanning angle range corresponding to the first current frame, the first obtaining unit is further used to: Acquire multiple scanning positions in the working area and multiple key frames associated with each of them; Calculating a merge of scanning angle ranges corresponding to a plurality of key frames associated with each scanning position among the plurality of scanning positions; The hash table is constructed, wherein the hash table includes a plurality of key-value pairs, the key of each key-value pair corresponds to a scanning position, and the value of each key-value pair is a combination of scanning angle ranges corresponding to a plurality of key frames associated with the scanning position.

17. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores computer program instructions, and the processor implements the method according to any one of claims 1 to 8 by executing the computer program instructions.

18. A lawn mower, characterized in that: The lawn mower comprises the device according to any one of claims 9 to 16 or the electronic device according to claim 17.

19. A computer program product comprising instructions, characterized in that The computer program product is a software or program product containing instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one electronic device, the at least one electronic device executes the method as described in any one of claims 1 to 8.

20. A computer-readable storage medium, characterized in that: The method comprises computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images