A rapid target detection and localization method, device and unmanned aerial vehicle system
By combining target detection models and depth information, the three-dimensional attitude of UAV targets can be quickly obtained, solving the problems of slow computing speed and high resource consumption in UAV systems. This achieves efficient target recognition and localization, and is suitable for complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA NANHU ACAD OF ELECTRONICS & INFORMATION TECH
- Filing Date
- 2022-08-10
- Publication Date
- 2026-06-02
AI Technical Summary
Unmanned aerial vehicle (UAV) systems suffer from slow computation speed and high resource consumption during target detection and localization, making it difficult to meet real-time requirements, especially in complex environments where it is difficult to obtain complete target information.
By combining a target detection model with scene depth information, and through a target space mapping model and a pose mapping model, the three-dimensional pose information of the target can be quickly obtained, reducing the amount of computation and resource consumption.
It improves the target recognition and positioning efficiency of the UAV system, extends the mission endurance of the UAV, and is suitable for complex environments.
Smart Images
Figure CN115439535B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of target detection and image recognition technology, specifically relating to target detection technology for unmanned aerial vehicle (UAV) platforms, and particularly to a rapid target detection and localization method, device, and UAV system. Background Technology
[0002] Unmanned aerial vehicles (UAVs) are characterized by their small size, light weight, and high flexibility, and have been widely used in various fields such as military, counter-terrorism, agriculture, and wildlife conservation. Fully autonomous UAV systems that do not require real-time human operator control have become an important direction in UAV system research.
[0003] When performing missions, unmanned aerial vehicle (UAV) systems typically acquire images of the entire scene, then reconstruct the scene using point cloud or binocular vision technology. Finally, they perform tasks such as target recognition and action execution based on the reconstructed scene. For example, CN114359707A discloses an obstacle recognition method suitable for UAV systems, which acquires dense point clouds of the scene from images captured by the UAV and then performs obstacle recognition. However, this process is slow and resource-intensive (considering the computing resources of the onboard computer and battery power), limiting the application of UAVs. Furthermore, CN113569766A discloses a method for detecting abnormal pedestrian behavior in UAV patrols. This method uses a spatiotemporal graph convolutional architecture to describe the changes in human skeleton maps over time, thereby predicting human skeleton features within future frames. This method has certain advantages in simple environments such as urban roads, but it is difficult to obtain complete human skeleton information in complex environments.
[0004] Therefore, existing solutions either have high requirements for the completeness of the detected object, such as needing to obtain complete human skeleton information to determine the target's pose information, making the solution difficult to implement in complex environments, or they need to use point cloud and other technologies to obtain the object's pose information, which is slow and cannot meet real-time requirements. Summary of the Invention
[0005] This invention discloses a rapid target detection and localization method, device, and unmanned aerial vehicle (UAV) system, aiming to solve the problems of slow target detection and localization process in current UAV systems during mission execution, which cannot meet real-time requirements, as well as high resource consumption, which affects the UAV's endurance.
[0006] According to a first aspect of the present invention, a target detection and localization method is disclosed, the method comprising:
[0007] Acquire a 2D scene image and scene depth information of the predetermined scene;
[0008] The scene image is processed using an object detection model to identify the target and its key features;
[0009] Solve for the target information of the target and the key feature information of the key features;
[0010] Based on the target information and the key feature information, the spatial three-dimensional pose information of the target is obtained using a target space mapping model;
[0011] The depth information of the target and the key features relative to the UAV is calculated based on the scene depth information;
[0012] The spatial pose information of the target relative to the UAV is calculated based on the spatial three-dimensional attitude information and the depth information of the target and its key features relative to the UAV.
[0013] In some examples, the target information includes the target's center point, length, width, and average length and width; the key feature information includes the key feature's center point, length, and width.
[0014] In some examples, the target space mapping model is a pose mapping model for representing the target's pose from a two-dimensional image to a three-dimensional pose, based on the relationship between the target information and its key feature information.
[0015] In some examples, the spatial pose information is also converted into the spatial pose information of the target relative to one of the drones by using a spatial pose change matrix between the two drones.
[0016] In some examples, the spatial pose information is converted into the target's absolute pose information in three-dimensional space by utilizing the drone's own pose information.
[0017] According to a second aspect of the present invention, a target detection and positioning device is disclosed, the device comprising:
[0018] The data acquisition module is configured to acquire a two-dimensional scene image and scene depth information of a predetermined scene;
[0019] The target detection module is configured to process the scene image using a target detection model to identify targets and key features of the targets;
[0020] The first solution module is configured to solve for the target information of the target and the key feature information of the key features;
[0021] The attitude mapping module is configured to obtain the spatial three-dimensional attitude information of the target using a target space mapping model based on the target information and the key feature information.
[0022] The second solving module is configured to solve for the depth information of the target and the key features relative to the UAV based on the scene depth information;
[0023] The positioning module is configured to solve the spatial pose information of the target relative to the UAV based on the spatial three-dimensional attitude information and the depth information of the target and its key features relative to the UAV.
[0024] In some examples, the data acquisition module includes a two-dimensional image acquisition unit and a depth information acquisition unit, or a data interface connected to the two-dimensional image acquisition unit and the depth information acquisition unit.
[0025] In some examples, the localization module uses a spatial pose change matrix between the two drones to convert the spatial pose information into spatial pose information of the target relative to one of the drones.
[0026] In some examples, the positioning module utilizes the UAV's own pose information to convert the spatial pose information into the target's absolute pose information in three-dimensional space.
[0027] According to a third aspect of the present invention, an unmanned aerial vehicle (UAV) system includes an UAV body, an image acquisition unit for acquiring two-dimensional images of a scene, a depth information acquisition unit for acquiring scene depth information, and a target detection and positioning device according to any one of the above-described solutions.
[0028] This invention has the advantages of low computational load, high computational speed, and low resource consumption, and can improve the target recognition and positioning efficiency of UAV systems and the mission endurance of UAVs.
[0029] Furthermore, this invention is applicable to multiple fields such as robot grasping and wildlife protection, and has good versatility and scalability. Attached Figure Description
[0030] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0031] Figure 1 This is a schematic diagram of the composition of an unmanned aerial vehicle (UAV) system according to an embodiment of the present invention;
[0032] Figure 2 This is a schematic flowchart of a target detection and localization method according to an embodiment of the present invention;
[0033] Figure 3 This is a schematic diagram of the target detection and positioning device 30 according to an embodiment of the present invention;
[0034] Figure 4 This is a schematic diagram of the predicted bounding box of a target in the YOLO model.
[0035] Figure 5 This is a coordinate diagram of Embodiment 1 of the present invention;
[0036] Figure 6 This is a schematic diagram of human head detection and positioning in Embodiment 2 of the present invention. Detailed Implementation
[0037] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.
[0038] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0039] Figure 1 This is a schematic diagram of the composition of an unmanned aerial vehicle (UAV) system according to an embodiment of the present invention. As shown in the figure, the UAV system includes a UAV body 10, an image acquisition unit 20 (e.g., an RGB camera) for acquiring two-dimensional images of a scene, a laser rangefinder 40 for acquiring scene depth information, and a target detection and positioning device 30. The target detection and positioning device 30 is used to acquire the scene image and scene depth information to detect and locate targets in the scene.
[0040] Alternatively, an RGBD depth camera can be used to simultaneously acquire color and depth images of the scene, thus simplifying the system architecture.
[0041] Figure 2 This is a schematic flowchart of a target detection and localization method according to an embodiment of the present invention. As shown in the figure, the method is implemented by the target detection and localization device 30 and includes the following steps:
[0042] Step 201: Obtain scene images and scene depth information of the predetermined scene;
[0043] The target detection and positioning device 30 acquires a color image (containing the target to be detected and positioned) acquired by the image acquisition unit 20 for a predetermined scene and the depth information of the scene measured by the laser rangefinder 40.
[0044] Step 202: Process the scene image using an object detection model to identify the target and its key features;
[0045] For example, the object detection model employs the YOLO algorithm, such as YOLO v5. The YOLO algorithm uses a single CNN model to achieve end-to-end object detection, which, compared to the R-CNN algorithm, provides a unified framework and is faster. Unlike YOLO series 1-4, YOLO v5 uses a cross-domain grid and matches across different output layers, significantly increasing the number of positive anchors, accelerating model convergence, and improving model recall.
[0046] It is understood that the object detection model is a trained model, and its training process includes, for example, the following steps:
[0047] 1) Based on the requirements of the UAV mission, collect images of the target objects and construct a dataset;
[0048] If necessary, the images in the dataset are cleaned and organized to remove unsuitable images in order to improve the accuracy and reliability of model training.
[0049] 2) Use annotation tools to annotate the target and its key features for each image in the dataset;
[0050] The key features are one or more features of the target, and these features are the most representative of the target's characteristics among all its features. It is understood that different targets have different key features. For example, the key features of an apple could be its stem or calyx. The key features of a human head could be facial features or facial features, etc. Furthermore, depending on the different purposes of the drone mission, the key features selected for the same type of target may also differ.
[0051] 3) Divide the dataset into training and testing sets, and select a suitable object detection model, such as the YOLO model, for training to obtain a well-trained object detection model.
[0052] The target detection model based on the YOLO algorithm has the characteristics of high speed and high accuracy, and can meet the real-time requirements.
[0053] Step 203: Solve for the target information of the target and the key feature information of the key features;
[0054] Based on the target information output by the target detection model, the target's center point, length, width, and average length and width can be calculated. Since the required parameters may differ for different objects, taking a spherical fruit as an example, when simplifying it to a spherical model during the solution process, the length and width of the rectangle output by the target detection model are actually the fruit's diameter in the horizontal and vertical directions. Therefore, calculating the average length and width can provide a more accurate parameter for the fruit's diameter.
[0055] Based on the key feature information output by the target detection model, the center point, length, width, and other information of the target's key features can be calculated.
[0056] The specific process of solving the above problem can be achieved by conventional methods, which is not the focus of this invention and will not be elaborated here.
[0057] Step 204: Based on the target information and the key feature information, use the target space mapping model to obtain the spatial three-dimensional pose information of the target;
[0058] For example, this invention employs a target space mapping model based on expert knowledge. Based on the target object of the UAV mission, the relationship between target information (including length, width, average length and width, target center point, etc.) and its key feature information (length, width, center point, etc.) is analyzed to establish an attitude mapping model of the target from a two-dimensional pattern to a three-dimensional pose. This attitude mapping model can be, for example, a mapping table, which represents the attitude mapping relationship of the target from a two-dimensional pattern to a three-dimensional pose.
[0059] Therefore, after obtaining the target information and the key feature information, the spatial three-dimensional pose information of the target can be obtained using the target space mapping model.
[0060] Step 205: Solve for the depth information of the target and the key features relative to the UAV based on the scene depth information;
[0061] Based on the target and key feature area information output by the target detection model and the scene depth information obtained by the laser rangefinder 40, the depth information of the target and key features relative to the UAV is solved.
[0062] Step 206: Solve the spatial pose information of the target relative to the UAV based on the spatial three-dimensional attitude information and the depth information of the target and its key features relative to the UAV.
[0063] If the task is performed by the current UAV, the UAV performs the task based on the spatial pose information of the target relative to the UAV obtained by solving.
[0064] If the task is performed by other drones within the cluster, the spatial pose information is converted into the target's spatial pose information relative to the task drone using the spatial pose change matrix between the current drone and the task drone (i.e., the other drones).
[0065] If the task is performed by a system other than a drone or drone swarm, the spatial pose information is converted into the target's absolute pose information in three-dimensional space using the drone's own pose information (e.g., a transformation matrix relative to a ground station or other target).
[0066] Figure 3 This is a schematic diagram of the target detection and positioning device 30 according to an embodiment of the present invention. As shown in the figure, the device includes:
[0067] The data acquisition module 301, connected to the image acquisition unit 20 and the laser rangefinder 40, is configured to acquire scene images and scene depth information of a predetermined scene. It can be understood that this data acquisition module can be a separate data interface or an integrated data interface on the image acquisition unit 20 and the laser rangefinder 40.
[0068] The target detection module 302 is configured to process the scene image using a target detection model to identify targets and key features of the targets;
[0069] For example, the target detection model employs the YOLO algorithm, such as the YOLO v5 algorithm.
[0070] The first solving module 303 is configured to solve for the target information of the target and the key feature information of the key features;
[0071] The first solving module 303 can solve for the center point, length, width and average length and width of the target based on the target information output by the target detection model, and can solve for the center point, length and width of the key features of the target based on the key feature information output by the target detection model.
[0072] The attitude mapping module 304 is configured to obtain the spatial three-dimensional attitude information of the target using a target space mapping model based on the target information and the key feature information.
[0073] For example, the present invention employs a target space mapping model based on expert knowledge. This attitude mapping model can be, for example, a mapping table used to represent the attitude mapping relationship of a target from a two-dimensional pattern to a three-dimensional attitude.
[0074] Having obtained the target information and the key feature information, the attitude mapping module 304 can obtain the spatial three-dimensional attitude information of the target using the target space mapping model.
[0075] The second solving module 305 is configured to solve for the depth information of the target and the key features relative to the UAV based on the scene depth information;
[0076] Based on the target and key feature area information output by the target detection model and the scene depth information obtained by the laser rangefinder 40, the second solving module 305 solves for the depth information of the target and key features relative to the UAV.
[0077] The positioning module 306 is configured to solve the spatial pose information of the target relative to the UAV based on the spatial three-dimensional attitude information and the depth information of the target and its key features relative to the UAV.
[0078] If the task is performed by the current UAV, the UAV performs the task based on the spatial pose information of the target relative to the UAV obtained by solving.
[0079] If the task is performed by other drones in the cluster, the positioning module 306 uses the spatial pose change matrix between the current drone and the task drone (i.e., the other drones) to convert the spatial pose information into the spatial pose information of the target relative to the task drone.
[0080] If the task is performed by a system other than a drone or drone swarm, the positioning module 306 uses the drone's own pose information (e.g., a transformation matrix relative to a ground station or other target) to convert the spatial pose information into the target's absolute pose information in three-dimensional space.
[0081] This invention utilizes a two-dimensional to three-dimensional attitude determination method incorporating prior expert knowledge, effectively reducing computational load and improving computational efficiency, thereby reducing the energy consumption of unmanned aerial vehicle (UAV) systems. Furthermore, it possesses significant potential in military reconnaissance, military strikes, and wildlife protection, as it can determine the spatial attitude of a target without requiring complete human or animal images. Since obtaining complete images of human or animal bodies is difficult in the wild, this invention is particularly suitable for complex outdoor environments.
[0082] The technical solution of the present invention will be further described below through two specific examples.
[0083] Example 1:
[0084] Taking apple harvesting by drone as an example, when building the target detection model, target image acquisition mainly refers to collecting images of apples in the orchard. These images should be of apples in their natural growth state. The collected images of apples in the orchard are then processed, removing low-resolution images and those with excessive occlusion. Finally, the target and its key features are labeled to construct a dataset. In this embodiment, the apple, fruit stem, and calyx are labeled.
[0085] The dataset was divided into training and testing sets, and the YOLO v5 object detection model was used for training to obtain a trained object detection model.
[0086] The target object's feature information is analyzed, including the apple's height, equatorial diameter, stem position, and calyx position. Based on this apple's size information, a spatial mapping model of the apple from a two-dimensional pattern to a three-dimensional pose is established, resulting in a pose mapping model based on expert knowledge. Since apples have relatively obvious spherical characteristics, this embodiment simplifies the apple into a sphere with these features.
[0087] During the drone mission execution phase, the drone's onboard camera acquires color images of the scene, and a laser rangefinder acquires scene depth information. The acquired color images are then used for target and key feature recognition. Expert knowledge is then combined to solve for the target's spatial attitude information. By combining the depth information of the target and its key features acquired by the drone, the target's spatial attitude information relative to the drone can be recovered. Finally, depending on the mission requirements and the specific method of mission execution, the target's spatial attitude information relative to the drone is converted, and the corresponding execution system completes the corresponding task.
[0088] When inputting color images into a target detection model for target and key feature recognition, scene images can be obtained by subscribing to data streams published by cameras via an onboard computer on the drone.
[0089] Figure 4 This is a schematic diagram of the predicted bounding box for a target in the YOLO model. After identifying the input image, the YOLO v5 target recognition model outputs the target and the regions containing key features, namely the regions containing the apple, the stem, and the calyx (sometimes one of the stem or calyx may be invisible due to the apple's posture). It then uses a similar method... Figure 4 The prediction box is shown in the figure.
[0090] like Figure 5 As shown, based on the identified apple, fruit stalk, or calyx region, a coordinate system x'o'y' centered on the apple is established. The coordinates of the apple's center in the original xoy coordinate system are:
[0091]
[0092]
[0093] The coordinates of the fruit stalk center in the original xoy coordinate system are:
[0094]
[0095]
[0096] The corresponding coordinates of the fruit stalk center in the x'o'y' coordinate system are:
[0097]
[0098]
[0099] The radius of the apple is:
[0100]
[0101] The spatial orientation information of the apple can be obtained by using the obtained information on the apple center, stem center, and apple radius.
[0102] Here, the xoy coordinate system can be referenced from the pixel coordinate system, or for ease of understanding, the bottom left corner can be set as the origin, the horizontal axis as the x-axis, and the vertical axis as the y-axis. The x'o'y' coordinate system takes the center of the apple as its origin, the horizontal axis as the x'-axis, and the vertical axis as the y'-axis.
[0103] By identifying the target and its key feature regions, along with the obtained depth information, the depth information of the apple, stem, or calyx relative to the drone is acquired. Finally, by combining the spatial pose information of the apple with the depth information of the apple, stem, or calyx relative to the drone, the spatial pose information of the apple relative to the drone can be calculated.
[0104] Example 2:
[0105] This embodiment uses a drone for human head recognition and positioning as an example.
[0106] When constructing the dataset, target image acquisition mainly refers to collecting human head images. The collected human head images should be as close as possible to the UAV mission environment. The collected human head images are then processed to remove low-resolution or excessively occluded images. In this embodiment, image annotation primarily focuses on features such as the head, eyes, nose, mouth, and ears.
[0107] The dataset is divided into training and testing sets, and a suitable object detection model (such as YOLO v5 mentioned above) is selected for training to obtain a well-trained object detection model.
[0108] Constructing a pose mapping model based on expert knowledge involves analyzing the target object's feature information and establishing a pose mapping model of the target object from a 2D pattern to a 3D pose. Analyzing the target object's feature information mainly refers to analyzing the size and center of the human head and the spatial relationships between the eyes, nose, mouth, ears, etc. Establishing a pose mapping model of the target object from a 2D pattern to a 3D pose mainly involves using the aforementioned size and position information of the human head, eyes, nose, mouth, ears, etc. Since the human head does not possess well-defined spherical features, this step requires a certain amount of research to summarize patterns, which can then be used to establish a spatial mapping model of the human head from a 2D pattern to a 3D pose.
[0109] During the drone mission execution phase, the drone's onboard camera acquires color images and depth information of the scene. The acquired color images are then used for target and key feature recognition. Expert knowledge is then used to solve for the target's spatial attitude information. Combined with the depth information of the target and its key features acquired by the drone, the target's spatial attitude information relative to the drone can be recovered. Finally, depending on the mission requirements and the specific method of mission execution, the target's spatial attitude information relative to the drone is converted, and the corresponding execution system completes the corresponding task.
[0110] When identifying targets and their key features, images of the scene can be obtained by subscribing to data streams published by cameras through the drone's onboard computer.
[0111] The YOLO v5 target recognition model identifies the face, eyes, nose, mouth, and ear regions of the input image.
[0112] like Figure 6 As shown, by combining the obtained information on the location of the face, eyes, nose, mouth and ears and the center point of each part with the expert knowledge of the human head model, the three-dimensional pose of the human head can be recovered from the two-dimensional image.
[0113] When solving for the depth information of the target and its key features, the depth information of various parts such as the face, eyes, nose, mouth, and ears relative to the UAV is obtained by using the identified target and the area where its key features are located and the obtained depth information. Finally, by combining the spatial pose information of the human head and the depth information of the face, eyes, nose, mouth, and ears relative to the UAV, the spatial pose information of the human head relative to the UAV can be obtained.
[0114] Although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Modifications or equivalent substitutions to the technical solutions of the embodiments of the present invention without departing from the inventive concept of the present invention should not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A target detection and localization method for an unmanned aerial vehicle (UAV) system, the UAV system comprising a UAV, an image acquisition unit for acquiring two-dimensional images of a scene, and a laser rangefinder for acquiring scene depth information, characterized in that, The method includes: Acquire a 2D scene image and scene depth information of the predetermined scene; The scene image is processed using a target detection model to identify the target and its key features, and the target and the region where its key features are located are output; wherein, the key features are one or more features of the target, and the feature is the one that best represents the characteristics of the target among all its features; The target information and key feature information of the target and its key features are solved based on the regional information of the target and its key features. The target information includes the center point, length, width, and average length and width of the target. The key feature information includes the center point, length, and width of the key feature. Based on the target information and the key feature information, the spatial three-dimensional attitude information of the target is obtained using an expert knowledge-based attitude mapping model; wherein, the attitude mapping model is a mapping table, which is used to represent the attitude mapping relationship of the target from a two-dimensional pattern to a three-dimensional attitude; The depth information of the target and its key features relative to the UAV is calculated based on the area information where the target and its key features are located and the scene depth information. The spatial pose information of the target relative to the UAV is calculated based on the target's three-dimensional spatial attitude information and the depth information of the target and its key features relative to the UAV.
2. The target detection and localization method according to claim 1, characterized in that, It also includes using a spatial pose change matrix between two drones to convert the spatial pose information into the spatial pose information of the target relative to one of the drones.
3. The target detection and localization method according to claim 1, characterized in that, It also includes using the drone's own pose information to convert the spatial pose information into the target's absolute pose information in three-dimensional space.
4. A target detection and positioning device for an unmanned aerial vehicle (UAV) system, the UAV system comprising a UAV, an image acquisition unit for acquiring two-dimensional images of a scene, and a laser rangefinder for acquiring scene depth information, characterized in that, The device includes: The data acquisition module is configured to acquire a two-dimensional scene image and scene depth information of a predetermined scene; The target detection module is configured to process the scene image using a target detection model to identify targets and key features of the targets, and output the target and the region where the key features are located; wherein, the key features are one or more features of the target, and the features are the features that best represent the characteristics of the target among all the features of the target; The first solution module is configured to solve for the target information and the key feature information of the key features based on the region information where the target and its key features are located; the target information includes the center point, length, width, and average length and width of the target; the key feature information includes the center point, length, and width of the key features; The attitude mapping module is configured to obtain the spatial three-dimensional attitude information of the target using an expert knowledge-based attitude mapping model based on the target information and the key feature information; wherein, the attitude mapping model is a mapping table used to represent the attitude mapping relationship of the target from a two-dimensional pattern to a three-dimensional attitude. The second solving module is configured to solve for the depth information of the target and the key features relative to the UAV based on the area information where the target and its key features are located and the scene depth information; The positioning module is configured to solve the spatial pose information of the target relative to the UAV based on the target's three-dimensional spatial pose information and the depth information of the target and its key features relative to the UAV.
5. The target detection and positioning device according to claim 4, characterized in that, The data acquisition module includes a two-dimensional image acquisition unit and a depth information acquisition unit, or a data interface connected to the two-dimensional image acquisition unit and the depth information acquisition unit.
6. The target detection and positioning device according to claim 4, characterized in that, The positioning module uses the spatial pose change matrix between the two UAVs to convert the spatial pose information into the spatial pose information of the target relative to one of the UAVs.
7. The target detection and positioning device according to claim 4, characterized in that, The positioning module uses the UAV's own pose information to convert the spatial pose information into the target's absolute pose information in three-dimensional space.
8. An unmanned aerial vehicle (UAV) system, characterized in that, It includes a drone body, an image acquisition unit for acquiring two-dimensional images of the scene, a depth information acquisition unit for acquiring scene depth information, and a target detection and positioning device according to any one of claims 4-7.