Target detection method, device, electronic device and storage medium
By using two object detection modules to process the pupil image on the smart fundus camera device, the calculation complexity and speed problems of existing object detection algorithms on low-performance devices are solved, and fast and accurate pupil positioning is achieved.
Patent Information
- Application Number
- CN202110814956.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-19
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-07-19
AI Technical Summary
Existing object detection algorithms, such as YOLO, Faster RCNN and SSD, have high computational complexity and are not suitable for embedded devices with low computing performance such as smart fundus cameras, and have slower detection speeds.
Two object detection modules are used to process the pupil image. The first module provides preliminary position information. The second module intercepts the pupil area image based on the preliminary position and outputs the final position information and object probability. It uses a simple convolutional neural network to construct it. Module, reduces computational complexity, and is suitable for low-performance equipment.
It realizes rapid positioning of the pupil position on devices with low computing performance, improving detection speed and applicability.
Smart Images

Figure CN113688675B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning, and specifically to target detection methods, devices, electronic devices and storage media. Background Art
[0002] When analyzing the user's eye health, it is often necessary to locate the user's pupil. Currently, object detection networks such as YOLO (You Only Look Once), Faster RCNN, and SSD are usually used to locate the user's pupil. However, such object detection algorithms have high computational complexity and are not suitable for use on embedded devices with low computational performance such as smart fundus cameras. At the same time, the detection speed is slow. Summary of the invention
[0003] The present application provides a target detection method, device, electronic device and storage medium.
[0004] According to a first aspect of an embodiment of the present application, a target detection method is provided, including:
[0005] Processing the target pupil image using a first target detection module to obtain a first result, the first result comprising: preliminary position information of the pupil to be located in the target pupil image, wherein the first target detection module is pre-trained by first training pupil images and annotated data of each first training pupil image;
[0006] Based on the preliminary position information of the pupil to be located, extracting a pupil area image from the target pupil image;
[0007] The pupil area image is processed by a second target detection module to obtain a second result, wherein the second result includes: final position information of the pupil to be located, and a probability that the object in the pupil area image is the pupil, wherein the second target detection module is pre-trained by training data corresponding to each second training pupil image, and the training data corresponding to the second training pupil image includes: training data corresponding to each rectangular box of the pupil in the second training pupil image, and the training data of the rectangular box includes: difference information of the rectangular box, and an image block surrounded by the rectangular box in the second training pupil image, wherein the difference information of the rectangular box indicates the difference between the rectangular box and the labeled box of the pupil in the second training pupil image.
[0008] According to a second aspect of an embodiment of the present application, there is provided a target detection device, including:
[0009] A first detection unit is configured to process the target pupil image using a first target detection module to obtain a first result, wherein the first result includes: preliminary position information of the pupil to be located in the target pupil image, wherein the first target detection module is pre-trained by a first training pupil image and annotated data of each first training pupil image;
[0010] A capture unit, configured to capture a pupil area image from the target pupil image based on the preliminary position information of the pupil to be located;
[0011] The second detection unit is configured to process the pupil area image using a second target detection module to obtain a second result, wherein the second result includes: final position information of the pupil to be located, and probability that the object in the pupil area image is the pupil, wherein the second target detection module is pre-trained by training data corresponding to each second training pupil image, and the training data corresponding to the second training pupil image includes: training data corresponding to each rectangular box of the pupil in the second training pupil image, and the training data of the rectangular box includes: difference information of the rectangular box, and an image block surrounded by the rectangular box in the second training pupil image, wherein the difference information of the rectangular box indicates the difference between the rectangular box and the labeled box of the pupil in the second training pupil image.
[0012] The target detection method, device, electronic device and storage medium provided in the embodiments of the present application use a first target detection module to process the target pupil image to obtain a first result, the first result including: preliminary position information of the pupil to be located in the target pupil image; based on the preliminary position information of the pupil to be located, extracting the pupil area image from the target pupil image; using a second target detection module to process the pupil area image to obtain a second result, the second result including: final position information of the pupil to be located, and the probability that the object in the pupil area image is the pupil. Both the first target detection module and the second target detection module can be modeled using a simple convolutional neural network, which has a lower computational complexity than using a more complex target detection algorithm such as YOLO, Faster RCNN and SSD for target detection, and is suitable for application on embedded devices with lower computational performance such as fundus cameras, and at the same time, has a faster detection speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0014] Figure 1 A flow chart of a target detection method provided by an embodiment of the present application is shown;
[0015] Figure 2 A schematic diagram showing the effect of a rectangular frame output by the first target detection module and a rectangular frame output by the second target detection module;
[0016] Figure 3 A schematic diagram showing the effect of the speed of performing target detection on pupil images using the target detection method provided in an embodiment of the present application;
[0017] Figure 4 The structure block diagram of the target detection device provided by the embodiment of the present application is shown;
[0018] Figure 5 A structural block diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0019] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0020] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0021] Figure 1 A flow chart of a target detection method provided in an embodiment of the present application is shown, and the method includes:
[0022] Step 101: Use a first target detection module to process a target pupil image to obtain a first result.
[0023] The pupil image can be collected by the smart fundus camera, for example, the auxiliary camera of the smart fundus camera collects any pupil image. One pupil image includes one eye of the photographed person, and one pupil image only includes one pupil, that is, the pupil in the one eye.
[0024] In the present application, the target pupil image does not specifically refer to a certain pupil image. For a pupil image, when the position of the pupil in the pupil image needs to be determined, the pupil image is used as the target pupil image.
[0025] In the present application, a pupil in the target pupil image may be referred to as a pupil to be located.
[0026] When the target pupil image is processed by the first target detection module to obtain the first result, the target pupil image is input into the first target detection module, and the first target detection module can output a preliminary predicted rectangular frame corresponding to the pupil to be located. The coordinates of the center point of the preliminary predicted rectangular frame can be used as the preliminary coordinates of the center point of the pupil to be located, the width of the preliminary predicted rectangular frame can be used as the preliminary width of the pupil to be located, and the height of the preliminary predicted rectangular frame can be used as the preliminary height of the pupil to be located. The preliminary position information of the pupil to be located may include: the preliminary coordinates of the center point of the pupil to be located, the preliminary width of the pupil to be located, and the preliminary height of the pupil to be located.
[0027] In the present application, the first target detection module may include: a branch for predicting the pupil position. The branch for predicting the pupil position may be a neural network such as a convolutional neural network.
[0028] After the target pupil image is input into the first target detection module, the target pupil image is used as the input of the branch for predicting the pupil position, and the branch for predicting the pupil position outputs preliminary position information of the pupil to be located.
[0029] In some embodiments, the first target detection module includes a convolutional neural network for predicting pupil position or a logistic regression model for predicting pupil position.
[0030] The branch for predicting the pupil position in the first target detection module may be a convolutional neural network for predicting the pupil position or a logistic regression model for predicting the pupil position.
[0031] In the present application, the pupil image used for training the first object detection module may be referred to as the first training pupil image. The first object detection module is pre-trained by the first training pupil image and the annotated data of each first training pupil image.
[0032] Each time the first target detection module is trained, a first training pupil image and the annotated data of the first training pupil image are used to train the first target detection module. Each time the first target detection module is trained, the first training pupil image used is different.
[0033] Each time the first target detection module is trained, a first training pupil image used in the training is input into the first target detection module, and the first target detection module outputs the predicted position information of the pupil in the first training pupil image. Specifically, the branch for predicting the pupil position in the first target detection module outputs the predicted position information of the pupil in the first training pupil image.
[0034] The predicted position information of the pupil in the first training pupil image may include: predicted coordinates of a center point of the pupil in the first training pupil image, predicted width of the pupil in the first training pupil image, and predicted height of the pupil in the first training pupil image.
[0035] In the present application, for each first training pupil image, the annotation information of the first training pupil image may include: annotation position information of the pupil in the first training pupil image.
[0036] For each first training pupil image, the annotated position information of the pupil in the first training pupil image may include: the annotated coordinates of the center point of the pupil in the first training pupil image, the annotated width of the pupil in the first training pupil image, and the annotated height of the pupil in the first training pupil image.
[0037] Each time the first target detection module is trained, the loss between the predicted position information of the pupil in the first training pupil image and the annotated position information of the pupil in the first training pupil image can be calculated, and the parameters of the branch for predicting the pupil position in the first target detection module can be updated based on the calculated loss.
[0038] In some embodiments, for each first training pupil image, each item in the annotated position information of the pupil in the first training pupil image is transformed to obtain a conversion value of each item; a label for training of the first training pupil image is generated, and the label for training of the first training pupil image includes the conversion value of each item.
[0039] For each first training pupil image, the label used for training of the first training pupil image is used to calculate the loss. When the first training pupil image is used to train the first target detection module, the loss between the predicted position information of the pupil in the first training pupil image and the label used for training of the first training pupil image is calculated.
[0040] The following formula can be used to transform each item of the annotated position information of the pupil in the first training pupil image:
[0041] d1=min((x+w*0.5) / img_w,1)
[0042] d2=min((y+h*0.5) / img_h,1)
[0043] d3=-log(w / img_w)
[0044] d4=-log(h / img_h)
[0045] d1 is the conversion value of the x-axis coordinate value of the predicted coordinates of the center point of the pupil in the first training pupil image, d2 is the conversion value of the y-axis coordinate value of the predicted coordinates of the center point of the pupil in the first training pupil image, d3 is the conversion value of the predicted width of the pupil in the first training pupil image, d4 is the conversion value of the predicted height of the pupil in the first training pupil image, x is the x-axis coordinate value of the predicted coordinates of the center point of the pupil in the first training pupil image, y is the y-axis coordinate value of the predicted coordinates of the center point of the pupil in the first training pupil image, w is the predicted width of the pupil in the first training pupil image, h is the predicted height of the pupil in the first training pupil image, img_w is the width of the first training pupil image, and img_h is the height of the first training pupil image.
[0046] Step 102: based on the preliminary position information of the pupil to be located, extract a pupil area image from the target pupil image.
[0047] In the present application, the pupil area to be located can be determined based on the preliminary position information of the pupil to be located. The preliminary position information of the pupil to be located may include: preliminary coordinates of the center point of the pupil to be located, preliminary width of the pupil to be located, and preliminary height of the pupil to be located. The shape of the pupil area to be located is a rectangle, the coordinates of the center point of the pupil area to be located are the preliminary coordinates, the width of the pupil area to be located is the preliminary width, and the height of the pupil area to be located is the preliminary height. After determining the pupil area to be located in the target pupil image, the pupil area image can be cut out from the target pupil image. In other words, the pupil area image can be an image block in the target pupil image, the area occupied by which is the pupil area to be located.
[0048] Step 103: Process the pupil area image using a second target detection module to obtain a second result.
[0049] In the present application, when the pupil area image is processed by the second target detection module to obtain the second result, the pupil area image is input into the second target detection module, and the second target detection module can output the final predicted rectangular frame corresponding to the pupil to be located and the probability that the object in the pupil area image is the pupil. The coordinates of the center point of the final predicted rectangular frame can be used as the final coordinates of the center point of the pupil to be located, the width of the final predicted rectangular frame can be used as the final width of the pupil to be located, and the height of the final predicted rectangular frame can be used as the final height of the pupil to be located. The second result includes: the final position information of the pupil to be located, and the probability that the object in the pupil area image is the pupil. The final position information of the pupil to be located includes: the final coordinates of the center point of the pupil to be located, the final width of the pupil to be located, and the final height of the pupil to be located.
[0050] Please refer to Figure 2, which shows a schematic diagram of the effect of the rectangular box output by the first target detection module and the rectangular box output by the second target detection module.
[0051] exist Figure 2 In the figure, the dotted rectangular box is the rectangular box output by the first target detection module, that is, the preliminary predicted rectangular box corresponding to the pupil to be located, and the solid rectangular box is the rectangular box output by the second target detection module, that is, the final predicted rectangular box corresponding to the pupil to be located.
[0052] Please refer to Figure 3 , which is a schematic diagram showing the effect of the speed of target detection on pupil images using the target detection method provided in an embodiment of the present application.
[0053] The processor for running the target detection method provided in the embodiment of the present application can be an ARM processor such as an A7@1.2GHz ARM processor.
[0054] exist Figure 3 , the detection speed of performing target detection on each of 10 pupil images is exemplarily shown. The size of each pupil image may be 800×600, and the average detection speed is about 19 ms.
[0055] In the present application, the second target detection module may include: a regression branch for predicting the pupil position, and a classification branch for predicting the probability that the object in the pupil area image is the pupil. The regression branch for predicting the pupil position may be a neural network such as a convolutional neural network, and the classification branch for predicting the probability that the object in the pupil area image is the pupil may be a neural network such as a convolutional neural network.
[0056] After the pupil area image is input into the second object detection module, the pupil area image is used as the input of the regression branch in the second object detection module, and the regression branch outputs the final position information of the pupil to be located. At the same time, the pupil area image is used as the input of the classification branch in the second object detection module, and the classification branch outputs the probability that the object in the pupil area image is the pupil.
[0057] In some embodiments, the second target detection module includes a convolutional neural network for predicting pupil position or a logistic regression model for predicting pupil position.
[0058] The regression branch for predicting the pupil position in the second target detection module may be a convolutional neural network for predicting the pupil position or a logistic regression model for predicting the pupil position.
[0059] In some embodiments, using the second target detection module to process the pupil area image to obtain the second result includes: adjusting the size of the pupil area image to a preset size; and using the second target detection module to process the pupil area image to obtain the second result.
[0060] For example, the preset size is 48×48, the size of the pupil area image is adjusted to 48×48, and after the size of the pupil area image is adjusted to 48×48, the pupil area image is input into the second target detection module, and the second target detection module outputs a second result.
[0061] In the present application, the second object detection module is pre-trained by the training data corresponding to each second training pupil image.
[0062] During the training of the second object detection model, the training data corresponding to each second training pupil image can be used to train the regression branch in the second object detection module and the classification branch in the second object detection module.
[0063] In the present application, the pupil image used to train the second object detection module may be referred to as a second training pupil image.
[0064] For each second training pupil image, the training data corresponding to the second training pupil image includes: training data corresponding to each rectangular box of the pupil in the second training pupil image, and the training data corresponding to the rectangular box of the pupil in the second training pupil image includes: difference information of the rectangular box, an image block in the second training pupil image surrounded by the rectangular box, and annotation data of the second training pupil image, wherein the difference information of the rectangular box indicates the difference between the rectangular box and the annotation box of the pupil in the second training pupil image.
[0065] For each second training pupil image, the annotation information of the second training pupil image includes: annotation position information of the pupil in the second training pupil image.
[0066] For each second training pupil image, the annotated position information of the pupil in the second training pupil image may include: the annotated coordinates of the center point of the pupil in the second training pupil image, the annotated width of the pupil in the second training pupil image, and the annotated height of the pupil in the second training pupil image.
[0067] In the present application, for each second training pupil image, a preset number of rectangular frames different from the annotation frame of the pupil in the second training pupil image can be generated around the annotation frame of the pupil in the second training pupil image.
[0068] In the present application, for each second training pupil image, a rectangular box generated around the annotation box of the pupil in the second training pupil image, which is different from the annotation box of the pupil in the second training pupil image, is a rectangular box corresponding to the pupil in the second training pupil image.
[0069] For each second training pupil image, the shape of the pupil annotation box in the second training pupil image is a rectangle, the coordinates of the center point of the pupil annotation box in the second training pupil image are the annotated coordinates of the center point of the pupil in the second training pupil image, the width of the pupil annotation box in the second training pupil image is the annotated width of the pupil in the second training pupil image, and the height of the pupil annotation box in the second training pupil image is the annotated height of the pupil in the second training pupil image.
[0070] When training the second target detection module, for a rectangular box corresponding to a second training pupil image, the image block in the training data of the rectangular box is used as the input of the second target detection module, and the difference information in the training data of the rectangular box can be used as the supervision information to calculate the regression loss. If the intersection-over-union ratio (IOU) between the rectangular box and the labeled box of the pupil in the second training pupil image is greater than a preset threshold related to classification, the rectangular box can be used as a positive sample for training the classification branch in the second target detection module, and if the intersection-over-union ratio between the rectangular box and the labeled box of the pupil in the second training pupil image is less than a preset threshold related to classification, the rectangular box can be used as a negative sample for training the classification branch in the second target detection module.
[0071] In some embodiments, for a second training pupil image, a rectangular box whose corresponding intersection-and-union ratio in each rectangular box corresponding to the pupil in the second training pupil image is greater than a first intersection-and-union ratio threshold is used to train a regression branch in the second target detection module and serves as a positive sample for training a classification branch in the second target detection module, a rectangular box whose corresponding intersection-and-union ratio in each rectangular box is less than a first intersection-and-union ratio threshold and greater than a second intersection-and-union ratio threshold is used to train the regression branch, and a rectangular box whose corresponding intersection-and-union ratio in each rectangular box is less than a third intersection-and-union ratio threshold is used as a negative sample for training the classification branch, wherein the intersection-and-union ratio corresponding to the rectangular box is the intersection-and-union ratio of the rectangular box and the labeled box of the pupil in the second training pupil image, the first intersection-and-union ratio threshold is greater than the second intersection-and-union ratio threshold, and the second intersection-and-union ratio threshold is greater than the third intersection-and-union ratio threshold.
[0072] For each rectangular box corresponding to the pupil in a second training pupil image, the intersection-and-union ratio of the rectangular box and the labeled box of the pupil in the second training pupil image can be calculated to obtain the intersection-and-union ratio corresponding to the rectangular box. According to the intersection-and-union ratio corresponding to the rectangular box corresponding to the pupil in the second training pupil image, it is determined whether the rectangular box corresponding to the pupil in the second training pupil image is used to train the regression branch and the classification branch.
[0073] For example, the first intersection-and-union threshold is 0.65, the second intersection-and-union threshold is 0.5, and the third intersection-and-union threshold is 0.3. For a rectangular box corresponding to a pupil in a second training pupil image, if the corresponding intersection-and-union of the rectangular box is greater than 0.65, the rectangular box is used to train the regression branch and the rectangular box is used as a positive sample for training the classification branch. If the corresponding intersection-and-union of the rectangular box is less than 0.65 and greater than 0.5, the rectangular box is used to train the regression branch. If the corresponding intersection-and-union of the rectangular box is less than 0.3, the rectangular box is used as a negative sample for training the classification branch.
[0074] When training the second object detection module, for a rectangular box corresponding to a second training pupil image, the image block in the training data of the rectangular box is used as the input of the second object detection module. If the rectangular box is used to train the regression branch, the difference information in the training data of the rectangular box can be used as the supervision information to calculate the regression loss.
[0075] Please refer to Figure 4 , which shows a structural block diagram of a target detection device provided in an embodiment of the present application. The target detection device includes: a first detection unit 401, a capture unit 402, and a second detection unit 403.
[0076] The first detection unit 401 is configured to process the target pupil image using a first target detection module to obtain a first result, wherein the first result includes: preliminary position information of the pupil to be located in the target pupil image, wherein the first target detection module is pre-trained by the first training pupil image and the annotated data of each first training pupil image;
[0077] The interception unit 402 is configured to intercept a pupil area image from the target pupil image based on the preliminary position information of the pupil to be located;
[0078] The second detection unit 403 is configured to process the pupil area image using a second target detection module to obtain a second result, wherein the second result includes: final position information of the pupil to be located, and probability that the object in the pupil area image is the pupil, wherein the second target detection module is pre-trained with training data corresponding to each second training pupil image, and the training data corresponding to the second training pupil image includes: training data corresponding to each rectangular box of the pupil in the second training pupil image, and the training data of the rectangular box includes: difference information of the rectangular box, and an image block surrounded by the rectangular box in the second training pupil image, wherein the difference information of the rectangular box indicates the difference between the rectangular box and the labeled box of the pupil in the second training pupil image.
[0079] In some embodiments, the rectangular boxes in each of the rectangular boxes whose corresponding IoU ratio is greater than the first IoU ratio threshold are used to train the regression branch in the second target detection module and serve as positive samples for training the classification branch in the second target detection module, the rectangular boxes in each of the rectangular boxes whose corresponding IoU ratio is less than the first IoU ratio threshold and greater than the second IoU ratio threshold are used to train the regression branch, and the rectangular boxes in each of the rectangular boxes whose corresponding IoU ratio is less than the third IoU ratio threshold are used as negative samples for training the classification branch, wherein the IoU ratio corresponding to the rectangular box is the IoU ratio of the rectangular box and the annotation box, the first IoU ratio threshold is greater than the second IoU ratio threshold, and the second IoU ratio threshold is greater than the third IoU threshold.
[0080] In some embodiments, the annotation information of the first training pupil image includes: annotated position information of the pupil in the first training pupil image; the apparatus further includes:
[0081] The label generation unit is configured to transform each item of the annotated position information of the pupil in the first training pupil image to obtain a transformation value of each item; generate a label for training of the first training pupil image, and the label for training of the first training pupil image includes the transformation value of each item.
[0082] In some embodiments, the first target detection module includes a convolutional neural network for predicting pupil position or a logistic regression model for predicting pupil position, and the second target detection module includes a convolutional neural network for predicting pupil position or a logistic regression model for predicting pupil position.
[0083] In some embodiments, the second detection unit 403 is further configured to adjust the size of the pupil area image to a preset size; and use a second object detection module to process the pupil area image to obtain a second result.
[0084] Figure 5 5 is a block diagram of a structure of an electronic device provided by this embodiment. The electronic device includes a processing component 522, which further includes one or more processors, and a memory resource represented by a memory 532, which is used to store instructions that can be executed by the processing component 522, such as an application. The application stored in the memory 532 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 522 is configured to execute instructions to perform the above method.
[0085] The electronic device may also include a power supply component 526 configured to perform power management of the electronic device, a wired or wireless network interface 550 configured to connect the electronic device to a network, and an input / output (I / O) interface 558. The electronic device may operate based on an operating system stored in the memory 532, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™ or the like.
[0086] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory including instructions, and the instructions can be executed by an electronic device to complete the above-mentioned target detection method. Optionally, the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.
[0087] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0088] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A target detection method, characterized in that: The method comprises: Processing the target pupil image using a first target detection module to obtain a first result, the first result including: preliminary position information of the pupil to be located in the target pupil image, wherein the first target detection module is pre-trained by the first training pupil image and the annotation data of each first training pupil image, the preliminary position information of the pupil to be located includes: preliminary coordinates of the center point of the pupil to be located, preliminary width of the pupil to be located, and preliminary height of the pupil to be located, and the first target detection module includes a convolutional neural network for predicting the pupil position or a logistic regression model for predicting the pupil position; Based on the preliminary position information of the pupil to be located, extracting a pupil area image from the target pupil image; The pupil area image is processed by a second target detection module to obtain a second result, wherein the second result includes: final position information of the pupil to be located, and probability that the object in the pupil area image is the pupil, wherein the second target detection module is pre-trained by training data corresponding to each second training pupil image, and the training data corresponding to the second training pupil image includes: training data corresponding to each rectangular box of the pupil in the second training pupil image, and the training data of the rectangular box includes: difference information of the rectangular box, and an image block surrounded by the rectangular box in the second training pupil image, wherein the difference information of the rectangular box indicates the difference between the rectangular box and the labeled box of the pupil in the second training pupil image, and the second target detection module includes a convolutional neural network for predicting the pupil position or a logistic regression model for predicting the pupil position.
2. The method according to claim 1, characterized in that The rectangular boxes whose corresponding intersection-and-union ratios in each rectangular box are greater than the first intersection-and-union ratio threshold are used to train the regression branch in the second target detection module and serve as positive samples for training the classification branch in the second target detection module. The rectangular boxes whose corresponding intersection-and-union ratios in each rectangular box are less than the first intersection-and-union ratio threshold and greater than the second intersection-and-union ratio threshold are used to train the regression branch. The rectangular boxes whose corresponding intersection-and-union ratios in each rectangular box are less than the third intersection-and-union ratio threshold are used as negative samples for training the classification branch, wherein the intersection-and-union ratios corresponding to the rectangular boxes are the intersection-and-union ratios of the rectangular boxes and the labeled boxes, the first intersection-and-union ratio threshold is greater than the second intersection-and-union ratio threshold, and the second intersection-and-union ratio threshold is greater than the third intersection-and-union ratio threshold.
3. The method according to claim 1, characterized in that The annotation information of the first training pupil image includes: the annotation position information of the pupil in the first training pupil image; the method further includes: Convert each item of the annotated position information of the pupil in the first training pupil image to obtain a conversion value of each item; A label for training of the first training pupil image is generated, wherein the label for training of the first training pupil image includes the conversion value of each item.
4. The method according to claim 1, characterized in that: The pupil area image is processed by a second target detection module to obtain a second result including: Adjusting the size of the pupil area image to a preset size; The pupil area image is processed using a second object detection module to obtain a second result.
5. A target detection device, characterized in that: The device comprises: A first detection unit is configured to process the target pupil image using a first target detection module to obtain a first result, wherein the first result includes: preliminary position information of the pupil to be located in the target pupil image, wherein the first target detection module is pre-trained by the first training pupil image and the annotation data of each first training pupil image, and the preliminary position information of the pupil to be located includes: preliminary coordinates of the center point of the pupil to be located, preliminary width of the pupil to be located, and preliminary height of the pupil to be located, and the first target detection module includes a convolutional neural network for predicting the pupil position or a logistic regression model for predicting the pupil position; A capture unit, configured to capture a pupil area image from the target pupil image based on the preliminary position information of the pupil to be located; The second detection unit is configured to process the pupil area image using a second target detection module to obtain a second result, wherein the second result includes: final position information of the pupil to be located, and probability that the object in the pupil area image is the pupil, wherein the second target detection module is pre-trained by training data corresponding to each second training pupil image, and the training data corresponding to the second training pupil image includes: training data corresponding to each rectangular box of the pupil in the second training pupil image, and the training data of the rectangular box includes: difference information of the rectangular box, and an image block surrounded by the rectangular box in the second training pupil image, wherein the difference information of the rectangular box indicates the difference between the rectangular box and the labeled box of the pupil in the second training pupil image, and the second target detection module includes a convolutional neural network for predicting the pupil position or a logistic regression model for predicting the pupil position.
6. The device according to claim 5, characterized in that The rectangular boxes in each rectangular box whose corresponding intersection-and-union ratio is greater than the first intersection-and-union ratio threshold are used to train the regression branch and serve as positive samples for training the classification branch in the second target detection module. The rectangular boxes in each rectangular box whose corresponding intersection-and-union ratio is less than the first intersection-and-union ratio threshold and greater than the second intersection-and-union ratio threshold are used to train the regression branch in the second target detection module. The rectangular boxes in each rectangular box whose corresponding intersection-and-union ratio is less than the third intersection-and-union ratio threshold are used as negative samples for training the classification branch, wherein the intersection-and-union ratio corresponding to the rectangular box is the intersection-and-union ratio of the rectangular box and the annotation box, the first intersection-and-union ratio threshold is greater than the second intersection-and-union ratio threshold, and the second intersection-and-union ratio threshold is greater than the third intersection-and-union ratio threshold.
7. The device according to claim 5, characterized in that The annotation information of the first training pupil image includes: the annotation position information of the pupil in the first training pupil image; the device further includes: The label generation unit is configured to transform each item of the annotated position information of the pupil in the first training pupil image to obtain a transformation value of each item; generate a label for training of the first training pupil image, and the label for training of the first training pupil image includes the transformation value of each item.
8. An electronic device comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 4. 9 . A storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as claimed in any one of claims 1 to 4.
