A method and device for fast labeling of autonomous driving scene images

Abstract

The present invention belongs to the field of autonomous driving, and specifically relates to a method for rapid annotation of autonomous driving scene images, comprising the following steps: S1: displaying the image to be annotated on the screen; S2: the camera obtains the facial information of the operator; S3: the data processing module identifies the eye direction of the operator through the facial information, and calculates the position on the screen where the operator is looking based on the eye direction; if the time facing a position exceeds a threshold, the area at the position is confirmed as the initial candidate area, and the geometric center of the initial candidate area is selected as the pre-selected point; S4: the data processing module obtains the coordinates of the pre-selected point, and moves the mouse cursor to the pre-selected point position; S5: identifying the pixel points in the initial candidate area, finding the boundary point between the vehicle and the environment through grayscale recognition, highlighting it, and generating it as a recommended point... It also includes a device applicable to the above method, which can reduce the operation process of the annotator.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and specifically relates to a method and device for quickly labeling autonomous driving scene images. Background Art

[0002] With the rapid development of autonomous driving technology, there is a growing demand for efficient and accurate image annotation methods. Autonomous driving systems need to process a large amount of visual data, including roads, vehicles, pedestrians, traffic signs, etc., to ensure safe and reliable driving. Image annotation, as a key step in training autonomous driving systems, directly affects the performance of the system. When selecting and annotating images, if the bounding box range is too large, it will cause: Information confusion: If the bounding box is too loose, it may include other objects or environments around the vehicle, such as pedestrians, trees, traffic signs, etc., which will cause information confusion and affect the model's accurate recognition of the vehicle. Data noise: An overly large bounding box may introduce additional data noise, requiring the model to process more irrelevant features during the learning process, thereby affecting learning efficiency and accuracy. Slow system response: In practical applications, an overly large bounding box may cause the autonomous driving system to locate and track the vehicle inaccurately, thereby affecting the system's response speed and decision-making ability. If the bounding box range is too small: Missing important features: If the bounding box is too small, part of the vehicle may be missed, especially the edge of the vehicle, which will cause the model to be unable to obtain the complete features of the vehicle and affect the accuracy of recognition. Weak model generalization ability: If the bounding boxes in the training data are generally small, the model may not be able to generalize well when facing complete vehicles in actual scenes, and may not be able to effectively identify them. Unstable tracking: In applications such as vehicle tracking, a too small bounding box may make it difficult for the tracking algorithm to stably lock the target, affecting the continuity and reliability of tracking.

[0003] The manual work of selecting the cars in the lane images is often referred to as "image annotation" or "manual image annotation". In the research and development of autonomous driving and computer vision, this is a common data preparation process used to provide training data for machine learning models. In this process, the annotator uses professional image annotation tools to draw bounding boxes around the vehicles in the image as required. These bounding boxes are usually presented as rectangles or polygons, accurately identifying the location and size of the vehicle. The annotation results will be used to train and verify the visual recognition algorithm of the autonomous driving system, enabling it to recognize and understand vehicles in actual driving scenarios. Manual image annotation is a labor-intensive task that requires a high degree of precision and concentration. Despite the existence of automated annotation technology, manual annotation is still necessary in some complex scenarios or special cases because it can provide more accurate and reliable annotation results. In addition, manual annotation can also be used to verify and improve the effectiveness of automated annotation technology.

[0004] When manual image annotation is used, each vehicle needs to be framed on the image and the vehicle type needs to be marked. The operator selects the frame on the image by clicking the mouse. Taking rectangular frame selection as an example, it is usually necessary to select an initial point in the upper left corner of the vehicle to be framed, that is, the first step is to drag the mouse to the position to be selected, and then click to confirm as the initial point; then drag the mouse to the lower right corner of the vehicle to be framed to select an end point, click the mouse to confirm the end point, and the annotation system or program uses the line connecting the initial point and the end point as the diagonal line to generate a rectangular frame to frame the vehicle to be framed. The initial point and the end point are selected by the operator after observing the image, that is, the position of the initial point and the end point is identified by the naked eye, and then the initial point is selected by moving the mouse to the initial point position, clicking to confirm the selection of the initial point, and then dragging the mouse to the end point position, clicking to confirm the selection of the end point, and completing the frame. At least 4 steps are required.

[0005] Too many repeated operations will cause fatigue in the operator's hands, and in severe cases, it will cause injuries. Long-term fatigue operations will also affect the accuracy of labeling. Therefore, a technology is needed to reduce the operating procedures of labelers. Summary of the invention

[0006] The purpose of this solution is to provide a method and device for rapid annotation of autonomous driving scene images, which can reduce the operation process of annotators.

[0007] In order to achieve the above objectives, this solution provides a fast annotation method for autonomous driving scene images, comprising the following steps:

[0008] S1: Display the image to be annotated on the screen;

[0009] S2: The camera obtains the operator’s facial information;

[0010] S3: The data processing module identifies the operator's eye direction through facial information, and calculates the position on the screen that the operator is looking at based on the eye direction. If the time spent looking at a position exceeds a threshold, the area at that position is identified as the initial candidate area, and the geometric center of the initial candidate area is selected as the pre-selected point.

[0011] S4: The data processing module obtains the coordinates of the pre-selected point and moves the mouse cursor to the pre-selected point;

[0012] S5: Identify the pixels in the initial candidate area, find the boundary between the vehicle and the environment through grayscale recognition, highlight it, and generate several recommended points;

[0013] S6: The data processing module generates auxiliary vertical lines in the vertical direction and the horizontal direction on the mouse cursor at the pre-selected point position;

[0014] S7: When the data processing module detects that the mouse cursor moves toward the recommended point, the data processing module selects a threshold number of pixel points on both sides of the auxiliary vertical line as the expansion area, selects a threshold number of pixel points on the left and right sides of the auxiliary vertical line in the vertical direction and pixel points on the upper and lower sides of the auxiliary vertical line in the horizontal direction as the expansion area, and the data processing module selects the image in the expansion area; if the distance between the mouse cursor and each recommended point is shortening, the image in the expansion area is enlarged by a threshold multiple; if the distance between the mouse cursor and each recommended point is increasing, the image in the expansion area is reduced by a threshold multiple; the operator identifies the boundary point between the vehicle to be selected and the environment according to the enlarged or reduced image of the expansion area, and moves the auxiliary vertical line in the vertical direction and the auxiliary vertical line in the horizontal direction to the boundary point of the corresponding area by moving the mouse;

[0015] When the mouse cursor moves away from the recommended point, the image in the direction of the auxiliary vertical line will be zoomed out;

[0016] S8: Click the mouse to confirm the initial point;

[0017] S9: Move the mouse to the end point, click to complete the selection of the end point, and the data processing module generates a labeling box according to the coordinate positions of the initial point and the end point;

[0018] S10: Repeat steps S2-S9 to mark other vehicles in the image.

[0019] The basic principle and beneficial effects of this solution: This solution uses the assistance of a camera to identify the eye direction of the operator (i.e., the annotator), confirm the area where the eye is directed as the initial candidate area, and determine the center position in the initial candidate area as the pre-selected point. The data processing module moves the mouse to the pre-selected point position, and then identifies the image of the initial candidate area, finds the boundary point between the vehicle and the environment, and sets it as a recommended point. The operator moves the mouse cursor at the pre-selected point by dragging the mouse. At this time, the data processing module also generates auxiliary vertical lines in the vertical and horizontal directions at the mouse cursor. When the mouse moves toward the recommended point, the data processing module uses the pixel points within the thresholds on the left and right sides of the auxiliary vertical line in the vertical direction and the pixel points within the thresholds on the upper and lower sides of the auxiliary vertical line in the horizontal direction as the expansion area, that is, the image within the threshold interval on both sides of the auxiliary vertical line is selected as the expansion area for further processing. Through corresponding movement, the image in the expansion area is enlarged or reduced. If the distance between the mouse cursor and each recommended point is shortened, the image in the extension direction of the auxiliary vertical line is enlarged, otherwise it is reduced. The operator then clicks the mouse at the appropriate location to determine the initial point, and finally moves the mouse to select the end point. After completing the selection of the initial point and the end point, a labeling box is generated based on the two points. Then the vehicle category selected in the box is labeled.

[0020] The confirmation of the initial point of this solution is achieved by moving the mouse by the data processing module according to the direction of the eyeball, which reduces the steps of manual dragging, and through the combination of auxiliary vertical lines and recommended points, the operator can accurately select the appropriate initial point by moving the mouse a little. Compared with the traditional marking method, this solution has a shorter dragging distance for the selection of the initial position (the mouse cursor is directly moved by the data processing module), that is, in the prior art, it is necessary to first drag the mouse to the initial point (determined by the human eye), then click the mouse to confirm the selection of the initial point, then drag the mouse to the end point, and then confirm the selection of the end point, and complete the frame according to the initial point and the end point; this solution first optimizes the dragging of the mouse to the initial point The distance is changed from being dragged by the operator to having a camera capture the operator's eye direction, so that the data processing module automatically moves the mouse cursor to the position of the pre-selected point, thereby reducing the dragging distance of the mouse and reducing the situation where the operator cannot find the mouse (in high-intensity frame work, due to a short period of distraction, the mouse may not be known where it has moved. At this time, the mouse needs to be moved significantly to keep the mouse cursor moving on the screen to increase exposure, so that it can be discovered by the operator and continue the next operation. However, this process will reduce the operator's experience and seriously make the operator impatient, affecting physical health), improving the operator's operating experience. In this solution, the confirmation of the initial point is optimized. Through the automatic identification of the recommended point, the image near the recommended point is enlarged / reduced according to the direction in which the operator moves the mouse. When approaching the recommended point, the image is enlarged, and vice versa; when the image is enlarged, the operator can more clearly observe the boundary point between the vehicle to be framed and the environment, so that the operator can select a suitable initial point in the enlarged image. That is, this solution achieves the effect of improving the operator's annotation experience, and the drawing of the annotation box is also more accurate.

[0021] Further, in step S7, the data processing module is also used to obtain the horizontal and vertical coordinates of each recommended point; calculate the horizontal coordinate difference and vertical coordinate difference between the pre-selected point and each recommended point, if any one of the horizontal coordinate difference and the vertical coordinate difference decreases when the pre-point moves, it is determined that the pre-selected point is approaching the recommended point, and the image of the expanded area is enlarged; if both differences increase, it is determined that the pre-selected point is moving away from the recommended point, and the image of the expanded area is reduced.

[0022] Beneficial effects: This solution calculates the horizontal and vertical distances between the pre-selected points and the recommended points. If any of the distances is shortened, it is judged to be approaching. If both distances are increased, it is judged to be moving away, thereby making it easier to control the zooming in and out of the image of the expansion area; it is convenient for operators to choose. In the prior art, there are technologies for zooming in or out of images when moving toward a single point, but for multiple points, especially when some moving distances are close and some are far away, there are internal conflicts, which may lead to inaccurate recognition. This solution uses the single value of the horizontal and vertical coordinates to make judgments, redefines the judgment criteria for approaching and moving away, and no longer uses the distance between two points (pre-selected points and recommended points) as the judgment, but uses the distance between the separate horizontal and vertical coordinates for judgment, and the results are more accurate. And when selecting and marking the vehicle, there will be no judgment conflicts.

[0023] Further, in step S7, when the operator operates the mouse to move from the pre-selected point to the recommended point, the initial moving direction is confirmed by the coordinates after the movement and the pre-selected point. If the distance between the mouse cursor and each recommended point is shortened, the image in the expansion area is magnified by a threshold multiple. The data processing module detects that the movement of the mouse cursor is opposite to the initial moving direction. If the speed of the reverse movement does not exceed the preset value, the enlarged image of the expansion area is magnified twice. If the reverse movement speed exceeds the preset value, the image of the expansion area is reduced by half.

[0024] Beneficial effect: When the operator moves the mouse, he may accidentally move the auxiliary vertical line beyond the intersection of the vehicle and the environment. At this time, he will move the mouse back, and the data processing module will identify the withdrawal process. If the speed is within the preset value, the image of the expansion area will be enlarged again to facilitate the operator's selection. If the speed exceeds the preset value, it is possible that the operator slipped, resulting in a far offset position. At this time, by reducing the expansion area, the mouse can be quickly returned to the vicinity of the dividing point. If the data processing module detects that the speed is about to reach the preset value, the image of the expansion area will be enlarged to the original multiple for easy viewing.

[0025] Furthermore, in step S9, it is also necessary to mark the type of the vehicle selected in the marking box, and the types of vehicles at least include cars, trucks, motorcycles and special vehicles.

[0026] Furthermore, it also includes left and right movement keys and up and down movement keys, which are used by the operator to move the mouse cursor in the left and right directions and up and down directions in the manual adjustment step S7, respectively, and each movement distance is one pixel.

[0027] Beneficial effect: The operator can complete the vehicle selection by controlling the buttons with the left hand and the mouse with the right hand, which improves efficiency.

[0028] Furthermore, when a single press of the left and right movement keys and the up and down movement keys exceeds a threshold, a long press signal is generated, and the data processing module controls the movement speed of the mouse cursor to double.

[0029] Furthermore, the vehicle type generates corresponding selection items which are displayed on the screen, and the selection items are scattered on the left and right sides and the upper and lower sides of the screen.

[0030] Furthermore, the selection items for cars are located on the left side of the screen, the selection items for trucks are located on the right side of the screen, the selection items for motorcycles are located on the upper side of the screen, and the selection items for special vehicles are located on the lower side of the screen; after the data processing module finishes generating a rectangular frame based on the first selection point and the second selection point, the camera captures the direction of the operator's eyes again, and determines the labeled type of the selected vehicle based on the selection items corresponding to the type of vehicle the operator is looking at.

[0031] Furthermore, in step S10, when selecting the next vehicle to be framed, the data processing module divides the photo into two equal parts, left and right. If a number of consecutive annotated photos exceeding a threshold are all from the same part, the data processing module highlights and flashes another part of the photo to prompt the operator to annotate in that area.

[0032] Beneficial effect: Since the nature of the operator's job is to sit in front of the display screen and operate the pictures on the screen through the mouse and keyboard (buttons), this work habit often requires the head to face one position for a long time, which will cause neck strain in the long run. Therefore, through the setting of this solution, when the operator faces an area for more than a threshold, the data processing module will move the cursor to another area in a flashing manner, thereby driving the operator's head to turn, causing the neck to twist, and preventing strain.

[0033] A rapid annotation device for autonomous driving scene pictures, comprising a display screen, a camera, a mouse and a data processing module, wherein the camera is installed above the display screen and is used to capture facial information of an operator, the display screen is used to display pictures to be annotated, and after receiving a signal from the operator clicking or moving the mouse, the data processing module controls the corresponding movement of the mouse cursor in the display screen, and the data processing module is used to process the facial information;

[0034] The data processing module identifies the operator's eye direction through facial information, and calculates the position on the screen that the operator is looking at based on the eye direction. If the time spent looking at a position exceeds a threshold, the area at the position is confirmed as the initial candidate area, and the geometric center of the initial candidate area is selected as the pre-selected point. The data processing module generates auxiliary vertical lines in the vertical and horizontal directions on the mouse cursor at the pre-selected point.

[0035] The data processing module obtains the coordinates of the pre-selected point and moves the mouse cursor to the pre-selected point. The data processing module is also used to identify the pixel points in the initial selected area, find the boundary point between the vehicle and the environment through grayscale recognition, highlight it, and generate it as a recommended point.

[0036] When the operator drags the mouse to move toward or away from the recommended point, the image in the direction in which the auxiliary vertical line extends will be enlarged or reduced accordingly;

[0037] The mouse is also used to confirm the end point on the picture according to the operation instruction of the operator;

[0038] The data processing module generates a rectangular frame based on the line connecting the initial point and the end point as the diagonal line, and the rectangular frame selects the vehicle to be selected; after the selection is completed, the data processing module recognizes the image information of the selected vehicle and marks the corresponding vehicle type.

[0039] Furthermore, the data processing module divides the photo to be annotated into two parts, left and right. If the number of annotated photos exceeding a threshold number are all from the same part, the data processing module highlights and flashes the other part of the photo to prompt the operator to annotate in this area. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the steps of an embodiment of the present invention.

[0041] Figure 2 This is a schematic diagram of the expansion area after the pre-selected point is moved.

[0042] Figure numerals in the specification: pre-selected point 1, auxiliary vertical line 12 in the vertical direction, auxiliary vertical line 11 in the horizontal direction, expansion area 3, recommended point 4, vehicle to be selected 4.

[0043] The following is a further detailed description through specific implementation methods: DETAILED DESCRIPTION

[0044] The embodiment is basically as shown in the attached Figure 1 As shown:

[0045] A fast annotation method for autonomous driving scene images, comprising the following steps:

[0046] S1: Display the image to be annotated on the screen;

[0047] S2: The camera obtains the operator’s facial information;

[0048] S3: The data processing module identifies the operator's eye direction through facial information, and calculates the position on the screen that the operator is looking at based on the eye direction. If the time spent looking at a position exceeds a threshold, the area at that position is identified as the initial candidate area, and the geometric center of the initial candidate area is selected as the pre-selected point 1.

[0049] S4: The data processing module obtains the coordinates of the pre-selected point 1 and moves the mouse cursor to the position of the pre-selected point 1;

[0050] S5: Identify the pixels in the initial candidate area, find the boundary between the vehicle and the environment through grayscale recognition, highlight it, and generate 4 recommended points;

[0051] S6: The data processing module generates auxiliary vertical lines 11 in the vertical and horizontal directions on the mouse cursor at the position of the pre-selected point 1; the distance of the auxiliary vertical lines can be set and modified by the operator, for example, 2 cm in the horizontal direction and 3 cm in the vertical direction. That is, the auxiliary vertical line is compared with the geometric midline point of the initial selected area, and an auxiliary vertical line of corresponding length is made. The initial point of this solution is determined at the upper left corner of the vehicle to be marked, so the auxiliary vertical line 11 in the horizontal direction extends to the right, and the auxiliary vertical line in the vertical direction extends downward.

[0052] S7: When the data processing module detects that the mouse cursor moves toward the recommended point 4, the data processing module selects a threshold number of pixel points on both sides of the auxiliary vertical line as the expansion area 3. The expansion area 3 is shown in the attached figure. Figure 2 As shown, the data processing module selects the image in the expansion area 3; if the distance between the mouse cursor and each of the 4 recommended points is shortened, the image in the expansion area 3 is enlarged by the threshold multiple; if the distance between the mouse cursor and each of the 4 recommended points is increased, the image in the expansion area 3 is reduced by the threshold multiple;

[0053] The data processing module is also used to obtain the horizontal and vertical coordinates of each recommended point 4; calculate the horizontal and vertical coordinate differences between the pre-selected point 1 and each recommended point 4. If any of the horizontal and vertical coordinate differences decreases when the pre-selected point moves, it is determined that the pre-selected point 1 is approaching the recommended point 4, and the image of the expansion area 3 is enlarged; if both differences increase, it is determined that the pre-selected point 1 is moving away from the recommended point 4, and the image of the expansion area 3 is reduced. The operator identifies the boundary point between the vehicle 4 to be framed and the environment based on the enlarged or reduced image of the expansion area 3, and moves the auxiliary vertical line in the vertical direction and the auxiliary vertical line in the horizontal direction to the boundary point of the corresponding area by moving the mouse;

[0054] S8: Click the mouse to confirm the initial point;

[0055] S9: Move the mouse to the end point and click to complete the selection of the end point. The data processing module generates a marking box according to the coordinate positions of the initial point and the end point. The operator marks the type of the selected vehicle in the marking box. The types of vehicles include at least cars, trucks, motorcycles and special vehicles.

[0056] S10: Repeat steps S2-S9; mark other vehicles in the image. After marking one vehicle, when marking the next vehicle, usually, the operator needs to drag the mouse cursor to the initial point of the next vehicle. However, this solution determines the initial candidate area through eyeball recognition, and then the data processing module automatically moves the mouse cursor to the geometric center of the initial candidate area, reducing the distance of manual movement.

[0057] It also includes a rapid annotation device for autonomous driving scene pictures, including a display screen, a camera, a mouse and a data processing module. The camera is installed above the display screen to capture the facial information of the operator. The display screen is used to display the picture to be annotated. After the data processing module receives a signal from the operator clicking or moving the mouse, it controls the corresponding movement of the mouse cursor in the display screen. The data processing module is used to process the facial information. The data processing module selects the i5 12400f CPU processor to process the image in this solution.

[0058] The data processing module identifies the eye direction of the operator through facial information, and calculates the position on the screen that the operator is looking at based on the eye direction. If the time spent looking at a position exceeds a threshold, the area at the position is confirmed as the initial candidate area, and the geometric center of the initial candidate area is selected as the pre-selected point 1; the data processing module generates auxiliary vertical lines 11 in the vertical and horizontal directions on the mouse cursor at the position of the pre-selected point 1;

[0059] The data processing module obtains the coordinates of the pre-selected point 1 and moves the mouse cursor to the position of the pre-selected point 1; the data processing module is also used to identify the pixel points in the initial candidate area, find the boundary point between the vehicle and the environment through grayscale recognition, highlight it, and generate it as recommended point 4; through image recognition, the pixel points in the initial candidate area are processed, and the outline of the vehicle to be marked in the initial candidate area is identified by comparing the grayscale color difference of adjacent pixels, and the points that make up the outline are generated as recommended point 4 to prompt the operator.

[0060] When the data processing module detects that the mouse cursor moves toward the recommended point 4, the data processing module selects the threshold number of pixel points on both sides of the auxiliary vertical line as the expansion area 3, and the data processing module selects the image in the expansion area 3; if the distance between the mouse cursor and each recommended point 4 is shortening, the image in the expansion area 3 is enlarged by the threshold multiple; if the distance between the mouse cursor and each recommended point 4 is increasing, the image in the expansion area 3 is reduced by the threshold multiple;

[0061] When the operator drags the mouse to move toward or away from the recommended point 4, the image in the direction of the auxiliary vertical line will be enlarged or reduced accordingly; by enlarging and reducing the image, the selection boundary can be accurately controlled to prevent the selection of too much or too little content.

[0062] The mouse is also used to confirm the end point on the image according to the operator's operating instructions; the method for determining the end point can be through conventional dragging of the mouse to the point determined by the operator, or referring to the method for determining the initial point, and completing the long-distance dragging of the mouse by eye direction, thereby reducing the situation where the operator drags the mouse for too long a distance, which is easy to cause fatigue.

[0063] The data processing module generates a rectangular frame based on the line connecting the initial point and the end point as the diagonal line, and the rectangular frame selects the vehicle 4 to be selected; after the selection is completed, the data processing module recognizes the image information of the selected vehicle and marks the corresponding vehicle type.

[0064] The data processing module divides the photo to be annotated into two parts, left and right. If the number of annotated photos exceeding a threshold number are all from the same part, the data processing module highlights and flashes the other part of the photo to prompt the operator to annotate in this area.

[0065] In this solution, operators only need one click to complete the selection, which greatly reduces the workload and process, and reduces hand fatigue. It reduces the long-distance dragging of the mouse when the operator confirms the initial point in a group of annotations. At the same time, through the setting of auxiliary axes, it is convenient for operators to select the vehicle more accurately.

[0066] Since the working habit of data labeling often requires the head to be facing one position for a long time, it will cause neck strain in the long run. Therefore, through the setting of this solution, when the operator faces one area for more than a threshold, the data processing module will move the cursor to another area by flashing, thereby driving the operator's head to turn, causing the neck to twist, and preventing strain.

[0067] Embodiment 2

[0068] The difference between this solution and the first embodiment is that it also includes left and right movement keys and up and down movement keys, which are used by the operator to manually adjust the mouse cursor in step S7 in the left and right directions and up and down directions, respectively, and each movement distance is one pixel. When a single press of the left and right movement keys and the up and down movement keys exceeds the threshold, a long press signal is generated, and the data processing module controls the movement speed of the mouse cursor to double. By setting the up and down movement keys and the left and right movement keys, the problem that the mouse is not easy to stop at a precise position when dragging is solved. For example, the mouse is pulled too far and cannot stop accurately at a certain pixel point. By setting the movement keys, the position of the initial point can be accurately set to prevent excessive or insufficient offsets, making the labeled samples more accurate.

[0069] Embodiment 3

[0070] The difference between Example 3 and Example 1 is that: in step S7, when the operator operates the mouse to move from the pre-selected point 1 to the recommended point 4, the initial moving direction is confirmed by the coordinates after the movement and the pre-selected point 1. If the distance between the mouse cursor and each recommended point 4 is shortened, the image in the expansion area 3 is magnified by the threshold multiple. The data processing module detects that the mouse cursor movement is opposite to the initial moving direction. If the speed of the reverse movement does not exceed the preset value, the enlarged image of the expansion area 3 is enlarged by two times. If the reverse movement speed exceeds the preset value, the image of the expansion area 3 is reduced by half. When the operator moves the mouse, he may accidentally move the auxiliary vertical line beyond the intersection of the vehicle and the environment. At this time, he will move the mouse back. The data processing module identifies the withdrawal process. If the speed is within the preset value, the image of the expansion area 3 will be enlarged again to facilitate the operator's selection. If the speed exceeds the preset value, it is possible that the operator slipped, resulting in a farther offset position. At this time, by reducing the expansion area 3, the mouse can be quickly returned to the vicinity of the dividing point. If the data processing module detects that the speed is about to reach the preset value, the image of the expansion area 3 will be enlarged to the original multiple for easy viewing.

[0071] The above is only an embodiment of the present invention, and the common sense of the known specific structure and characteristics in the scheme is not described too much here. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the scope of protection of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims. The above is only an embodiment of the present invention, and the common sense of the known specific structure and characteristics in the scheme is not described too much here. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the scope of protection of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A fast annotation method for autonomous driving scene images, characterized in that: The following steps are involved: S1: Display the image to be annotated on the screen; S2: The camera obtains the operator’s facial information; S3: The data processing module identifies the operator's eye direction through facial information, and calculates the position on the screen that the operator is looking at based on the eye direction. If the time spent looking at a position exceeds a threshold, the area at that position is identified as the initial candidate area, and the geometric center of the initial candidate area is selected as the pre-selected point. S4: The data processing module obtains the coordinates of the pre-selected point and moves the mouse cursor to the pre-selected point; S5: Identify the pixels in the initial candidate area, find the boundary between the vehicle and the environment through grayscale recognition, highlight it, and generate several recommended points; S6: The data processing module generates auxiliary vertical lines in the vertical direction and the horizontal direction on the mouse cursor at the pre-selected point position; S7: When the data processing module detects that the mouse cursor moves toward the recommended point, the data processing module selects a threshold number of pixel points on both sides of the auxiliary vertical line as the expansion area, selects a threshold number of pixel points on the left and right sides of the auxiliary vertical line in the vertical direction and pixel points on the upper and lower sides of the auxiliary vertical line in the horizontal direction as the expansion area, and the data processing module selects the image in the expansion area; if the distance between the mouse cursor and each recommended point is shortening, the image in the expansion area is enlarged by a threshold multiple; if the distance between the mouse cursor and each recommended point is increasing, the image in the expansion area is reduced by a threshold multiple; the operator identifies the boundary point between the vehicle to be selected and the environment according to the enlarged or reduced image of the expansion area, and moves the auxiliary vertical line in the vertical direction and the auxiliary vertical line in the horizontal direction to the boundary point of the corresponding area by moving the mouse; S8: Click the mouse to confirm the initial point; S9: Move the mouse to the end point, click to complete the selection of the end point, and the data processing module generates a labeling box according to the coordinate positions of the initial point and the end point; S10: Repeat steps S2-S9 to mark other vehicles in the image.

2. The method for rapid annotation of autonomous driving scene images according to claim 1, characterized in that: In step S7, the data processing module is also used to obtain the horizontal and vertical coordinates of each recommended point; Calculate the horizontal and vertical coordinate differences between the pre-selected point and each recommended point. If either the horizontal and vertical coordinate differences decrease when the pre-selected point moves, it is determined that the pre-selected point is approaching the recommended point, and the image of the expanded area is enlarged; if both differences increase, it is determined that the pre-selected point is moving away from the recommended point, and the image of the expanded area is reduced.

3. The method for rapid annotation of autonomous driving scene images according to claim 1, characterized in that: In step S7, when the operator operates the mouse to move from the pre-selected point to the recommended point, the initial moving direction is confirmed by the coordinates after the movement and the pre-selected point. If the distance between the mouse cursor and each recommended point is shortened, the image in the expansion area is magnified by a threshold multiple. The data processing module detects that the movement of the mouse cursor is opposite to the initial moving direction. If the speed of the reverse movement does not exceed the preset value, the enlarged image of the expansion area is enlarged by two times. If the reverse movement speed exceeds the preset value, the image of the expansion area is reduced by half.

4. The method for rapid annotation of autonomous driving scene images according to claim 1, characterized in that: It also includes a left-right movement key and an up-down movement key, which are used by the operator to move the mouse cursor in the left-right direction and the up-down direction in the corresponding manual adjustment step S7, and each movement distance is one pixel point; In step S9, it is also necessary to mark the type of the vehicle selected in the marking box, and the types of vehicles at least include cars, trucks, motorcycles and special vehicles.

5. The method for rapid annotation of autonomous driving scene images according to claim 4, characterized in that: When a single press of the left and right movement keys and the up and down movement keys exceeds the threshold, a long press signal is generated, and the data processing module controls the movement speed of the mouse cursor to double.

6. The method for rapid annotation of autonomous driving scene images according to claim 4, characterized in that: The vehicle type generates corresponding selection items which are displayed on the screen, and the selection items are scattered on the left and right sides and the upper and lower sides of the screen.

7. The method for rapid annotation of autonomous driving scene images according to claim 6, characterized in that: The selection items for sedans are on the left side of the screen, the selection items for trucks are on the right side of the screen, the selection items for motorcycles are on the upper side of the screen, and the selection items for special vehicles are on the lower side of the screen; after the data processing module generates a rectangular frame based on the first selection point and the second selection point, the camera captures the operator's eye direction again, and determines the label type of the selected vehicle based on the selection items corresponding to the type of vehicle the operator is looking at.

8. The method for rapid annotation of autonomous driving scene images according to claim 6, characterized in that: In step S10, when selecting the next vehicle to be framed, the data processing module divides the photo into two parts, left and right. If the number of consecutive annotated photos exceeding the threshold is from the same part, the data processing module highlights the other part of the photo to prompt the operator to annotate in this area.

9. A fast labeling device for autonomous driving scene pictures, characterized in that: It includes a display screen, a camera, a mouse and a data processing module. The camera is installed above the display screen to capture the facial information of the operator. The display screen is used to display the picture to be annotated. After the data processing module receives the signal that the operator clicks or moves the mouse, it controls the mouse cursor in the display screen to move accordingly. The data processing module is used to process the facial information. The data processing module identifies the operator's eye direction through facial information, and calculates the position on the screen that the operator is looking at based on the eye direction. If the time spent looking at a position exceeds a threshold, the area at the position is confirmed as the initial candidate area, and the geometric center of the initial candidate area is selected as the pre-selected point. The data processing module generates auxiliary vertical lines in the vertical and horizontal directions on the mouse cursor at the pre-selected point. The data processing module obtains the coordinates of the pre-selected point and moves the mouse cursor to the pre-selected point. The data processing module is also used to identify the pixel points in the initial selected area, find the boundary point between the vehicle and the environment through grayscale recognition, highlight it, and generate it as a recommended point. When the operator drags the mouse to move toward or away from the recommended point, the image in the direction in which the auxiliary vertical line extends will be enlarged or reduced accordingly; The mouse is also used to confirm the end point on the picture according to the operation instruction of the operator; The data processing module generates a rectangular frame based on the line connecting the initial point and the end point as the diagonal line, and the rectangular frame selects the vehicle to be selected; after the selection is completed, the data processing module recognizes the image information of the selected vehicle and marks the corresponding vehicle type.

10. The rapid annotation device for autonomous driving scene images according to claim 9, characterized in that: The data processing module divides the photo to be annotated into two parts, left and right. If the number of annotated photos exceeding a threshold number are all from the same part, the data processing module highlights and flashes the other part of the photo to prompt the operator to annotate in this area.

Images