Light source bounding box labeling method, region detection model training method and equipment

Abstract

This application discloses a method for annotating light source bounding boxes, a method for training a region detection model, and an apparatus. The method includes: acquiring a target image; wherein the target image is obtained by collecting data from a light source in a working state; determining a corresponding first pixel threshold for a plurality of first pixels in the target image; wherein the first pixel threshold is obtained by statistically analyzing the pixel values ​​of reference pixels of the first pixels using threshold influence parameters, and the reference pixels are pixels in the target image that have a specific positional relationship with the first pixels; selecting second pixels belonging to bright regions of the target image from the plurality of first pixels using the comparison results between the pixel values ​​of each first pixel and the corresponding first pixel threshold; and determining the annotated bounding box corresponding to the light source in the target image using the second pixels. Through the above method, this application can improve the quality of the annotated light source bounding boxes.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method for annotating light source bounding boxes, a method for training region detection models, and an apparatus. Background Technology

[0002] Light sources, such as car headlights and streetlights, appear as point sources in images when in operation. Their brightness gradually transitions from the center outwards, lacking clear edge information, making the annotation of their bounding boxes difficult. Currently, the bounding boxes are mainly annotated manually, which is susceptible to the subjective influence of the annotator, resulting in poor quality bounding boxes. Summary of the Invention

[0003] The main technical problem addressed in this application is to provide a method for annotating light source bounding boxes, a method for training region detection models, and an apparatus that can improve the quality of annotated light source bounding boxes.

[0004] To address the aforementioned technical problems, this application provides a method for annotating the bounding box of a light source. The method includes: acquiring a target image; wherein the target image is obtained by collecting data from a light source in a working state; determining corresponding first pixel thresholds for several first pixels in the target image; wherein the first pixel threshold is obtained by statistically analyzing the pixel values ​​of reference pixels of the first pixel using threshold influence parameters, and the reference pixels are pixels in the target image that have a specific positional relationship with the first pixel; selecting second pixels belonging to the bright area of ​​the target image from among the several first pixels using the comparison results between the pixel values ​​of each first pixel and the corresponding first pixel threshold; and determining the bounding box corresponding to the light source in the target image using the second pixels.

[0005] To address the aforementioned technical problems, another technical solution adopted in this application is: providing a training method for a region detection model, the method comprising: acquiring sample images; wherein, the sample images are obtained by acquiring light sources in a working state, and the sample images are marked with the ground truth bounding boxes of the light sources, the ground truth bounding boxes being marked by the aforementioned method for marking the light source bounding boxes; using a region detection model to detect the sample images to obtain the predicted bounding boxes of the light sources; and adjusting the network parameters of the region detection model based on the difference between the ground truth bounding boxes and the predicted bounding boxes.

[0006] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide an electronic device, which includes a processor and a memory. The memory stores program instructions, and the processor executes the program instructions to implement the above-mentioned training method for the region detection model or the above-mentioned annotation method for the light source bounding box.

[0007] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium for storing program instructions that can be executed to implement the above-mentioned training method for the region detection model or the above-mentioned annotation method for the light source bounding box.

[0008] In the above technical solution, since the first pixel threshold corresponding to each first pixel is associated with pixels that have a specific positional relationship with each first pixel, the determination of the first pixel threshold corresponding to each first pixel takes into account the pixel value information of pixels that have a specific positional relationship with itself. Therefore, the determined first pixel threshold corresponding to each first pixel is more reasonable and accurate. Thus, the second pixel in the target image that belongs to the bright area is more accurate when determined by comparing the pixel value of each first pixel with the corresponding first pixel threshold. This results in higher quality of the bounding box corresponding to the light source in the target image determined by the second pixel. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating an embodiment of the light source boundary box annotation method provided in this application;

[0010] Figure 2 This is a schematic diagram of an embodiment of the target image after grayscale processing provided in this application;

[0011] Figure 3 Figure 1 The flowchart of step S12 shown is a schematic diagram of one embodiment;

[0012] Figure 4 This is a flowchart illustrating an embodiment of the method for determining threshold influence parameters provided in this application;

[0013] Figure 5 yes Figure 4 The flowchart of step S42 shown is a schematic diagram of one embodiment;

[0014] Figure 6 yes Figure 4 The flowchart of step S43 shown is a schematic diagram of one embodiment.

[0015] Figure 7 yes Figure 1 The flowchart of an embodiment of step S14 is shown below;

[0016] Figure 8 yes Figure 7 The flowchart of step S71 shown is a schematic diagram of an embodiment.

[0017] Figure 9 This is a schematic diagram of an embodiment of the binarized image corresponding to the target image provided in this application;

[0018] Figure 10 yes Figure 7 The flowchart of step S72 shown is a schematic diagram of one embodiment;

[0019] Figure 11 This is a schematic diagram of an embodiment of the candidate bounding box for the light source and the second annotation key point provided in this application;

[0020] Figure 12 This is a schematic diagram of an embodiment of the annotation boundary box of the light source provided in this application;

[0021] Figure 13 This is a flowchart illustrating an embodiment of the training method for the region detection model provided in this application;

[0022] Figure 14 This is a schematic diagram of an embodiment of the network structure of the region detection model provided in this application;

[0023] Figure 15 The detection results provided in this application are obtained by using a region detection model trained to convergence to detect images.

[0024] Figure 16 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application;

[0025] Figure 17 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0026] The following describes the embodiments of the present application in detail with reference to the accompanying drawings.

[0027] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0028] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] Please see Figure 1 , Figure 1This is a flowchart illustrating an embodiment of the light source bounding box annotation method provided in this application. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily reflect that result. Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment includes:

[0030] Step S11: Obtain the target image.

[0031] When a light source is in operation, it appears as a point light source in the image, with brightness gradually transitioning from the center outwards. There is no obvious edge information, making the annotation of the light source's bounding box difficult and easily affected by the subjective factors of the annotator, resulting in poor quality annotated light source bounding boxes. The method in this embodiment aims to improve the quality of annotated light source bounding boxes and avoid the problem of poor quality caused by subjective factors in manual annotation. The light sources mentioned herein include, but are not limited to, streetlights, vehicle lights, and household lighting, etc., and are not specifically limited thereto.

[0032] Since a light source only appears as a point light source in an image when it is in operation, the lack of clear edge information, leading to difficulties in annotating the light source's bounding box, is a problem that arises when the light source is in operation. Therefore, in this embodiment, a target image is acquired, which is obtained by capturing images of the light source in operation.

[0033] In one embodiment, the target image may be acquired at night, where light sources appear as blurred outlines and lack edge information in the image. Of course, in other embodiments, the target image may also be acquired in environments such as rain, snow, or fog. In environments with poor lighting conditions such as rain, snow, or fog, light sources also appear as blurred outlines and lack edge information in the image.

[0034] In one embodiment, the target image can be obtained from local storage or cloud storage. Of course, in other embodiments, it can also be obtained in real time by using an image acquisition device to capture the environment, including the light source in operation; this is not specifically limited here.

[0035] Step S12: Determine the corresponding first pixel threshold for each of the first pixel points in the target image.

[0036] In this embodiment, a first pixel threshold is determined for each of several first pixels in the target image. The first pixel threshold is obtained by statistically analyzing the pixel values ​​of reference pixels for each first pixel using threshold influence parameters. The reference pixels are pixels in the target image that have a specific positional relationship with the first pixel. In other words, the first pixel threshold corresponding to a first pixel is associated with pixels that have a specific positional relationship with that first pixel. The determination of the first pixel threshold takes into account the pixel value information of pixels with a specific positional relationship with that first pixel, making the determined first pixel threshold more reasonable and accurate, and improving the quality of the bounding boxes corresponding to the light source in the subsequently determined target image. Furthermore, it should be noted that since the first pixel threshold corresponding to a first pixel is associated with pixels that have a specific positional relationship with that first pixel, and the pixels with specific positional relationships corresponding to several first pixels in the target image are different, it is necessary to determine the first pixel threshold for each of the several first pixels in the target image separately.

[0037] In one embodiment, the threshold influence parameter includes at least one of a position parameter for determining a specific positional relationship and a statistical parameter for influencing the pixel value of a reference pixel.

[0038] In one embodiment, the threshold influence parameter is preset. Of course, in other embodiments, the threshold influence parameter is obtained through analysis of several images, specifically determined based on the bounding box quality parameters of the light sources in the images. Since it is determined based on the bounding box quality parameters of the light sources in the images, a higher bounding box quality parameter indicates a higher quality of the subsequently generated labeled bounding boxes. Therefore, the labeled bounding boxes of the light sources generated by the threshold influence parameter determined based on the bounding box quality parameters of the light sources will have higher quality. Furthermore, it should be noted that the several images are all acquired from light sources in a working state, and the several images may or may not include the target image.

[0039] In one embodiment, the plurality of first pixels refers to all first pixels in the target image. To improve the efficiency of determining the bounding box corresponding to the light source in the target image, in other embodiments, the plurality of first pixels may also refer to all first pixels in a certain region of the target image.

[0040] like Figure 2 As shown, Figure 2This is a schematic diagram of an embodiment of a target image after grayscale processing provided in this application. To improve the efficiency of determining the first pixel threshold corresponding to a plurality of first pixels, in one embodiment, the target image is grayscale processed before determining the corresponding first pixel threshold for each of the plurality of first pixels in the target image. To improve the accuracy of the determined first pixel threshold corresponding to the plurality of first pixels, in one embodiment, the target image is preprocessed to remove noise points in the target image; wherein, the preprocessing of the target image includes, but is not limited to, linear normalization, Gaussian blur, etc., and is not specifically limited here.

[0041] In one specific implementation, after grayscale processing of the target image, linear normalization is performed on the target image. The formula for linear normalization is as follows:

[0042]

[0043] Where dst(i,j) represents the linearly normalized value of the first pixel (i,j) in the target image after grayscale processing; src(i,j) represents the original value of the first pixel (i,j) in the target image after grayscale processing; max and min represent the maximum and minimum grayscale values ​​in the input target image after grayscale processing, respectively; max' and min' represent the maximum and minimum grayscale values ​​in the scaled target range, respectively.

[0044] In one specific implementation, the target image is subjected to Gaussian blurring after grayscale processing. The specific formula for Gaussian blurring is shown below:

[0045]

[0046] Where G(u,v) represents the weight of the center point (u,v) of the filter matrix; σ represents the blur radius. Specifically, the weights of all first pixels in the grayscale-processed target image are calculated to obtain the filter matrix; then, the filter matrix is ​​multiplied by the input grayscale-processed target image (the multiplied part has the same size as the filter matrix); finally, the results are added together to obtain the blur value of the first pixel (i,j); by performing the above process on all first pixels in the grayscale-processed target image, the Gaussian blurred target image can be obtained.

[0047] In one specific implementation, after grayscale processing of the target image, linear normalization and Gaussian blur processing are performed on the target image.

[0048] Step S13: Using the comparison results between the pixel value of each first pixel and the corresponding first pixel threshold, select the second pixel that belongs to the bright area of ​​the target image from a number of first pixels.

[0049] In this embodiment, by comparing the pixel value of each first pixel with the corresponding first pixel threshold, a second pixel belonging to the bright area of ​​the target image is selected from a plurality of first pixels. Since the pixel value of the first pixel can characterize the brightness information of the first pixel, and the first pixel threshold corresponding to the first pixel is obtained by statistically analyzing the pixel values ​​of reference pixels with a specific positional relationship to the first pixel using threshold influence parameters, the first pixel threshold of the first pixel can characterize the brightness information of the area with a specific positional relationship to the first pixel. Therefore, by comparing the pixel value of the first pixel with the first pixel threshold, it can be determined whether the first pixel belongs to the bright area in the target image, thereby identifying the bright area in the target image. The first pixel belonging to the bright area of ​​the target image is defined as the second pixel.

[0050] Since the first pixel threshold corresponding to each first pixel is associated with pixels having a specific positional relationship with it, the determination of the first pixel threshold for each first pixel takes into account the pixel value information of pixels with a specific positional relationship to itself. Therefore, the determined first pixel thresholds for each first pixel are more reasonable and accurate. Consequently, the second pixels belonging to the bright area in the target image determined by comparing the pixel values ​​of each first pixel with their corresponding first pixel thresholds are more accurate, resulting in higher quality bounding boxes corresponding to light sources in the subsequently determined target image. Because each first pixel has its own corresponding first pixel threshold, selecting the second pixels belonging to the bright area of ​​the target image from several first pixels using the comparison results between the pixel values ​​of each first pixel and their corresponding first pixel thresholds can be seen as using a dynamic thresholding method to determine pixels belonging to the bright area in the target image.

[0051] In one embodiment, a first pixel with a pixel value greater than a first pixel threshold can be selected from a plurality of first pixels and designated as a second pixel belonging to a bright area of ​​the target image. Since the pixel value of a first pixel can characterize the brightness information of the first pixel, and the first pixel threshold can characterize the brightness information of a region with a specific positional relationship to the first pixel, a first pixel with a pixel value greater than its corresponding first pixel threshold indicates that the brightness of the first pixel is greater than the brightness of the region with a specific positional relationship to the first pixel, and thus it can be designated as a pixel belonging to a bright area in the target image.

[0052] Specifically, if the pixel value of the first pixel is greater than its corresponding first pixel threshold, then the first pixel is determined to be a pixel belonging to the bright area of ​​the target image; if the pixel value of the first pixel is less than or equal to its corresponding first pixel threshold, then the first pixel is determined to be a pixel belonging to the non-bright area of ​​the target image.

[0053] Step S14: Use the second pixel to determine the bounding box corresponding to the light source in the target image.

[0054] In this embodiment, the second pixel is used to determine the bounding box corresponding to the light source in the target image. Since the second pixel is a pixel belonging to the bright area in the target image, and the pixels belonging to the bright area are determined based on the first pixel threshold corresponding to each first pixel, which is obtained by statistically analyzing the pixel values ​​of the reference pixels of the first pixel using threshold influence parameters; and since the first pixel threshold corresponding to each first pixel is associated with pixels having a specific positional relationship with itself, the determination of the first pixel threshold takes into account the pixel value information of pixels with a specific positional relationship with itself. Therefore, the determined first pixel thresholds corresponding to each first pixel are more reasonable and accurate. Thus, the second pixels belonging to the bright area in the target image determined using the first pixel thresholds corresponding to each first pixel are more accurate, resulting in higher quality bounding boxes corresponding to the light source in the target image determined using the second pixel.

[0055] In one embodiment, the bounding box corresponding to the light source in the target image can be directly determined based on the second pixel. To improve the quality of the bounding box corresponding to the light source in the determined target image, in other embodiments, candidate bounding boxes for the light source can be generated first using the second pixel, and then the bounding box corresponding to the light source can be obtained based on the candidate bounding boxes.

[0056] In the above embodiments, since the first pixel threshold corresponding to each first pixel is associated with a pixel that has a specific positional relationship with each first pixel, the determination of the first pixel threshold corresponding to each first pixel takes into account the pixel value information of the pixel that has a specific positional relationship with itself. The determined first pixel threshold corresponding to each first pixel is more reasonable and accurate. Therefore, the second pixel in the target image that belongs to the bright area is more accurate when determined by comparing the pixel value of each first pixel with the corresponding first pixel threshold. This makes the quality of the bounding box corresponding to the light source in the target image determined by the second pixel higher.

[0057] Please see Figure 3 , Figure 3 Figure 1The flowchart shown is a schematic diagram of one embodiment of step S12. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily follow the same pattern. Figure 3 The process sequence shown is limited. Figure 3 As shown, in this embodiment, the threshold influence parameters include positional parameters used to determine specific positional relationships and statistical parameters used to influence the statistical results of the pixel values ​​of reference pixels, specifically including:

[0058] Step S31: For each first pixel, determine multiple reference pixels that satisfy a specific positional relationship with the first pixel based on the position parameters.

[0059] In this embodiment, for each first pixel, multiple reference pixels that satisfy a specific positional relationship with the first pixel are determined based on position parameters. Since the position parameters are used to determine the specific positional relationship, multiple reference pixels that satisfy the specific positional relationship with the first pixel can be determined based on the position parameters.

[0060] The specific positional relationship is not limited and can be set according to actual usage needs. For example, multiple reference pixels that satisfy a specific positional relationship with the first pixel are pixels located on the same straight line as the first pixel; or, multiple reference pixels that satisfy a specific positional relationship with the first pixel are pixels within a preset size area centered on the first pixel.

[0061] In one embodiment, the position parameter is the window width. Specifically, determining multiple reference pixels that satisfy a specific positional relationship with the first pixel based on the position parameter involves: acquiring pixels in the target image centered on the first pixel and with a width equal to the position parameter, as reference pixels for the first pixel. For example, if the window width is 3 and the first pixel is (i,j), then the multiple reference pixels that satisfy a specific positional relationship with the first pixel (i,j) are the pixels in the window centered on the first pixel (i,j) and with a window width of 3.

[0062] Step S32: Obtain the central tendency representation value of the pixel values ​​of each reference pixel of the first pixel.

[0063] In this embodiment, the central tendency representation value of the pixel values ​​of each reference pixel of the first pixel is obtained. The type of central tendency representation value is not limited; for example, the average, median, mode, etc., of the pixel values ​​of each reference pixel of the first pixel can be used as the central tendency representation value of the pixel values ​​of each reference pixel of the first pixel.

[0064] In one embodiment, the central tendency characterization value is the average value. Then, the average value of the pixel values ​​of each reference pixel of the first pixel is obtained to obtain the central tendency characterization value of the pixel values ​​of each reference pixel of the first pixel.

[0065] Step S33: Using the central tendency characterization value, the pixel difference between the central tendency characterization value and the pixel value of the first pixel, and statistical parameters, obtain the first pixel threshold of the first pixel.

[0066] In this embodiment, the first pixel threshold of the first pixel is obtained using the central tendency characteristic value, the pixel difference between the central tendency characteristic value and the pixel value of the first pixel, and statistical parameters. In one embodiment, obtaining the first pixel threshold of the first pixel using the central tendency characteristic value, the pixel difference between the central tendency characteristic value and the pixel value of the first pixel, and statistical parameters specifically involves: First, obtaining the ratio between the pixel difference and the corresponding first difference value; then multiplying the first constant by the second difference value between the ratio and the statistical parameters to obtain the multiplication result; wherein the first difference value is the difference between the second constant and the pixel difference; then, multiplying the third constant by the sum of the multiplication results by the central tendency characteristic value to obtain the first pixel threshold of the first pixel. The first, second, and third constants are not limited and can be set according to actual usage needs; for example, the first, second, and third constants are all 1.

[0067] In one specific implementation, the position parameters are the window width and the first, second, and third constants are all 1. The specific formula for obtaining the first pixel threshold of the first pixel using the central tendency characterization value, the pixel difference between the central tendency characterization value and the pixel value of the first pixel, and statistical parameters is as follows:

[0068]

[0069] Where k represents a statistical parameter; T(x,y) represents the first pixel threshold of the first pixel (x,y); μ(x,y) represents the central tendency of the pixel values ​​of each reference pixel of the first pixel (x,y), where the reference pixels of the first pixel (x,y) are the pixels in a window centered on the first pixel (x,y) and with a window width of ω; Δ(x,y) represents the pixel difference between the central tendency and the pixel value of the first pixel, specifically Δ(x,y) = I(x,y) - μ(x,y); 1 - Δ(x,y) represents the difference between the second constant and the pixel difference, i.e., the first difference. This represents the ratio between the pixel difference and the corresponding first difference value; The result of the multiplication is obtained by multiplying the second difference between the first constant and the ratio by the statistical parameter; This represents the sum of the product of the third constant and the product of the two constants.

[0070] It should be noted that the positional and statistical parameters can be preset or determined based on the bounding box quality parameters of light sources in several images; no specific limitations are imposed here. Furthermore, for each of the several first pixels in the target image, steps S31-S33 need to be executed separately.

[0071] Please see Figure 4 , Figure 4 This is a flowchart illustrating an embodiment of the method for determining threshold influence parameters provided in this application. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily reflect that outcome. Figure 4 The process sequence shown is limited. Figure 4 As shown, in this embodiment, the threshold influence parameter is determined based on the bounding box quality parameters of the light sources in several images. Therefore, before determining the corresponding first pixel threshold for several first pixel points in the target image, the specific steps include:

[0072] Step S41: Obtain the value range corresponding to each threshold influence parameter, and determine the initial current value of each threshold influence parameter within the value range.

[0073] In this embodiment, Bayesian hyperparameter tuning is used to obtain the threshold influence parameters that maximize the quality of the bounding boxes of the labeled light sources. The specific process is as follows: By specifying the value range of the threshold influence parameters, the bounding box quality parameters affected by the threshold influence parameters, and the number of iterations, the threshold influence parameters that optimize the bounding box quality parameters can be obtained. The advantage of this approach is that during iteration, Bayesian optimization only needs to know the input and output of the bounding box quality parameters, without needing to consider the specific process of setting the bounding box quality parameters, which are then determined based on the user's task. Furthermore, Bayesian hyperparameter tuning fully considers information from the previous iteration, enabling better adjustment of the threshold influence parameters.

[0074] Therefore, in this embodiment, the value range corresponding to each threshold influence parameter is obtained, and the initial current value of each threshold influence parameter is determined within the value range.

[0075] In one embodiment, the threshold influence parameter includes a position parameter for determining a specific positional relationship and a statistical parameter for influencing the pixel value of a reference pixel; the value range of the threshold influence parameter obtained at this time is: the value range of the position parameter is [5, 25] and the value range of the statistical parameter is [0.25, 0.75].

[0076] Step S42: Using the current value of the threshold influence parameter, obtain the current bounding box corresponding to the light source in each image.

[0077] In this embodiment, the current bounding box corresponding to the light source in each image is obtained by utilizing the current value of the threshold influence parameter. In one embodiment, such as Figure 5 As shown, Figure 5 yes Figure 4 The flowchart of step S42 shown is a schematic diagram of an embodiment. Obtaining the current bounding box corresponding to the light source in each image using the current value of the threshold influence parameter specifically includes the following sub-steps:

[0078] Step S51: For each image, determine the corresponding second pixel threshold for several third pixel points in the image.

[0079] Step S51 is similar to step S12, and will not be described again here.

[0080] Step S52: Using the comparison results between the pixel value of each third pixel and the corresponding second pixel threshold, select the fourth pixel that belongs to the bright area of ​​the image from several third pixels.

[0081] Step S52 is similar to step S13, and will not be described again here.

[0082] Step S53: Using the fourth pixel, generate the current bounding box corresponding to the light source in the image.

[0083] Step S53 is similar to step S14, and will not be described again here.

[0084] Step S43: Evaluate the quality of the current bounding box corresponding to the light source in each image to obtain the bounding box quality parameter corresponding to the threshold influence parameter.

[0085] In this embodiment, the quality of the current bounding box corresponding to the light source in each image is evaluated to obtain the bounding box quality parameter corresponding to the threshold influence parameter.

[0086] In one embodiment, the quality of the current bounding box corresponding to the light source in each image can be directly evaluated. In other embodiments, the quality of the current bounding box corresponding to the light source in each image can also be evaluated by combining the first annotation key points of the light source marked on each image.

[0087] In one embodiment, if Figure 6 As shown, Figure 6 yes Figure 4 The flowchart shown in step S43 is a schematic diagram of an embodiment. Several images are marked with first annotation key points of light sources. The quality assessment of the current bounding box corresponding to the light source in each image is performed by combining the first annotation key points of the light source marked on each image. Specifically, it includes the following sub-steps:

[0088] Step S61: Statistically count the current bounding boxes and first annotation key points of several images to obtain the first total number of current bounding boxes contained in several images, the second total number of first annotation key points contained in several images, the number of key points of the first annotation key points contained in each current bounding box in several images, and the number of bounding boxes corresponding to each first annotation key point in several images. The number of bounding boxes corresponding to the first annotation key points is the number of current bounding boxes containing the first annotation key points.

[0089] In this embodiment, the current bounding boxes and first annotation key points of several images are statistically analyzed to obtain the first total number of current bounding boxes contained in several images, the second total number of first annotation key points contained in several images, the number of key points of the first annotation key points contained in each current bounding box in several images, and the number of bounding boxes corresponding to each first annotation key point in several images. The number of bounding boxes corresponding to the first annotation key points is the number of current bounding boxes containing the first annotation key points.

[0090] For example, taking two images, specifically image A and image B, as an example; image A contains 3 current bounding boxes, and image B contains 4 current bounding boxes, so the first total number of current bounding boxes in the two images is 7; image A contains 3 first keypoints, and image B contains 3 first keypoints, so the second total number of first keypoints in the two images is 6; in image A, the current bounding box α1 contains 1 first keypoint, the current bounding box α2 contains 1 first keypoint, and the current bounding box α3 contains 0 first keypoints. In image B, the number of key points of the first annotation key point contained in the current bounding box β1 is 1, the number of key points of the first annotation key point contained in the current bounding box β2 is 2, the number of key points of the first annotation key point contained in the current bounding box β3 is 0, and the number of key points of the first annotation key point contained in the current bounding box β4 is 0; in image A, the number of bounding boxes corresponding to the first annotation key point a1 is 1, the number of bounding boxes corresponding to the first annotation key point a2 is 1, and the number of bounding boxes corresponding to the first annotation key point a3 is 0; in image B, the number of bounding boxes corresponding to the first annotation key point b1 is 1, the number of bounding boxes corresponding to the first annotation key point b2 is 1, and the number of bounding boxes corresponding to the first annotation key point b3 is 1.

[0091] Step S62: Count the number of key points corresponding to each valid bounding box and the number of bounding boxes corresponding to each valid key point, and obtain the corresponding point count and bounding box count results.

[0092] In this embodiment, the number of key points corresponding to each valid bounding box and the number of bounding boxes corresponding to each valid key point are counted respectively, and the corresponding point count results and box count results are obtained. Among them, the valid bounding box is the current bounding box containing a first labeled key point greater than or equal to 1, and the valid key point is the first labeled key point contained by the current bounding box containing a first labeled key point greater than or equal to 1.

[0093] In one embodiment, the reciprocals of the number of keypoints corresponding to each valid bounding box are summed to obtain the point count result, and the reciprocals of the number of bounding boxes corresponding to each valid keypoint are summed to obtain the bounding box count result. The specific formulas are as follows:

[0094]

[0095]

[0096] Where, n K (b) represents the number of key points corresponding to the effective bounding box b; n B (k) represents the number of bounding boxes corresponding to valid keypoints k; This indicates the statistical results of the number of points; This displays the statistical results of the number of boxes.

[0097] For example, consider two images, specifically image A and image B. If the number of first-label keypoints in the current bounding box α1 of image A is 1, the number of first-label keypoints in the current bounding box α2 is 1, and the number of first-label keypoints in the current bounding box α3 is 0, and the number of first-label keypoints in the current bounding box β1 of image B is 1, the number of first-label keypoints in the current bounding box β2 is 2, the number of first-label keypoints in the current bounding box β3 is 0, and the number of first-label keypoints in the current bounding box β4 is 0; therefore, the current bounding boxes α1 and α2 in image A and β1 and β2 in image B are valid bounding boxes. If the number of bounding boxes corresponding to the first keypoint a1 in image A is 1, the number of bounding boxes corresponding to the first keypoint a2 is 1, and the number of bounding boxes corresponding to the first keypoint a3 is 0, and the number of bounding boxes corresponding to the first keypoint b1 in image B is 1, the number of bounding boxes corresponding to the first keypoint b2 is 1, and the number of bounding boxes corresponding to the first keypoint b3 is 1; therefore, the first keypoints a1 and a2 in image A and the first keypoints a1, a2, and a3 in image B are valid keypoints.

[0098] Furthermore, the reciprocals of the number of key points corresponding to each valid bounding box are summed to obtain the point count result, which is 1 + 1 + 1 + 1 / 2; the reciprocals of the number of bounding boxes corresponding to each valid key point are summed to obtain the box count result, which is 1 + 1 + 1 + 1 + 1.

[0099] Step S63: Based on the point count statistics, box count statistics, and the first and second total counts, obtain the bounding box quality parameters corresponding to the threshold influence parameters.

[0100] In this embodiment, the bounding box quality parameter corresponding to the current threshold influence parameter is obtained based on the point count statistics, the bounding box count statistics, the first total number of bounding boxes contained in several images, and the second total number of first labeled keypoints contained in several images.

[0101] In one embodiment, the reciprocal of the product of the first total quantity and the second total quantity, the point count result, and the box count result are multiplied together to obtain the bounding box quality parameter corresponding to the threshold influence parameter. The specific formula is as follows:

[0102]

[0103] Where q represents the bounding box quality parameter corresponding to the threshold influence parameter; N B N represents the first total quantity; K Indicates the second total quantity; This indicates the statistical results of the number of points; This displays the statistical results of the number of boxes.

[0104] Step S44: In response to the failure to meet the preset requirements, based on the current bounding box quality parameter corresponding to the threshold influence parameter, the threshold influence parameter is re-valued within the value range to obtain a new current value of the threshold influence parameter; and the current value of the threshold influence parameter is used to obtain the current bounding box corresponding to the light source in each image and subsequent steps.

[0105] In this embodiment, in response to the failure to meet the preset requirements, the threshold influence parameter is revalued within its range based on the current bounding box quality parameter corresponding to the threshold influence parameter, resulting in a new current value for the threshold influence parameter. The current value of the threshold influence parameter is then used to re-execute the process of obtaining the current bounding box corresponding to the light source in each image and subsequent steps. The preset requirements are that the current bounding box quality parameter corresponding to the threshold influence parameter meets the requirements and / or that the number of times the current value of the threshold influence parameter is used to obtain the current bounding box corresponding to the light source in each image has met the required number of executions. The final current value of the threshold influence parameter is used to determine the first pixel threshold. No restrictions are placed on the requirements for the bounding box quality parameter or the number of executions; for example, the requirements for the bounding box quality parameter to meet include a bounding box quality parameter greater than a preset value and a number of executions of 20.

[0106] In other words, when the quality parameter of the bounding box corresponding to the current threshold influence parameter does not meet the preset requirements, it indicates that the quality of the current bounding box of the light source generated using the current value of the threshold influence parameter is poor. At this point, the threshold influence parameter needs to be re-valued within its range, and the new current value of the threshold influence parameter is used to regenerate the current bounding box corresponding to the light source in each image. The quality of the current bounding box corresponding to the new current value of the threshold influence parameter is then re-evaluated. That is, during the iteration process, the threshold influence parameter is continuously adjusted based on the bounding box quality parameter to obtain the final threshold influence parameter. A higher quality bounding box parameter corresponding to the final threshold influence parameter makes the subsequent first pixel threshold determined using the final threshold influence parameter more closely match the corresponding first pixel, resulting in a more reasonable and accurate first pixel threshold. This leads to higher quality annotated bounding boxes of the light source generated using the first pixel thresholds corresponding to each first pixel.

[0107] For example, consider a threshold-affected parameter that includes a positional parameter ω used to determine specific positional relationships and a statistical parameter k used to influence the pixel values ​​of reference pixels. The positional parameter ω ranges from [5, 25], and the statistical parameter k ranges from [0.25, 0.75]. First, the initial current values ​​of the positional parameter ω and the statistical parameter k are determined, and the current bounding boxes corresponding to the light sources in each image are generated based on these initial current values. Then, the quality of the current bounding boxes corresponding to the light sources in each image generated based on the initial current values ​​of the positional parameter ω and the statistical parameter k is evaluated to obtain the bounding box quality parameter q corresponding to the initial current values ​​of the positional parameter ω and the statistical parameter k. Then, during the iteration process, the values ​​of the positional parameter ω and the statistical parameter k are continuously adjusted based on the bounding box quality parameter q corresponding to the current values ​​of the positional parameter ω and the statistical parameter k, finally obtaining ω = 19 and k = 0.4.

[0108] Please see Figure 7 , Figure 7 yes Figure 1 The diagram shows a flowchart of one embodiment of step S14. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily follow the same pattern. Figure 7 The process sequence shown is limited. Figure 7 As shown, this embodiment includes:

[0109] Step S71: Using the second pixel, generate at least one candidate bounding box for the light source in the target image.

[0110] In this embodiment, at least one candidate bounding box for a light source in the target image is generated using the second pixel. In one embodiment, as... Figure 8 As shown, Figure 8 yes Figure 7 The flowchart of step S71 shown is a schematic diagram of an embodiment. Generating at least one candidate bounding box for a light source in the target image using the second pixel specifically includes the following sub-steps:

[0111] Step S81: Obtain the binarized image corresponding to the target image.

[0112] In this embodiment, a binarized image corresponding to the target image is obtained. In this binarized image, the pixel value corresponding to the second pixel is the first pixel value, and the pixel values ​​of all other pixels are the second pixel values. The specific values ​​of the first and second pixel values ​​are not limited and can be set according to actual usage needs; for example, the first pixel value can be 1, and the second pixel value can be 0.

[0113] In one embodiment, if Figure 9 As shown, Figure 9 This is a schematic diagram of an embodiment of the binarized image corresponding to the target image provided in this application. The first pixel value is 1 and the second pixel value is 0. Specifically, the binarized image corresponding to the target image is obtained using the following formula:

[0114]

[0115] Where B(x,y) represents the pixel value of the first pixel (x,y) in the binarized image; I(x,y) represents the pixel value of the first pixel (x,y) in the target image; and T(x,y) represents the first pixel threshold corresponding to the first pixel (x,y). Specifically, when the pixel value of the first pixel (x,y) in the target image is greater than the corresponding first pixel threshold, it indicates that the first pixel (x,y) belongs to the bright area of ​​the target image, and the pixel value of the first pixel (x,y) is adjusted to the first pixel value, i.e., 1. When the pixel value of the first pixel (x,y) in the target image is less than or equal to the corresponding first pixel threshold, it indicates that the first pixel (x,y) belongs to the non-bright area of ​​the target image, and the pixel value of the first pixel (x,y) is adjusted to the second pixel value, i.e., 0. By traversing several first pixels in the target image, the adjusted pixel values ​​of each first pixel are obtained, thus obtaining the binarized image corresponding to the target image.

[0116] Step S82: Based on the pixel value distribution in the binarized image, obtain at least one connected component composed of at least one second pixel.

[0117] In this embodiment, at least one connected component composed of at least one second pixel is obtained based on the pixel value distribution in the binarized image. That is, at least one connected component composed of at least one second pixel is obtained using the position information of the second pixel.

[0118] Step S83: For each connected component, generate a bounding box containing the connected component and having a preset shape, as the candidate bounding box of the light source corresponding to the connected component.

[0119] In this embodiment, for each connected component, a bounding box containing the connected component and having a preset shape is generated as the candidate bounding box of the light source corresponding to the connected component. Since the connected component is composed of second pixels, and the second pixels correspond to the bright areas in the target image, the connected component composed of the second pixels also corresponds to the bright areas in the target image; and the bright areas in the target image are very likely to be light sources, so the generated bounding box containing the connected component and having a preset shape can be regarded as the candidate bounding box of the light source corresponding to the connected component.

[0120] The preset shape is not limited and can be set according to actual usage needs. For example, the preset shape is a rectangle. Furthermore, the size of the border is not limited; for example, the border is the smallest border containing connected components and of the preset shape.

[0121] Step S72: Based on at least one candidate bounding box of a light source, obtain the labeled bounding box corresponding to the light source.

[0122] In this embodiment, the labeled bounding box corresponding to the light source is obtained based on at least one candidate light source bounding box. In other words, the labeled bounding box corresponding to the light source is determined from at least one candidate light source bounding box, and the determined labeled bounding box of the light source is of high quality.

[0123] In one embodiment, if Figure 10 , Figure 11 As shown, Figure 10 yes Figure 7 The flowchart shown in step S72 is a schematic diagram of one embodiment. Figure 11 This is a schematic diagram of an embodiment of the candidate bounding box and second annotation key points of the light source provided in this application. The target image is annotated with the second annotation key points of the light source. The annotation bounding box corresponding to the light source can be determined by combining at least one candidate bounding box and the second annotation key points. At this time, obtaining the annotation bounding box corresponding to the light source based on at least one candidate bounding box specifically includes the following sub-steps:

[0124] Step S1001: For each candidate bounding box of a light source, obtain the number of second annotation key points contained in the candidate bounding box of the light source.

[0125] like Figure 11 As shown, since the second annotation key points of the light source on the target image are manually annotated, there must be a light source at the location marked with the second annotation key point, and there must be no light source at the location without the second annotation key point. Therefore, in this embodiment, for each candidate bounding box of the light source, the number of second annotation key points contained in the candidate bounding box of the light source is obtained, so as to select the annotation bounding box corresponding to the light source from the candidate bounding box in combination with the number of second annotation key points contained in the candidate bounding box of the light source, thereby improving the quality of the annotation bounding box of the light source.

[0126] Step S1002: In response to the light source candidate bounding box containing a second annotation key point, the light source candidate bounding box is determined as the annotation bounding box of the light source corresponding to the contained second annotation key point, and the position of the light source candidate bounding box is adjusted so that the contained second annotation key point is located at a preset position of the light source candidate bounding box.

[0127] In this embodiment, in response to a candidate light source bounding box containing a second annotation key point, the candidate light source bounding box is determined as the annotation bounding box of the light source corresponding to the contained second annotation key point, and the position of the candidate light source bounding box is adjusted so that the contained second annotation key point is located at a preset position within the candidate light source bounding box. Since the candidate light source bounding box contains a second annotation key point, it indicates that the region corresponding to the candidate light source bounding box includes the light source, and since the candidate light source bounding box contains only one second annotation key point, this candidate light source bounding box is the annotation bounding box corresponding to the contained second annotation key point. Furthermore, it is necessary to adjust the position of the candidate light source bounding box so that the contained second annotation key point is located at a preset position within the candidate light source bounding box, thereby making the annotation bounding box corresponding to the second annotation key point surround the entire light source corresponding to the second annotation key point, improving the quality of the annotation bounding box corresponding to the light source, that is, improving the quality of the annotated light source bounding box.

[0128] The preset positions are not limited and can be set according to actual needs. For example, since light source A appears as a point light source in the image when it is in operation, in order to reduce the difficulty of annotating key points of light source A, the optical center of light source A can be annotated as a key point to obtain the second annotation key point a of light source A. That is, the second annotation key point a of light source A is the optical center of light source A. At this time, in order to make the annotation bounding box B corresponding to the second annotation key point a enclose the entire light source A, the center of the annotation bounding box B needs to be adjusted to be located on the second annotation key point a of light source A, i.e., the optical center, thereby improving the quality of the annotated bounding box of light source A.

[0129] For example, such as Figure 11 As shown, the candidate bounding box A1 contains only the second annotation keypoint a, and the candidate bounding box B1 contains only the second annotation keypoint b. Therefore, candidate bounding box A1 is the annotation bounding box of the light source corresponding to the second annotation keypoint a, and candidate bounding box B1 is the annotation bounding box of the light source corresponding to the second annotation keypoint b. At this point, it is necessary to adjust the positions of candidate bounding boxes A1 and B1 so that the second annotation keypoint a is located at the center of candidate bounding box A1, and the second annotation keypoint b is located at the center of candidate bounding box B1, specifically as follows: Figure 12 As shown, Figure 12 This is a schematic diagram of an embodiment of the annotation boundary box of the light source provided in this application. Figure 11 After adjusting the positions of the candidate light source bounding boxes A1 and B1, the following results are obtained: Figure 12 The candidate bounding box A2 and candidate bounding box B2 shown are labeled bounding boxes.

[0130] Step S1003: In response to the light source candidate bounding box containing at least two second annotation key points, generate derived bounding boxes that each contain one second annotation key point in the light source candidate bounding box, so as to serve as the annotation bounding boxes of the light sources corresponding to each second annotation key point in the light source candidate bounding box, and adjust the position of each derived bounding box so that the contained second annotation key point is located at a preset position in the derived bounding box.

[0131] In this embodiment, in response to the light source candidate bounding box containing at least two second annotation key points, a derived bounding box containing one second annotation key point in the light source candidate bounding box is generated based on the light source candidate bounding box, so as to serve as the annotation bounding box of the light source corresponding to each second annotation key point in the light source candidate bounding box, and the position of each derived bounding box is adjusted so that the included second annotation key point is located at a preset position of the derived bounding box.

[0132] Since the candidate bounding box of a light source contains a second annotation keypoint, it indicates that the region corresponding to the candidate bounding box includes a light source. And since the candidate bounding box contains at least two second annotation keypoints, it indicates that there are at least two light sources at that location. Therefore, this candidate bounding box needs to be copied to generate derived bounding boxes. The sum of the number of derived bounding boxes and the candidate bounding box is equal to the number of second annotation keypoints contained in the candidate bounding box. This ensures that each light source contained in the candidate bounding box has a corresponding annotation bounding box, thus annotating the bounding boxes of each light source. Furthermore, the position of the annotation bounding box corresponding to each light source needs to be adjusted so that the contained second annotation keypoints are located at preset positions on the annotation bounding box. This ensures that the annotation bounding box corresponding to the second annotation keypoint surrounds the entire light source corresponding to that second annotation keypoint, improving the quality of the annotation bounding box for that light source; that is, improving the quality of the annotated light source bounding boxes.

[0133] Step S1004: In response to the light source candidate bounding box not containing a second annotation key point, remove the light source candidate bounding box.

[0134] In this embodiment, the candidate bounding box of a light source is discarded in response to the fact that it does not contain the second annotation key point. Since the second annotation key point of the light source on the target image is manually annotated, there must be no light source at the location where the second annotation key point is not marked. Therefore, when it is determined that the candidate bounding box of a light source does not contain the second annotation key point, this candidate bounding box of a light source is discarded.

[0135] For example, such as Figure 11 As shown, if the candidate bounding box C1 of the light source does not contain the second annotation key point, it means that there is no light source at the candidate bounding box C1 of the light source, and the candidate bounding box C1 of the light source is removed.

[0136] Please see Figure 13 , Figure 13 This is a flowchart illustrating an embodiment of the training method for the region detection model provided in this application. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily reflect that outcome. Figure 13 The process sequence shown is limited. Figure 13 As shown, this embodiment includes:

[0137] Step S1301: Obtain sample images.

[0138] The method described in this implementation is used to train a region detection model based on sample images containing light sources in operation. This paper does not limit the number or size of the sample images, which can be set according to actual needs. The sample images containing light sources in operation are annotated with the ground truth bounding boxes of the light sources. These ground truth bounding boxes are obtained using the light source bounding box annotation method provided in this application. The light source bounding box annotation method provided in this application produces high-quality bounding boxes, enabling the region detection model trained on the sample images to accurately detect and identify the bounding boxes of the light sources.

[0139] In one embodiment, the sample images can be obtained from local storage or cloud storage. It is understood that in other embodiments, sample images showing the sample in operation can also be acquired in real time using an image acquisition device.

[0140] In one specific implementation, the sample dataset contains 15,798 images, which are divided into a training set, a validation set, and a test set in an 8:1:1 ratio. Images in the training set are used as sample images. The batch size is set to 16, the initial learning rate to 0.01, and the maximum number of iterations to 1000, ensuring that the resulting region detection model has high detection quality and is used as the training result.

[0141] Step S1302: Use the region detection model to detect the sample image and obtain the predicted bounding box of the light source.

[0142] In this embodiment, a region detection model is used to detect sample images and obtain the predicted bounding box of the light source, so that the network parameters of the region detection model can be adjusted according to the difference between the actual bounding box and the predicted bounding box of the light source.

[0143] In one embodiment, if Figure 14 As shown, Figure 14This is a schematic diagram of an embodiment of the network structure of the region detection model provided in this application. The region detection model mainly consists of a backbone feature extraction network, Feature Pyramid Networks (FPNs), and a head structure. The backbone feature extraction network is composed of residual structures and outputs four feature res of different sizes. i , i∈[2,5]. The FPN upsamples features from different depths in the backbone from deep to shallow layers to ensure consistent feature sizes. Then, a 1×1 convolutional layer is used to change the number of channels, and the resized features are summed. Finally, a convolution is performed to fuse features from different depths, outputting the fused feature p. i , i∈[2,5]; for res i After two more 3×3 convolutions, p6 and p7 are output. The head structure receives fused features p from the FPN. i The data are input into the classification network and the regression network, which are composed of 4 layers of 3×3 convolutions. Since the number of categories is 1, the classification network outputs the feature of the number of categories through one convolution operation, and the regression network outputs the feature of 4 channels through convolution to represent the location information of the predicted bounding box.

[0144] Step S1303: Adjust the network parameters of the region detection model based on the difference between the real bounding box and the predicted bounding box.

[0145] In this embodiment, the network parameters of the region detection model are adjusted based on the difference between the ground truth bounding box and the predicted bounding box. The network parameters of the region detection model are adjusted by comparing the ground truth bounding box of the light source labeled in the sample image with the predicted bounding box detected by the region detection model, until the region detection model converges. This allows the trained region detection model to have good detection and recognition capabilities for light source bounding boxes and a certain degree of generalization ability.

[0146] In one embodiment, the loss of the region detection model can be determined by the difference between the true bounding box of the light source labeled in the sample image and the predicted bounding box of the light source detected by the region detection model. The network parameters of the region detection model are then adjusted based on the loss of the region detection model using optimization methods such as gradient descent. The above process is repeated until the region detection model is trained to convergence using the sample images.

[0147] In one specific implementation, the network parameters of the region detection model can be adjusted based on the loss of the region detection model using a target loss function. The target loss function includes the classification L... qfl L of loss and regression giou Loss, the loss of the entire network L total =L qfl +wL giouWhere w is the balanced classification loss L qfl and regression loss L giou The weights are denoted by L. Wherein, the classification loss L... qfl and regression loss L giou The specific formula is as follows:

[0148] L qfl =-α t |y-σ|[(1-y)log(1-σ)+ylog(σ)]

[0149]

[0150] Among them, L qfl σ represents the true bounding box of the light source; α represents the predicted bounding box of the light source; t Indicates balancing positive and negative samples, α t =yα+(1-y)(1-α); A i A represents the area of ​​the intersection region between the true bounding box and the predicted bounding box; u This represents the sum of the predicted bounding box and the true bounding box, minus A. i A c This represents the area of ​​the smallest rectangle that encloses both the true bounding box and the predicted bounding box.

[0151] Please see Figure 15 , Figure 15 This application provides a method for detecting light sources by using a convergent region detection model trained on an image. In one embodiment, the light source is a vehicle headlight. By using the convergent region detection model to detect the image, the bounding box of the headlight can be obtained. Since the region detection model is trained on sample images with labeled ground truth bounding boxes of the light source, and the labeled ground truth bounding boxes of the light source on the sample images are labeled using the light source bounding box labeling method provided in this application, the labeled light source bounding boxes are of high quality. Therefore, the region detection model trained on the sample images can accurately detect and identify the bounding box of the light source. Figure 15 The displayed method demonstrates high accuracy in detecting and identifying the bounding boxes of vehicle lights using a region detection model. In one specific implementation, the confidence level and category of the detected bounding boxes can also be obtained.

[0152] Please see Figure 16 , Figure 16This is a schematic diagram of an embodiment of the electronic device provided in this application. The electronic device 160 includes a memory 1601 and a processor 1602 coupled to each other. The processor 1602 executes program instructions stored in the memory 1601 to implement the steps of any of the above-described embodiments of the region detection model training method or the light source bounding box annotation method. In a specific implementation scenario, the electronic device 160 may include, but is not limited to, a microcomputer or a server. Furthermore, the electronic device 160 may also include mobile devices such as laptops and tablets, without limitation.

[0153] Specifically, processor 1602 controls itself and memory 1601 to implement the steps of any of the above-described region detection model training methods or light source bounding box annotation methods. Processor 1602 can also be referred to as a CPU (Central Processing Unit). Processor 1602 may be an integrated circuit chip with signal processing capabilities. Processor 1602 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 1602 can be implemented using integrated circuit chips.

[0154] Please see Figure 17 , Figure 17This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 170 of this application embodiment stores program instructions 1701. When executed, these program instructions 1701 implement any embodiment of the training method for the region detection model or the annotation method for the light source bounding box, as well as any non-conflicting combination thereof. The program instructions 1701 can be formed into a program file and stored in the aforementioned computer-readable storage medium 170 in the form of a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) can execute all or part of the steps of the methods of various embodiments of this application. The aforementioned computer-readable storage medium 170 includes various media capable of storing program code, such as a USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or terminal devices such as computers, servers, mobile phones, and tablets.

[0155] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0156] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for annotating the bounding box of a light source, characterized in that, The method includes: Acquire a target image; wherein the target image is obtained by acquiring the light source in a working state; A first pixel threshold is determined for each of the first pixels in the target image; wherein, the first pixel threshold of the first pixel is obtained by statistically analyzing the pixel values ​​of a reference pixel of the first pixel using a threshold influence parameter, and the reference pixel is a pixel in the target image that has a specific positional relationship with the first pixel; By comparing the pixel value of each first pixel with the corresponding first pixel threshold, second pixels belonging to the bright area of ​​the target image are selected from the plurality of first pixels; Using the second pixel, determine the bounding box corresponding to the light source in the target image; The step of selecting second pixels belonging to the bright region of the target image from the plurality of first pixels by comparing the pixel value of each first pixel with the corresponding first pixel threshold includes: From the plurality of first pixels, select the first pixel whose pixel value is greater than the first pixel threshold, and use it as the second pixel; The step of using the second pixel to determine the bounding box corresponding to the light source in the target image includes: Using the second pixel, at least one candidate bounding box for a light source in the target image is generated; Based on the at least one candidate bounding box of the light source, the labeled bounding box corresponding to the light source is obtained.

2. The method according to claim 1, characterized in that, The threshold influence parameter includes at least one of a positional parameter used to determine the specific positional relationship and a statistical parameter used to influence the statistical results of the pixel value of the reference pixel.

3. The method according to claim 2, characterized in that, The threshold influence parameters include the positional parameters and the statistical parameters; determining the corresponding first pixel threshold for a plurality of first pixels in the target image includes: For each first pixel, a plurality of reference pixels that satisfy the specific positional relationship with the first pixel are determined based on the position parameters; Obtain the central tendency characterization value of the pixel values ​​of each of the reference pixels of the first pixel; The first pixel threshold of the first pixel is obtained by using the central tendency characterization value, the pixel difference between the central tendency characterization value and the pixel value of the first pixel, and the statistical parameters.

4. The method according to claim 3, characterized in that, The position parameter is the window width; determining multiple reference pixels that satisfy the specific positional relationship with the first pixel based on the position parameter includes: Obtain the pixel in the target image within a window centered on the first pixel and with the position parameter as its width, and use it as the reference pixel for the first pixel; The step of obtaining the central tendency characterization value of the pixel values ​​of each of the reference pixels of the first pixel includes: The average pixel value of each of the reference pixels of the first pixel is obtained as the central tendency characterization value; The step of obtaining the first pixel threshold of the first pixel using the central tendency characterization value, the pixel difference between the central tendency characterization value and the pixel value of the first pixel, and the statistical parameters includes: Obtain the ratio between the pixel difference and the corresponding first difference, and multiply the second difference between the first constant and the ratio with the statistical parameter to obtain the multiplication result; wherein, the first difference is the difference between the second constant and the pixel difference; The sum of the product of the third constant and the product of the product is multiplied by the central tendency value to obtain the first pixel threshold of the first pixel.

5. The method according to claim 1, characterized in that, The threshold influence parameter is obtained by analyzing several images, all of which are acquired from light sources in operation. The images may or may not include the target image. The threshold influence parameter is determined based on the bounding box quality parameters of the light sources in the images.

6. The method according to claim 5, characterized in that, Before determining the corresponding first pixel threshold for each of the first pixels in the target image, the method further includes: Obtain the value range corresponding to each threshold influence parameter, and determine the initial current value of each threshold influence parameter within the value range; Using the current value of the threshold influence parameter, obtain the current bounding box corresponding to the light source in each of the plurality of images; The quality of the current bounding box corresponding to the light source in each of the plurality of images is evaluated to obtain the bounding box quality parameter corresponding to the current threshold influence parameter. In response to the failure to meet the preset requirements, based on the bounding box quality parameter currently corresponding to the threshold influence parameter, the threshold influence parameter is revalued within the specified range to obtain a new current value for the threshold influence parameter; and the process of using the current value of the threshold influence parameter to obtain the current bounding box corresponding to the light source in each of the plurality of images and subsequent steps is re-executed. The preset requirements are that the bounding box quality parameter currently corresponding to the threshold influence parameter meets the requirements and / or the number of times the process of using the current value of the threshold influence parameter to obtain the current bounding box corresponding to the light source in each of the plurality of images has met the required number of times. The final current value of the threshold influence parameter is used to determine the first pixel threshold.

7. The method according to claim 6, characterized in that, The threshold influence parameter includes at least one of a position parameter used to determine the specific positional relationship and a statistical parameter used to influence the statistical results of the pixel value of the reference pixel. And / or, the step of obtaining the current bounding box corresponding to the light source in each of the plurality of images by utilizing the current value of the threshold influence parameter includes: For each of the plurality of images, a corresponding second pixel threshold is determined for a plurality of third pixels in the image. The second pixel threshold of the third pixel is obtained by statistically analyzing the pixel values ​​of the reference pixels of the third pixel using the current value of the threshold influence parameter. The reference pixels of the third pixel are pixels in the image that have a specific positional relationship with the third pixel. By comparing the pixel value of each third pixel with the corresponding second pixel threshold, a fourth pixel belonging to the bright area of ​​the image is selected from the plurality of third pixels; Using the fourth pixel, the current bounding box corresponding to the light source in the image is generated.

8. The method according to claim 6, characterized in that, Each of the images is labeled with a first annotation key point of the light source. The quality assessment of the current bounding box corresponding to the light source in each of the images, to obtain the bounding box quality parameter corresponding to the current threshold influence parameter, includes: The current bounding boxes and the first labeled key points of the plurality of images are statistically analyzed to obtain a first total number of the current bounding boxes contained in the plurality of images, a second total number of the first labeled key points contained in the plurality of images, a number of key points of the first labeled key points contained in each current bounding box of the plurality of images, and a number of bounding boxes corresponding to each first labeled key point of the plurality of images. The number of bounding boxes corresponding to the first labeled key points is the number of current bounding boxes containing the first labeled key points. The number of key points corresponding to each valid bounding box and the number of bounding boxes corresponding to each valid key point are counted respectively, and the corresponding point count results and box count results are obtained. The valid bounding box is the current bounding box that contains a first labeled key point greater than or equal to 1, and the valid key point is the first labeled key point contained in the current bounding box that contains a first labeled key point greater than or equal to 1. Based on the point count statistics, the box count statistics, the first total number, and the second total number, the bounding box quality parameter corresponding to the threshold influence parameter is obtained.

9. The method according to claim 8, characterized in that, The steps involve separately counting the number of keypoints corresponding to each valid bounding box and the number of bounding boxes corresponding to each valid keypoint, to obtain the corresponding point count statistics and bounding box count statistics, including: The reciprocal of the number of key points corresponding to each of the effective bounding boxes is summed to obtain the point count result, and the reciprocal of the number of bounding boxes corresponding to each of the effective key points is summed to obtain the box count result. The process of obtaining the bounding box quality parameter corresponding to the threshold influence parameter based on the point count statistics, the bounding box count statistics, the first total quantity, and the second total quantity includes: Multiply the reciprocal of the product of the first total quantity and the second total quantity, the point quantity statistics, and the box quantity statistics to obtain the bounding box quality parameter corresponding to the threshold influence parameter.

10. The method according to claim 1, characterized in that, The step of generating at least one candidate bounding box for a light source in the target image using the second pixel includes: Obtain the binarized image corresponding to the target image; wherein, in the binarized image, the pixel value corresponding to the second pixel is the first pixel value, and the pixel values ​​of other pixels besides the second pixel are the second pixel values; Based on the pixel value distribution in the binarized image, at least one connected component composed of the second pixel is obtained; For each of the connected components, a bounding box containing the connected component and having a preset shape is generated as the candidate bounding box of the light source corresponding to the connected component; And / or, the target image is labeled with second annotation key points of the light source, and the step of obtaining the annotation bounding box corresponding to the light source based on the at least one candidate bounding box of the light source includes: For each of the candidate light source bounding boxes, obtain the number of the second annotation key points contained in the candidate light source bounding box; In response to the light source candidate bounding box containing a second annotation key point, the light source candidate bounding box is determined as the annotation bounding box of the light source corresponding to the included second annotation key point, and the position of the light source candidate bounding box is adjusted so that the included second annotation key point is located at a preset position of the light source candidate bounding box; In response to the light source candidate bounding box containing at least two second annotation key points, a derived bounding box containing one of the second annotation key points in the light source candidate bounding box is generated based on the light source candidate bounding box, so as to serve as the annotation bounding box of the light source corresponding to each second annotation key point in the light source candidate bounding box, and the position of each derived bounding box is adjusted so that the included second annotation key point is located at a preset position in the derived bounding box; In response to the fact that the candidate bounding box of the light source does not contain the second labeled key point, the candidate bounding box of the light source is removed.

11. The method according to claim 1, characterized in that, The target image was acquired at night; And / or, the light source is a vehicle headlight; And / or, before determining the corresponding first pixel thresholds for a plurality of first pixels in the target image, the method further includes any one or more of the following steps: Perform grayscale processing on the target image; The target image is preprocessed; wherein the preprocessing includes at least one of linear normalization and Gaussian blur.

12. A training method for a region detection model, characterized in that, The method includes: Acquire a sample image; wherein the sample image is acquired by collecting data from the light source in a working state, and the sample image is marked with the true bounding box of the light source, which is marked using the annotation method for the light source bounding box according to any one of claims 1-11; The sample image is detected using a region detection model to obtain the predicted bounding box of the light source; The network parameters of the region detection model are adjusted based on the difference between the true bounding box and the predicted bounding box.

13. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing program instructions, and the processor executing the program instructions to implement the training method of the region detection model as described in claim 12 or the annotation method of the light source bounding box as described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions that can be executed to implement the training method of the region detection model as described in claim 12 or the annotation method of the light source bounding box as described in any one of claims 1-11.

Images