Method, device and equipment for detecting target in image and medium

Abstract

The embodiment of the invention discloses a method for detecting a target in an image. The method comprises the following steps: training a preparation model by using a sample data set; model optimization operation is executed circularly; the first model optimization operation comprises the steps of inputting a sample image into a preparation model, judging whether an output result meets a standardization requirement or not, screening the results meeting the standardization requirement by utilizing a physical rule matched with a target object, merging the screened results and existing training data into new training data, and training a new model; starting from the second model optimization operation, each model optimization operation comprises the steps of inputting a sample image into a new model, judging whether an output result meets a standardization requirement or not, screening the results meeting the standardization requirement by utilizing a physical rule matched with a target object, merging the screened results and existing training data into new training data, and outputting the new training data; training a new model; and performing target object detection operation on the to-be-detected image by using the latest model to obtain a target object detection result.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and medium for detecting a target in an image. Background Art

[0002] With the rapid development of deep learning and remote sensing technology, remote sensing images are increasingly used in geographic information acquisition and target positioning. Optical remote sensing images can provide rich information for the identification and positioning of ground objects. However, traditional image processing methods have limitations in terms of automation and positioning accuracy.

[0003] At present, the target detection and instance segmentation tasks in remote sensing images mainly rely on deep learning models, such as YOLO, which can detect and segment targets in the scene. For example, in order to improve the positioning accuracy and processing efficiency of remote sensing images, the deep learning model YOLOv11 is combined with optical remote sensing images, and the efficient target detection and instance segmentation capabilities of YOLOv11 are used to accurately locate the target. This combination can not only automatically identify the target in the image, but also significantly improve the automation and accuracy of positioning to meet the needs of actual applications.

[0004] However, when faced with small targets in remote sensing images (such as street lamps), existing technologies face certain challenges: (1) The model relies on a large amount of high-quality annotated data, but the annotation cost of small targets in the remote sensing field is high and data acquisition is difficult; (2) For small targets (such as street lamps), existing models are prone to missed detection or false detection in complex backgrounds or low-quality images; (3) Existing technologies usually only rely on pixel-level convolutional features, ignoring the physical relationship between the target and the environment (such as lamp poles and shadows); (4) The inference results of the model are usually directly used as output, lacking further screening and optimization based on physical laws or other prior knowledge.

[0005] The shortcomings of existing technologies in small target detection and instance segmentation are mainly concentrated in the weak ability to utilize the physical characteristics of the target and the lack of targeted optimization solutions for remote sensing images. Especially in the task of street light detection, the geometric physical laws such as the directionality and angle of the light pole and its shadow are not fully utilized, resulting in low reasoning accuracy, and there is an urgent need for technical solutions to improve the detection effect. Summary of the invention

[0006] The embodiments of the present specification provide a method, device, equipment and medium for detecting a target in an image, so as to solve the technical problem of how to improve the detection effect of a target in an image.

[0007] To solve the above technical problems, the embodiments of this specification provide the following technical solutions: The present invention provides a method for detecting an object in an image, the method comprising: Obtaining a sample data set, and using the sample data set to train a preliminary model; Execute model optimization operations in a loop; The first model optimization operation includes: obtaining a sample image, inputting the sample image into the preparatory model, and obtaining a result output by the preparatory model; judging whether the result meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and training the preparatory model using the new training data to obtain an optimized new model; Starting from the second model optimization operation, each model optimization operation includes: obtaining a sample image, inputting the sample image into a new model, and obtaining a result output by the new model; judging whether the result meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and training the new model using the new training data to obtain an optimized new model; After the end condition of the model optimization operation is reached, the target object detection operation is performed on the image to be detected using the latest model to obtain the target object detection result.

[0008] Optionally, obtaining a sample data set includes: Acquire a target area image, and mark the target object in the target area image to form shape feature data of the target object; Generate small slices of the annotation data, adjust the fixed line width and generate the annotation file; The annotation file is converted into a sample data set in a specific format.

[0009] Optionally, determining whether the result meets the standardization requirements includes: Each polygon is extracted from the result, and it is determined whether each polygon satisfies the standardization condition.

[0010] Optionally, judging whether each polygon satisfies the standardization condition includes: For any polygon, perform the standardization judgment operation on the polygon in a loop; Among them, the first standardized judgment operation includes: Extracting the center line of the polygon according to the pixel coordinates of the polygon, and determining whether the center line meets a preset condition; If yes, the center line is saved and the standardization judgment operation is no longer performed on the polygon; If not, the next standardization judgment operation is performed; The next standardization judgment operation includes: Simplify the polygon, extract the center line according to the pixel coordinates of the simplified polygon, and determine whether the newly extracted center line meets the preset conditions; If yes, the center line is saved and the standardization judgment operation is no longer performed; If not, the next standardization judgment operation is performed; If a center line that meets the preset conditions is obtained within a preset number of standardization judgment operations, the polygon meets the standardization conditions; if a center line that meets the preset conditions is not obtained within a preset number of standardization judgment operations, the polygon does not meet the standardization conditions.

[0011] Optionally, if the number of vertices of the center line is a first preset value, the center line satisfies the preset condition.

[0012] Optionally, using the physical rules matched by the target object to filter the results that meet the standardization requirements includes: Determine the screening factors based on the physical rules matched by the target object; The results that meet the standardization requirements are grouped according to the screening factors, and the groups whose number of results does not meet the preset number are eliminated.

[0013] Optionally, the screening factors include the size and / or direction of the angle between the target object and its own shadow.

[0014] The present invention provides a device for detecting an object in an image, the device comprising: An initial training module, used to obtain a sample data set and train a preliminary model using the sample data set; A loop optimization module, used to execute model optimization operations in a loop; The first model optimization operation includes: obtaining a sample image, inputting the sample image into the preparatory model, and obtaining a result output by the preparatory model; judging whether the result meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and training the preparatory model using the new training data to obtain an optimized new model; Starting from the second model optimization operation, each model optimization operation includes: obtaining a sample image, inputting the sample image into a new model, and obtaining a result output by the new model; judging whether the result meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and training the new model using the new training data to obtain an optimized new model; The object detection module is used to perform a target object detection operation on the image to be detected using the latest model after the end condition of the model optimization operation is met to obtain a target object detection result.

[0015] The embodiment of this specification provides a device for detecting an object in an image, including: at least one processor; as well as, a memory communicatively coupled to the at least one processor; in, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the above-mentioned method for detecting objects in images.

[0016] An embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the above-mentioned method for detecting objects in an image is implemented.

[0017] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects: By performing model optimization operations in a loop, the model can be gradually optimized to improve the detection effect of the model.

[0018] Each time in the model optimization operation, the output results of the model are standardized and judged, and the results that meet the standardization requirements are screened using the physical rules matched by the target object. The erroneous results or results that do not meet the requirements that may occur during the model optimization process are accurately identified and eliminated, effectively improving the model optimization efficiency, greatly improving the detection accuracy and reliability of the optimized model, and thus improving the detection effect of the target in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings required for use in the embodiments of this specification or the prior art description are briefly described below. Obviously, the following only describes the drawings required for use in some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0020] Figure 1 It is a flowchart of a method for detecting an object in an image provided by the first embodiment of this specification; Figure 2 This is a technical route diagram of the model optimization example in the first embodiment of this specification; Figure 3It is a schematic diagram of the three-point polyline marking mode in the first embodiment of this specification; Figure 4 This is a schematic diagram of the centerline extraction process in the first embodiment of this specification; Figure 5 is a schematic diagram of polygon pixel coordinates in the first embodiment of this specification; Figure 6 is a schematic diagram of the center line of the first embodiment of this specification; Figure 7 is a schematic diagram of a reconstructed polygon in the first embodiment of this specification; Figure 8 It is a schematic diagram of the structure of the device for detecting objects in images provided in the second embodiment of this specification. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments involved in the specific implementation methods are only part of the embodiments of this application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the specific implementation methods without making creative work should belong to the scope of protection of this application.

[0022] The first embodiment of this specification (hereinafter referred to as "embodiment one") provides a method for detecting targets in images. The execution subject of embodiment one includes but is not limited to a terminal or a server or an operating system or an application program, that is, the execution subject can be various and can be set, used or changed as needed.

[0023] like Figure 1 As shown, the method for detecting targets in an image provided by the first embodiment includes: S101: Obtain a sample data set, and use the sample data set to train a preliminary model; In the first embodiment, a sample data set may be obtained, and a preliminary model may be trained using the sample data set.

[0024] The specific method for obtaining the sample data set is not limited in the first embodiment. For example, obtaining the sample data set may include: Acquire a target area image, and mark the target object in the target area image to form shape feature data of the target object; Generate small slices of the annotation data, adjust the fixed line width and generate the annotation file; The annotation file is converted into a sample data set in a specific format.

[0025] The following is a specific example of obtaining a sample data set (the first embodiment is not limited to the following example): 1. Obtain the required target area image. For example, use the Bigemap platform and 1:5000 standard framing to download high-resolution remote sensing images with a resolution of Level_19 within the airport area (i.e. the target area can be the airport area), and the image projection is unified into the WGS84 / EPSG: 4326 coordinate system.

[0026] 2. Label the target object in the image. For example, if the target object is a street lamp, you can use the LabelMe tool to label the target in the image. The labeled target is the airport street lamp and its shadow. The labeling mode uses the three-point polyline mode, that is, labeling the top of the lamp pole, the root of the lamp pole and the end point of the shadow to form accurate shape feature data, such as Figure 3 shown.

[0027] 3. After the annotation is completed, the image can be converted into a sample data set in a specific format (for example, YOLO format). Specifically, the annotation data (for example, the target object is a street lamp, the annotation data is a broken line formed by the three top lines of the street lamp top, the street lamp base and the top of the shadow) can be sliced, the fixed line width can be adjusted (for example, the target object is a street lamp, after determining the fixed line width, a polygonal pixel coordinate of the street lamp and its shadow can be obtained) and the corresponding JSON file can be generated as the annotation file, and then the annotation file can be converted into a sample data set in YOLO format. The sample data set includes information such as bounding boxes and category labels.

[0028] In the first embodiment, after obtaining the sample data set, the sample data set can be used to train a preliminary model. The following takes the YOLOV11-seg (You Only Look Once version 11-Segmentation) model (this model is an extension of the YOLO series model and can be used for instance segmentation tasks) as an example to illustrate how to train the preliminary model.

[0029] (1) Environment construction: Complete the construction of the training framework in a GPU-supported environment, including upgrading the GPU driver, installing and configuring CUDA and cuDNN; using Conda to create a virtual environment and install PyTorch; and deploying the YOLO training framework.

[0030] (2) Dataset division: Divide the sample data set into training set, validation set, and test set in proportion to ensure reasonable data distribution. The division ratio of the training set, validation set, and test set can be set as needed, such as 8:1:1 or 7:2:1. In particular, the division ratio can be determined according to the size of the sample data set to ensure reasonable data distribution and improve the model training effect.

[0031] (3) Model training: Use the YOLOV11-seg model for training, load the formatted training set for segmentation task training, use the validation set for verification, and use the test set for testing. After reaching the predetermined training goal, save the training weight file best.pt to obtain the trained model, that is, the preliminary model, for use in the subsequent model inference stage.

[0032] S103: looping and executing the model optimization operation; In the first embodiment, the model optimization operation can be executed repeatedly. The first model optimization operation may include: obtaining a sample image, inputting the sample image into the preparatory model, obtaining the output result of the preparatory model; judging whether each result (i.e., the model output result) meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and using the new training data to train the preparatory model to obtain an optimized new model; Starting from the second model optimization operation, each model optimization operation may include: obtaining sample images, inputting the sample images into a new model, and obtaining the output results of the new model; judging whether each result (i.e., the model output result) meets the standardization requirements, using the physical rules matched by the target object to screen the results that meet the standardization requirements, merging the screened results with the existing training data into new training data, and using the new training data to train the new model to obtain an optimized new model.

[0033] Embodiment 1 In each model optimization operation, determining whether the model output result meets the standardization requirement may include: extracting each polygon from each model output result, and determining whether each polygon meets the standardization condition.

[0034] Wherein, judging whether each polygon satisfies the standardization condition may include: S1031: for any polygon, perform a standardization judgment operation on the polygon in a loop; The first standardization judgment operation may include: Extracting the center line of the polygon according to the pixel coordinates of the polygon, and determining whether the center line meets a preset condition; If yes, the center line is saved and the standardization judgment operation is no longer performed on the polygon; If not, the next standardization judgment operation is performed; The next standardization judgment operation may include: Simplify the polygon, extract the center line according to the pixel coordinates of the simplified polygon, and determine whether the newly extracted center line meets the preset conditions; If yes, the center line is saved and the standardization judgment operation is no longer performed; If not, the next standardization judgment operation is performed.

[0035] S1033: If a center line that meets the preset conditions is obtained within a preset number of standardization judgment operations, the polygon meets the standardization conditions; if a center line that meets the preset conditions is not obtained within a preset number of standardization judgment operations, the polygon does not meet the standardization conditions.

[0036] The above preset conditions can be set as needed. For example, if the number of vertices of the center line is a preset value, it is determined that the center line meets the standardization condition.

[0037] For any model output result, if the polygons extracted from the model output result meet the standardization conditions, then the model output result meets the standardization requirements.

[0038] In each model optimization operation of Example 1, using the physical rules matched by the target object to screen the results that meet the standardization requirements may include: determining the screening factors according to the physical rules matched by the target object; grouping the results that meet the standardization requirements according to the screening factors, and eliminating the groups whose number of results does not meet the preset number.

[0039] The specific content of the above-mentioned screening factors is not limited. For example, the screening factors include the size and / or direction of the angle between the target object and its own shadow.

[0040] Combine the following Figure 2 , give specific examples of each model optimization operation (embodiment 1 is not limited to the following examples): First model optimization run In this example, the first model optimization operation can be performed as follows: 1. Automatically download sample images. For example, use the airport geographic longitude and latitude coordinates to generate the longitude and latitude coordinates of the airport's border, use a script to automatically download Google images with a resolution of Level_19, and convert the image projection to the WGS84 / EPSG: 4326 coordinate system, and use the resulting image as a sample image.

[0041] 2. After obtaining the sample image, you can perform preprocessing operations on the sample image. For example, for images of very large areas, you can crop the large image into small tiles suitable for YOLO input, or combine it with the SAHI (Slicing Aided Hyper Inference) tool to achieve efficient inference.

[0042] 3. Input the sample image (or preprocessed image) into the preparatory model, and let the model perform inference and output the corresponding results. For example, input the image into the aforementioned best.pt model for inference, that is, perform instance segmentation on the image and output the corresponding results. In this example, the role of the model is to detect the target object, and the output results can include a txt file of the detection label (the content of the txt file can include the label type, normalized polygon coordinates, etc.) and the corresponding png file.

[0043] 4. Determine whether the results output by the preliminary model meet the standardization requirements, including extracting polygons from each output result (specifically extracting normalized polygon coordinates from a txt file) and determining whether each extracted polygon meets the standardization conditions.

[0044] refer to Figure 4 For any polygon, judging whether the polygon meets the standardization condition may include: Performing standardized judgment operation on the polygon repeatedly; The first standardized judgment operation is as follows: (1) Coordinate conversion: For any polygon, convert its normalized coordinates into pixel coordinates, for example Figure 5 shown.

[0045] (2) Centerline coordinate extraction: The centerline is extracted based on the polygon pixel coordinates of the polygon, for example Figure 6 shown.

[0046] (3) Determine whether the center line meets the preset conditions, specifically, determine whether the number of vertices of the center line is a first preset value. Depending on the target object, the preset condition or the first preset value may be different. For example, if the target object is a street lamp, and the three-point polyline mode is used to mark the street lamp, the preset condition can be that the number of vertices is 3, and the first preset value is 3. That is, if the number of vertices of the center line is 3, the center line meets the preset conditions; if the number of vertices of the center line is not 3, the center line does not meet the preset conditions. When the target object changes, the marking method or the number of vertices marked may be different, and the preset condition or the first preset value may be different. Usually, the preset condition or the first preset value corresponds to the marking method or the number of vertices marked for the target object.

[0047] (4) If the centerline meets the preset conditions, the centerline is saved and no standardization judgment operation is performed on the polygon.

[0048] If the center line does not meet the preset conditions, the next standardization judgment operation is performed.

[0049] The next standardization judgment operation is as follows: (1) Simplifying the polygon (for example, using the RDP algorithm or other suitable methods), specifically simplifying the pixel coordinates of the polygon, and then extracting the center line according to the pixel coordinates of the simplified polygon, and determining whether the newly extracted center line meets the above-mentioned preset conditions.

[0050] (2) If the newly extracted centerline meets the preset conditions, the centerline is saved and no standardization judgment operation is performed on the polygon.

[0051] If the newly extracted center line does not meet the preset condition, the next standardization judgment operation is continued, including continuing to simplify the polygon, extracting the center line, and judging whether the newly extracted center line meets the preset condition.

[0052] Of course, the standardization judgment operation is not infinite, and the standardization judgment operation can be set to be performed at most n times. If a center line that meets the preset conditions is obtained in a certain standardization judgment operation within the n standardization judgment operations, it can be determined that the polygon meets the standardization conditions. If a center line that meets the preset conditions is not obtained after n standardization judgment operations, it is determined that the polygon does not meet the preset conditions and can be discarded.

[0053] For each polygon extracted from the output results of each model, it can be judged as above whether it meets the standardization conditions. Since each polygon is the result of the preliminary model detecting the target object, the polygon that meets the standardization conditions is the polygon that matches the shape characteristics of the target object. In the above example, the target object is a street lamp. If the center line of the polygon has three vertices, it means that the shape characteristics of the polygon match the annotation mode and shape characteristics of the street lamp. Therefore, the polygon meets the shape characteristic standards that the target object should have and meets the standardization conditions. The target object may be contained in the polygon (whether the target object is actually contained in the polygon is determined by the following physical rules screening).

[0054] For any result output by the preparation model, if the polygon extracted from the result meets the standardization condition, the result can be considered to meet the standardization requirement.

[0055] 5. For the model output results that meet the standardization requirements, the physical rules matched by the target object can be used to screen the model output results that meet the standardization requirements. Among them, the physical rules can be selected as needed, including various geometric characteristic rules. For example, when the image shooting time is unified, in the same image, the size and direction of the angle formed by all target objects and shadows are consistent, then the physical rule matched by the target object can be the angle consistency rule, and the angle consistency rule includes the consistency of the size and direction of the angle.

[0056] The following uses the angle consistency rule as an example to illustrate how to use the physical rules matched by the target object to screen the results that meet the standardization requirements.

[0057] The screening factor is determined according to the physical rule matched by the target object. If the physical rule is the angle consistency rule, the screening factor may include the size and / or direction of the angle between the target object and its own shadow. If the physical rule is other rules, the screening factor may be determined accordingly.

[0058] The model output results that meet the standardization requirements are grouped according to the screening factors, and the groups whose number of results does not meet or is less than the preset number are eliminated. The specific process may include: through the aforementioned standardization judgment process, the polygonal center lines of the output results of each model that meet the standardization requirements have been obtained, and the obtained "polygonal center lines of the output results of each model that meet the standardization requirements" (hereinafter referred to as the center lines to be grouped) can be screened according to angle and direction.

[0059] Angle screening The angle between the two broken lines formed by the three vertices of each center line to be grouped can be used as the angle of the center line to be grouped, so that the angle of each center line to be grouped can be determined.

[0060] It is possible to determine the first groups that do not overlap with each other, and each first group covers a certain angle range. For example, the average value A of the angles of the center lines of the above polygons is calculated, and the span value a (variable), the minimum value A-pa (the minimum value is generally not greater than the minimum value of the angle of each center line to be grouped), and the maximum value A+qa (the maximum value is generally greater than the maximum value of the direction of each center line to be grouped) are determined. Then, the first groups are [A-pa, A-(p-2)a), [(p-2)a, (p-4)a), ..., [Aa, A+a), [A+a, A+2a), ..., [A+(q-4)a, A+(q-2)a), [A+(q-2)a, A+qa]. According to the angle of each center line to be grouped, each center line to be grouped is divided into the corresponding first group, so that each center line to be grouped is grouped according to the angle.

[0061] The number of center lines actually included in each first group is determined, and then the first groups containing less than the first preset number of center lines are eliminated, and other first groups are retained, thereby achieving angle screening. The center lines retained after the angle screening are used as the center lines to be grouped, and the following direction screening is performed.

[0062] Direction screening The direction of the angle bisector of the angle between the two broken lines formed by the three vertices of each center line to be grouped can be used as the direction of the center line to be grouped. A unified coordinate system is established for each center line to be grouped, and the direction of each center line to be grouped corresponds to a vector. The direction of each center line to be grouped can be assigned a value, for example, the slope of the direction of the center line to be grouped is used as the value of the direction of the center line to be grouped, or the angle value of the direction of the center line to be grouped in the above coordinate system is used as the value of the direction of the center line to be grouped.

[0063] It is possible to determine the second groups that do not overlap with each other, and each second group covers a certain range of values. For example, the average value B of the direction of the center lines of each to-be-grouped polygon is calculated, and the span value b (variable), the minimum value B-xb (the minimum value is generally not greater than the minimum value of the direction of the center lines of each to-be-grouped polygon) and the maximum value B+yb (the maximum value is generally greater than the maximum value of the direction of the center lines of each to-be-grouped polygon) are determined. Then, the second groups are [B-xb, B-(x-2)b), [(x-2)b, (x-4)b), ..., [Bb, B+b), [B+b, B+2b), ..., [B+(y-4)b, B+(y-2)b), [B+(y-2)b, B+yb]. According to the value of the direction of each center line to be grouped, each center line to be grouped is divided into the corresponding second group, so that each center line to be grouped is grouped according to direction.

[0064] The number of center lines actually contained in each second group is determined, and then the second groups containing center lines less than the second preset number are eliminated, and other second groups are retained, thereby achieving direction screening.

[0065] The above-mentioned angle screening and direction screening both belong to physical rule screening, and the first preset number and the second preset number both belong to preset numbers, and the first preset number and the second preset number are variable. In actual situations, there is no absolute order of priority between angle screening and direction screening. If direction screening is performed first, the center line retained after direction screening is used as the center line to be grouped for angle screening.

[0066] After the physical rule screening, the model output result to which the center line that is finally retained belongs or corresponds is the retained model output result. Correspondingly, the model output result to which the center line that is eliminated in the physical rule screening process belongs or corresponds is the eliminated model output result.

[0067] After physical rule screening, for any retained model output result, it can be considered that the model output result contains the target object, and the target object is contained in the aforementioned polygon extracted from the model output result.

[0068] It can be seen that through the above-mentioned standardized judgment and physical rule screening, it is possible to confirm whether the target object is included in the model output result and the position of the target object (the centerline position of the polygon of the model output result is the position of the target object).

[0069] 6. The retained model output results after screening can be combined with the existing training data to form new training data. The specific method can be as follows: (1) Generate a small image and regularize polygons for the retained model output results, including: for any retained model output result, generate a block of a specific size with a specific position of the target object in the model output result as the center, and reconstruct the polygon based on the center line of the target object. Taking the target object as a street lamp as an example, the specific position can be the root of the street lamp, that is, the second vertex of the center line of the street lamp, and the specific size is, for example, 320×320. The reconstructed polygon is, for example, Figure 7 shown.

[0070] (2) Performing geographic coordinate conversion operations, including: using the coordinate relationship between the original large image (i.e., the sample image) and the inference small image (i.e., the block of the specific size mentioned above) to convert the detected target position into actual geographic coordinates. The target position here can be determined as needed. For example, if the target object is a street lamp, the target position can be the pixel coordinates at the root of the street lamp, thereby converting the pixel coordinates at the root of the street lamp into the latitude and longitude coordinates of the root of the street lamp.

[0071] (3) Generate a JSON file, including: creating a corresponding standard JSON file, and writing the label category obtained above, the normalized polygon coordinates in the retained model output results, the target geographic longitude and latitude (i.e., the actual geographic coordinates mentioned above) and other information into the JSON file.

[0072] (4) Merge the newly generated JSON file with the existing training data to form new training data, that is, a new training set.

[0073] By continuing to iteratively train the prepared model with new training data and optimizing the prepared model, an optimized new model can be obtained.

[0074] The next model optimization operation Starting from the second model optimization operation, each model optimization operation can be performed as follows: 1. Obtain sample images. You can refer to the method of obtaining sample images in the first model optimization operation, but the sample images obtained each time may not be the same.

[0075] 2. If necessary, you can refer to the first model optimization operation to preprocess the sample images.

[0076] 3. Input the sample image (or preprocessed image) into the new model obtained by the previous model optimization operation to obtain the output result of the new model.

[0077] 4. To determine whether the output results of the new model meet the standardization requirements, you can refer to the content of the first model optimization operation to determine whether the output results of the model meet the standardization requirements.

[0078] 5. For the results that meet the standardization requirements, use the physical rules matched by the target object to screen the results that meet the standardization requirements. This can be done by referring to the content of the physical rule screening in the first model optimization operation.

[0079] 6. The model output results retained after screening are merged with the existing training data (the existing training data is the training data before the current model optimization operation, including the training data in S101 and the model output results retained by each model optimization operation performed before the current one) to form new training data. The new training data can be executed by referring to the content of the new training data merged in the first model optimization operation.

[0080] The new model obtained from the previous model optimization operation is iteratively trained using new training data to continue to obtain the optimized new model for the next model optimization operation.

[0081] S105: After the end condition of the model optimization operation is reached, the target object detection operation is performed on the image to be detected using the latest model to obtain a target object detection result.

[0082] In the first embodiment, the model optimization operation is not performed without limit, and the end condition of the model optimization operation can be preset. When the end condition of the model optimization operation is reached, the model optimization operation is no longer performed. The end condition of the model optimization operation can be set as needed, for example, the number of model optimization operations reaching a preset number of times can be set as the end condition, or the target object detection effect (including but not limited to the accuracy) of the latest model reaches a preset effect as the end condition.

[0083] After reaching the end condition of the model optimization operation, the latest model (that is, the optimized new model obtained by the last model optimization operation) can be used to perform target object detection operations on the image to be detected to obtain the target object detection results.

[0084] The content of the image to be detected and the method of obtaining it are not limited to the first embodiment.

[0085] Embodiment 1 can achieve the following beneficial effects: By performing model optimization operations in a loop, the model can be gradually optimized, achieving self-improvement of the model and improving the detection effect of the model on the target object.

[0086] Each time in the model optimization operation, the output results of the model are standardized and judged, and the results that meet the standardization requirements are screened using the physical rules matched by the target object, accurately identifying and eliminating erroneous results or results that do not meet the requirements that may occur during the model optimization process, effectively improving the model optimization efficiency, and greatly improving the accuracy and reliability of the optimized model's detection of the target object, thereby improving the effect and efficiency of using the optimized model to detect the image to be detected.

[0087] In particular, the physical laws such as the size and direction of the angle formed by the target object and its shadow in the remote sensing image can be used to screen the model output results based on the principle of geometric angle consistency. By calculating the size and direction of the angle between the target object and its shadow, the erroneous results or results that do not meet the requirements that may appear in the model reasoning process can be accurately identified and eliminated, greatly improving the screening accuracy and reliability of the model output results, thereby effectively improving the model optimization effect.

[0088] In each model optimization operation, the model output results retained after screening are merged with the existing training data as new training data for model training. Through this data merging and cyclic training optimization strategy, the scale of training data can be gradually increased and the detection capability of the model can be optimized.

[0089] During the model optimization operation, the sample images were preprocessed in many aspects, so that the preprocessed images were more suitable for model optimization operations, realizing the combination of model optimization and image processing, thereby improving the model optimization effect and efficiency.

[0090] In the process of model optimization, based on the model output results, combined with geometric processing techniques such as polygon coordinate extraction, centerline simplification, and regularized polygon construction, the centerline can be extracted from the instance segmentation results, simplified and regularized, and finally reconstructed to generate polygonal areas that meet the actual application needs for subsequent data merging. These geometric processing techniques can accurately describe the physical characteristics of the target, provide high-quality data support for standardized judgment operations, physical rule screening, and data merging, thereby improving the model optimization effect and efficiency. In addition, these geometric processing techniques can reconstruct polygonal areas that meet the actual application needs, especially for small area or small volume target objects, thereby ensuring that the optimized model can be used to detect small area or small volume target objects, and improve the detection effect and efficiency of small area or small volume target objects.

[0091] In Example 1, after obtaining the model output results in each model optimization operation, a series of post-processing techniques are used to further process the model output results. These post-processing techniques include: (1) filtering the model output results by physical rules; (2) extracting the polygons and center lines of each region (i.e., each model output result) after instance segmentation, and converting them into standardized data output; (3) eliminating erroneous detection results that do not conform to actual physical laws or results that do not meet the requirements, thereby improving the reliability of the reasoning results; (4) integrating the model output results retained after screening into a standard format suitable for model training, and merging them with existing training data to achieve cyclic training. Through these post-processing techniques, the model optimization effect and efficiency can be effectively improved, thereby improving the effect and efficiency of using the optimized model to detect the image to be detected.

[0092] like Figure 8 As shown, the second embodiment of this specification provides an image target detection device corresponding to the method described in the first embodiment, and the device includes: An initial training module 202 is used to obtain a sample data set and train a preliminary model using the sample data set; A loop optimization module 204, used for loop execution of model optimization operations; The first model optimization operation includes: obtaining a sample image, inputting the sample image into the preparatory model, and obtaining a result output by the preparatory model; judging whether the result meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and training the preparatory model using the new training data to obtain an optimized new model; Starting from the second model optimization operation, each model optimization operation includes: obtaining a sample image, inputting the sample image into a new model, and obtaining a result output by the new model; judging whether the result meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and training the new model using the new training data to obtain an optimized new model; The object detection module 206 is used to perform a target object detection operation on the image to be detected using the latest model after the end condition of the model optimization operation is met to obtain a target object detection result.

[0093] Optionally, obtaining a sample data set includes: Acquire a target area image, and mark the target object in the target area image to form shape feature data of the target object; Generate small slices of the annotation data, adjust the fixed line width and generate the annotation file; The annotation file is converted into a sample data set in a specific format.

[0094] Optionally, determining whether the result meets the standardization requirements includes: Each polygon is extracted from the result, and it is determined whether each polygon satisfies the standardization condition.

[0095] Optionally, judging whether each polygon satisfies the standardization condition includes: For any polygon, perform the standardization judgment operation on the polygon in a loop; Among them, the first standardized judgment operation includes: Extracting the center line of the polygon according to the pixel coordinates of the polygon, and determining whether the center line meets a preset condition; If yes, the center line is saved and the standardization judgment operation is no longer performed on the polygon; If not, the next standardization judgment operation is performed; The next standardization judgment operation includes: Simplify the polygon, extract the center line according to the pixel coordinates of the simplified polygon, and determine whether the newly extracted center line meets the preset conditions; If yes, the center line is saved and the standardization judgment operation is no longer performed; If not, the next standardization judgment operation is performed; If a center line that meets the preset conditions is obtained within a preset number of standardization judgment operations, the polygon meets the standardization conditions; if a center line that meets the preset conditions is not obtained within a preset number of standardization judgment operations, the polygon does not meet the standardization conditions.

[0096] Optionally, if the number of vertices of the center line is a first preset value, the center line satisfies the preset condition.

[0097] Optionally, using the physical rules matched by the target object to filter the results that meet the standardization requirements includes: Determine the screening factors based on the physical rules matched by the target object; The results that meet the standardization requirements are grouped according to the screening factors, and the groups whose number of results does not meet the preset number are eliminated.

[0098] Optionally, the screening factors include the size and / or direction of the angle between the target object and its own shadow.

[0099] A third embodiment of the present specification provides a device for detecting an object in an image, comprising: at least one processor; as well as, a memory communicatively coupled to the at least one processor; in, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the method for detecting objects in images described in the first embodiment.

[0100] A fourth embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the method for detecting objects in an image described in the first embodiment is implemented.

[0101] The above embodiments can achieve the same technical effect, and the embodiments can be used in combination.

[0102] The above is only an embodiment of this specification and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for detecting an object in an image, characterized in that: The method comprises: Obtaining a sample data set, and using the sample data set to train a preliminary model; Execute model optimization operations in a loop; The first model optimization operation includes: obtaining a sample image, inputting the sample image into the preparatory model, and obtaining a result output by the preparatory model; judging whether the result meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and training the preparatory model using the new training data to obtain an optimized new model; Starting from the second model optimization operation, each model optimization operation includes: obtaining a sample image, inputting the sample image into a new model, and obtaining a result output by the new model; judging whether the result meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and training the new model using the new training data to obtain an optimized new model; After the end condition of the model optimization operation is reached, the target object detection operation is performed on the image to be detected using the latest model to obtain the target object detection result.

2. The method according to claim 1, characterized in that Get the sample dataset including: Acquire a target area image, and mark the target object in the target area image to form shape feature data of the target object; Generate small slices of the annotation data, adjust the fixed line width and generate the annotation file; The annotation file is converted into a sample data set in a specific format.

3. The method according to claim 1, characterized in that Determining whether the result meets the standardization requirements includes: Each polygon is extracted from the result, and it is determined whether each polygon satisfies the standardization condition.

4. The method according to claim 3, characterized in that Judging whether each polygon meets the standardized conditions includes: For any polygon, perform the standardization judgment operation on the polygon in a loop; Among them, the first standardized judgment operation includes: Extracting the center line of the polygon according to the pixel coordinates of the polygon, and determining whether the center line meets a preset condition; If yes, the center line is saved and the standardization judgment operation is no longer performed on the polygon; If not, the next standardization judgment operation is performed; The next standardization judgment operation includes: Simplify the polygon, extract the center line according to the pixel coordinates of the simplified polygon, and determine whether the newly extracted center line meets the preset conditions; If yes, the center line is saved and the standardization judgment operation is no longer performed; If not, the next standardization judgment operation is performed; If a center line that meets the preset conditions is obtained within a preset number of standardization judgment operations, the polygon meets the standardization conditions; if a center line that meets the preset conditions is not obtained within a preset number of standardization judgment operations, the polygon does not meet the standardization conditions.

5. The method according to claim 4, characterized in that If the number of vertices of the center line is a first preset value, the center line satisfies the preset condition.

6. The method according to any one of claims 1 to 5, characterized in that Using the physical rules matched by the target object to screen the results that meet the standardization requirements includes: Determine the screening factors based on the physical rules matched by the target object; The results that meet the standardization requirements are grouped according to the screening factors, and the groups whose number of results does not meet the preset number are eliminated.

7. The method according to claim 6, characterized in that The screening factors include the size and / or direction of the angle between the target object and its own shadow.

8. A device for detecting an object in an image, characterized in that: The device comprises: An initial training module, used to obtain a sample data set and train a preliminary model using the sample data set; A loop optimization module, used to execute model optimization operations in a loop; The first model optimization operation includes: obtaining a sample image, inputting the sample image into the preparatory model, and obtaining a result output by the preparatory model; judging whether the result meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and training the preparatory model using the new training data to obtain an optimized new model; Starting from the second model optimization operation, each model optimization operation includes: obtaining a sample image, inputting the sample image into a new model, and obtaining a result output by the new model; judging whether the result meets the standardization requirements, screening the results that meet the standardization requirements using the physical rules matched by the target object, merging the screened results with the existing training data into new training data, and training the new model using the new training data to obtain an optimized new model; The object detection module is used to perform a target object detection operation on the image to be detected using the latest model after the end condition of the model optimization operation is met to obtain a target object detection result.

9. An object detection device in an image, characterized in that: include: at least one processor; as well as, a memory communicatively coupled to the at least one processor; in, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the method for detecting objects in images according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the method for detecting an object in an image according to any one of claims 1 to 7 is implemented.

Images