A method, device, equipment and storage medium for detecting a target object

Abstract

The present invention relates to a method, device, equipment and storage medium for detecting a target object, which obtains a target image, inputs the target image into a first model to identify the object in the target image, and obtains a first possibility that each object in the target image belongs to the target object; obtains feature data of the first object to obtain first feature data; inputs the first feature data into a second model to identify the first object, and obtains a second possibility that the first object belongs to the target object; and determines whether the target object is included in the target image based on the second possibility. The scheme can obtain a more accurate target detection result than a deep learning algorithm, and can detect target objects of different categories, thereby expanding the category range of target objects that can be detected and improving applicability.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a target object detection method, device, equipment and storage medium. Background Art

[0002] At present, the use of deep learning algorithms to detect target objects has achieved good results, but deep learning algorithms still have defects when detecting target objects in remote sensing images.

[0003] First, deep learning models are trained using a large number of image samples. The principle of deep learning models to detect target objects is to detect target objects based on image features such as texture, details, edges, pixel gradients, and shapes of pixels in the image. However, for marine remote sensing images, the resolution of the images varies due to the influence of factors such as lighting, weather, and sea conditions. For low-resolution remote sensing images, their image features are not obvious.

[0004] In addition, since the deep learning model uses supervised learning, it is easy to have category preference problems when the training data is unbalanced, resulting in poor detection of objects in images of the same type as those in the non-training set. For example, in the detection of ship targets in ocean remote sensing images, the deep learning model has a good detection effect on ships of the same category and structure as in the sample image, but often cannot detect other categories of ship objects that are significantly different from the sample image.

[0005] Therefore, the accuracy of target detection in remote sensing images using deep learning algorithms is low. For multi-category detection targets, the algorithm can detect fewer categories and has poor applicability. Summary of the invention

[0006] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the first aspect of the present invention provides a target object detection method, the method comprising:

[0007] Acquire the image to be detected;

[0008] Inputting the image to be detected into a first model to identify objects in the image to be detected, and obtaining a first possibility that each object in the image to be detected belongs to a target object; the first model is trained based on a first sample image of the target object;

[0009] Acquire feature data of a first object to obtain first feature data; the first object is an object whose first possibility is greater than a preset first threshold;

[0010] Inputting the first feature data into a second model to identify the first object, and obtaining a second possibility that the first object belongs to the target object; the second model is trained based on the second feature data, the second feature data is extracted from text feature information corresponding to features of each object in a second sample image, and the second sample image includes images of the target object of different categories;

[0011] Determine whether the image to be detected contains the target object based on the second possibility.

[0012] Optionally, after obtaining the second possibility that the first object belongs to the target object, the method further includes:

[0013] Acquire the second characteristic data;

[0014] A third possibility that the first object belongs to the target object is determined based on the second feature data and the first feature data.

[0015] Optionally, the first feature data includes at least: aspect ratio information, color information, shape information, and size information of the first object, and the third possibility of determining that the first object belongs to the target object includes:

[0016] Acquire size information of the first object from the first feature data to obtain a first size;

[0017] Acquire pixel equivalent information of the second sample image from the second feature data;

[0018] determining an actual size of the first object according to the pixel equivalent information and the first size;

[0019] A third possibility is determined, based on the actual size, that the first object belongs to the target object.

[0020] Optionally, determining whether the image to be detected contains the target object based on the second possibility includes:

[0021] Performing a weighted summation on the second possibility and the third possibility to obtain a target possibility;

[0022] According to the relationship between the target possibility and a preset second threshold, it is determined whether the image to be detected includes the target object; the second threshold is greater than the first threshold.

[0023] Optionally, after determining whether the image to be detected contains the target object based on the second possibility, the method further includes:

[0024] If it is determined that the target object is included in the image to be detected, acquiring feature data of the target object;

[0025] Acquire feature information corresponding to each target object category in the second feature data to obtain category feature information;

[0026] The characteristic data of the target object and the category characteristic information are matched to determine the category of the target object.

[0027] Optionally, before inputting the image to be detected into the first model to identify the object in the image to be detected, the method further includes:

[0028] Setting an output result of the first model, wherein the output result includes feature data of each object in the image to be detected;

[0029] The acquiring feature data of the first object includes:

[0030] Acquire feature data of the first object from the output result.

[0031] Optionally, the first model is trained by the following method:

[0032] Preprocessing the original image to obtain a first sample image; the preprocessing at least includes performing linear transformation, spatial transformation, and image enhancement on the original image;

[0033] Determine the usage scenario and usage requirements of the target object detection, and select a matching algorithm model from multiple algorithm models according to the usage scenario and usage requirements to obtain a first algorithm model.

[0034] The first algorithm model is trained using the first sample image to obtain a first model.

[0035] Optionally, the second model is trained by the following method:

[0036] Extracting text feature information corresponding to the target object to be detected from the second sample image to obtain second feature data;

[0037] Determine an output value type of the target object detection, and select an algorithm model matching the output value type from a plurality of algorithm models to obtain a second algorithm model; the output value type includes one of a continuous value and a type value;

[0038] The second algorithm model is trained using the second feature data to obtain a second model.

[0039] Optionally, after obtaining the second model, the method further includes:

[0040] Acquire a third sample image, where an object in the third sample image is an object that is not in the first sample image and the second sample image;

[0041] Extracting feature information of the target object to be detected from the third image sample data to obtain newly added sample feature data;

[0042] The second algorithm model is retrained using the newly added sample feature data and the sample feature data obtained according to the second sample image to obtain an updated second model.

[0043] Optionally, the second feature data includes at least: category information, aspect ratio information, color information, shape information, geographic location information of the target object and pixel equivalent information of the second sample image.

[0044] A second aspect of the present invention provides a target object detection device, the device comprising:

[0045] An acquisition module, used for acquiring an image to be detected;

[0046] A first recognition module is used to input the image to be detected into a first model to recognize objects in the image to be detected, and obtain a first possibility that each object in the image to be detected belongs to a target object; the first model is trained based on a first sample image of the target object;

[0047] A first feature data acquisition module, used to acquire feature data of a first object to obtain first feature data; the first object is an object whose first possibility is greater than a preset first threshold;

[0048] a second recognition module, configured to input the first feature data into a second model to recognize the first object, and obtain a second possibility that the first object belongs to the target object; the second model is trained based on the second feature data, the second feature data is extracted from text feature information corresponding to features of each object in a second sample image, and the second sample image at least includes the first sample image;

[0049] A determination module is used to determine whether the image to be detected contains the target object based on the second possibility.

[0050] Optionally, the device further comprises:

[0051] A second feature data acquisition module, used to acquire the second feature data;

[0052] A third possibility determination module is used to determine a third possibility that the first object belongs to the target object according to the second feature data and the first feature data.

[0053] Optionally, the first feature data at least includes: aspect ratio information, color information, shape information, and size information of the first object, and the third possibility determination module is specifically used to:

[0054] Acquire size information of the first object from the first feature data to obtain a first size;

[0055] Acquire pixel equivalent information of the second sample image from the second feature data;

[0056] determining an actual size of the first object according to the pixel equivalent information and the first size;

[0057] A third possibility is determined, based on the actual size, that the first object belongs to the target object.

[0058] Optionally, the determining module is specifically used to:

[0059] Performing a weighted summation on the second possibility and the third possibility to obtain a target possibility;

[0060] According to the relationship between the target possibility and a preset second threshold, it is determined whether the image to be detected includes the target object; the second threshold is greater than the first threshold.

[0061] Optionally, the device further comprises:

[0062] A target object feature data acquisition module, configured to acquire feature data of the target object if it is determined that the image to be detected contains the target object;

[0063] A category feature information acquisition module, used to acquire feature information corresponding to each target object category in the second feature data, to obtain category feature information;

[0064] The target object category determination module is used to match the feature data of the target object with the category feature information to determine the category of the target object.

[0065] Optionally, the device further comprises:

[0066] An output result setting module, used to set the output result of the first model, wherein the output result includes feature data of each object in the image to be detected;

[0067] The first feature data acquisition module is specifically used for:

[0068] Acquire feature data of the first object from the output result.

[0069] Optionally, the device further includes a first model training module, wherein the first model training module is specifically used for:

[0070] Preprocessing the original image to obtain a first sample image; the preprocessing at least includes performing linear transformation, spatial transformation, and image enhancement on the original image;

[0071] Determine the usage scenario and usage requirements of the target object detection, and select a matching algorithm model from multiple algorithm models according to the usage scenario and usage requirements to obtain a first algorithm model.

[0072] The first algorithm model is trained using the first sample image to obtain a first model.

[0073] Optionally, the device further includes a second model training module, wherein the second model training module is specifically configured to:

[0074] Extracting text feature information corresponding to the target object to be detected from the second sample image to obtain second feature data;

[0075] Determine an output value type of the target object detection, and select an algorithm model matching the output value type from a plurality of algorithm models to obtain a second algorithm model; the output value type includes one of a continuous value and a type value;

[0076] Optionally, the second model training module is further used for:

[0077] Acquire a third sample image, where an object in the third sample image is an object that is not in the first sample image and the second sample image;

[0078] Extracting feature information of the target object to be detected from the third image sample data to obtain newly added sample feature data;

[0079] The second algorithm model is retrained using the newly added sample feature data and the sample feature data obtained according to the second sample image to obtain an updated second model.

[0080] A third aspect of the present invention proposes a device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the target object detection method as described in the first aspect.

[0081] A fourth aspect of the present invention proposes a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the target object detection method as described in the first aspect.

[0082] According to the specific embodiments provided by the present invention, the present invention has the following technical effects:

[0083] The target detection method provided by the embodiment of the present invention obtains an image to be detected, inputs the image to be detected into a first model to identify the object in the image to be detected, and obtains a first possibility that each object in the image to be detected belongs to the target object; obtains feature data of the first object to obtain first feature data; inputs the first feature data into a second model to identify the first object to obtain a second possibility that the first object belongs to the target object; and determines whether the image to be detected contains the target object based on the second possibility. This scheme examines the image features of each object in the image to be detected through the first model to obtain the first object, and further examines the object features of the first object itself through the second model. In this way, the image features and object features of the image to be detected are combined to obtain a more accurate target detection result than a deep learning algorithm. In addition, the feature data of target objects of different categories are collected in the second model, so target objects of different categories can be detected, which expands the category range of target objects that can be detected and improves applicability.

[0084] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0086] Figure 1 A flowchart of a target object detection method provided by an embodiment of the present invention;

[0087] Figure 2 A flowchart of another target object detection method provided by an embodiment of the present invention;

[0088] Figure 3 A flowchart of the steps of the training method of the first model provided by an embodiment of the present invention;

[0089] Figure 4A flowchart of the steps of the training method of the second model provided by an embodiment of the present invention;

[0090] Figure 5 A structural block diagram of a target object detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0091] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0092] Figure 1 A flowchart of a target object detection method provided by an embodiment of the present invention. This specification provides method operation steps as described in the embodiment or flowchart, but more or fewer operation steps may be included based on conventional or non-creative labor. When the actual system or server product is executed, the method shown in the embodiment or the accompanying drawings may be executed sequentially or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0093] The method may include the following steps:

[0094] Step 101: Acquire an image to be detected.

[0095] The image to be detected may be an ordinary image or a remote sensing image. Specifically, as an important military target and main transport carrier at sea, ships have important significance and broad application prospects in the military and civilian fields. In the embodiment of the present invention, the marine remote sensing image is used as the image to be detected, and the detection of a ship target is used as an example to illustrate the method of the present invention.

[0096] Step 102: input the image to be detected into a first model to identify objects in the image to be detected, and obtain a first possibility that each object in the image to be detected belongs to a target object; the first model is trained based on a first sample image of the target object.

[0097] After the image to be detected is input into the first model for target object detection, each object in the image to be detected is marked with a label box in the output result of the first model, and the first possibility that the object belongs to the target object is marked in each label box. The first possibility is generally expressed as a probability value, such as a percentage or a decimal.

[0098] For example, an ocean remote sensing image is input into the first model for target object detection. In the output result, objects such as islands, flying birds, clouds, and ships on the detected ocean remote sensing image are respectively marked with probability values ​​of belonging to ships.

[0099] The first model is trained by inputting the first sample image, so that the first model detects the target object based on the image features such as texture, detail, edge, pixel gradient, shape, etc. of the pixel points in the image. However, for remote sensing images, due to the influence of factors such as lighting, weather, and sea conditions, the resolution of the image is diverse, and for low-resolution remote sensing images, their image features are not obvious.

[0100] Furthermore, the first model is trained using the first sample image. Since supervised learning is used, when the training data is unbalanced, a category preference problem is prone to occur, resulting in poor object detection results for images of the same type in non-training sets.

[0101] Therefore, if the target object is detected only by the first model, the accuracy of the detection result is low, and its applicability to multiple types of target objects is poor.

[0102] Step 103: Acquire feature data of a first object to obtain first feature data; the first object is an object whose first possibility is greater than a preset first threshold.

[0103] In order to further improve the accuracy of target detection, objects with a relatively high probability of belonging to the target object can be screened from the detection results of the first model, that is, objects with a first probability greater than a preset first threshold can be screened. Objects that are very different from the target object can be preliminarily eliminated by the first threshold, and the remaining objects are used as objects for the next step of detection. Therefore, the first threshold can be set to a lower threshold.

[0104] For example, if the first possibility is expressed in the form of a percentage, the first threshold may be set to 50%. Of course, the embodiment of the present invention does not specifically limit the value of the first threshold.

[0105] Step 104: input the first feature data into a second model to identify the first object, and obtain a second possibility that the first object belongs to the target object; the second model is trained based on the second feature data, and the second feature data is extracted from text feature information corresponding to the features of each object in a second sample image, and the second sample image includes images of the target object of different categories.

[0106] The second model is obtained by training the second feature data, and the second feature data is text information extracted from the second sample image, so the second model has a strong processing capability for text data. The second model can judge the similarity between the text feature data and the text feature data of the target object according to the mapping function between the feature data of the object to be detected and the target object formed during training, and further obtain the possibility that the object to be detected belongs to the target object.

[0107] Furthermore, the second sample images include images of target objects of different categories, so the second feature data obtained also includes feature data of target objects of different categories, so that the second model trained by the second feature data can be used to detect target objects of different categories.

[0108] In the embodiment of the present invention, the first model is used to first screen out the first object that is more likely to be the target object, and then the characteristic attribute-related data is extracted from the first object to obtain the first characteristic data. The first characteristic data is text data, and the second model is good at processing text data. By inputting the first characteristic data into the second model, a relatively accurate prediction can be made on the possibility that the first object belongs to the target object.

[0109] Optionally, the second feature data includes at least: category information, aspect ratio information, color information, shape information, geographic location information of the target object and pixel equivalent information of the second sample image.

[0110] Second feature data is extracted from the second sample image. The second feature data includes feature attribute information of the target object and feature attribute information of the image. These feature information can be used to comprehensively determine whether an object is a target object.

[0111] Among them, the category information of the target object refers to the category to which the target object belongs. For example, for a ship object, its categories include: aircraft carrier, battleship, cruiser, battlecruiser, destroyer, etc.; the length-to-width ratio information of the target object refers to the ratio between the length and width of the target object; the geographical location information of the target object refers to the longitude and latitude of the target object.

[0112] The pixel equivalent information of the second sample image belongs to the characteristic attribute information of the image. The pixel equivalent refers to the actual physical size represented by a pixel point in the image.

[0113] These feature information basically covers all the feature attributes of the target object, which is conducive to improving the detection accuracy of the second model and improving the applicability of the second model to multiple categories of target objects.

[0114] Step 105: Determine whether the image to be detected contains the target object based on the second possibility.

[0115] The second model outputs a second possibility that the first object belongs to the target object, and the second possibility can also be in the form of a percentage or a decimal. According to the size of the second possibility, it can be finally determined whether the image to be detected contains the target object.

[0116] The second possibility is to input the detection result of the first model into the second model for re-verification. It comprehensively considers the image information such as texture, details, edges, pixel gradients, and shapes of the image to be detected to obtain the first object, and further considers the feature information of the first object itself, such as aspect ratio information, color information, shape information, etc. In this way, the image information and feature information of the object to be detected are combined to obtain a more accurate target detection result compared to the first model or the second model alone.

[0117] Furthermore, the second feature data includes feature information of target objects of different categories, so target objects of different categories can be detected, thereby expanding the detection categories of target objects.

[0118] For example, an ocean remote sensing image is input into the first model for target object detection, and objects with a probability of belonging to a ship greater than a first threshold are selected from the output results as first objects, which may include objects that are not ships, such as small islands, dark waves, etc. The first feature data of the first object is extracted, and the first feature data is input into the second model. The second model can then determine the second possibility that the first feature data of each first object belongs to the feature data of a ship, that is, the second possibility that the first object belongs to a ship object is obtained. Therefore, it can be determined whether this ocean remote sensing image includes a ship based on the second possibility, and which object in the image is a ship.

[0119] For example, in the extracted first feature data, the length-to-width ratio of the first object is 5:1. When the first feature data is input into the second model, the second possibility obtained is only 20%. This is because there is no ship with a length-to-width ratio of 5:1 in the second feature data. Therefore, the possibility that the second model accurately determines that the first object belongs to a ship is very small.

[0120] In summary, the target detection method provided by the embodiment of the present invention obtains the image to be detected, inputs the image to be detected into the first model to identify the object in the image to be detected, and obtains the first possibility that each object in the image to be detected belongs to the target object; obtains the feature data of the first object to obtain the first feature data; inputs the first feature data into the second model to identify the first object, and obtains the second possibility that the first object belongs to the target object; and determines whether the image to be detected contains the target object based on the second possibility. This scheme examines the image features of each object in the image to be detected through the first model to obtain the first object, and further examines the object features of the first object itself through the second model. In this way, the image features and object features of the image to be detected are combined to obtain a more accurate target detection result than the deep learning algorithm, or the first model or the second model alone. In addition, the feature data of target objects of different categories are collected in the second model, so target objects of different categories can be detected, which expands the category range of target objects that can be detected and improves applicability.

[0121] Figure 2 A flowchart of another target object detection method provided by an embodiment of the present invention.

[0122] The method may include the following steps:

[0123] Step 201: Acquire an image to be detected.

[0124] In the embodiment of the present invention, step 201 may refer to step 101 and will not be described in detail here.

[0125] Step 202: setting an output result of the first model, wherein the output result includes feature data of each object in the image to be detected.

[0126] The output result of the first model generally marks each object in the image to be detected with a label box, and marks the first possibility that the object belongs to the target object in each label box. In order to facilitate the subsequent use of the feature data of each object, the first model can also output additional output results, that is, output the feature data of each object.

[0127] The feature data of each object may be obtained by an algorithm recognizing and calculating the object in the image, and may include aspect ratio information, color information, shape information, size information, etc. of the object.

[0128] Step 203: input the image to be detected into a first model to identify objects in the image to be detected, and obtain a first possibility that each object in the image to be detected belongs to a target object; the first model is trained based on a first sample image of the target object.

[0129] In the embodiment of the present invention, step 203 may refer to step 102 and will not be described in detail here.

[0130] Step 204: Acquire feature data of a first object to obtain first feature data; the first object is an object whose first possibility is greater than a preset first threshold.

[0131] In the embodiment of the present invention, step 204 may refer to step 103 and will not be described in detail here.

[0132] Optionally, the acquiring feature data of the first object includes: acquiring the feature data of the first object from the output result.

[0133] Specifically, the first object is screened out from the output result, and feature data of the first object is acquired from the output result.

[0134] It is simpler, faster and more accurate to directly obtain the feature data of the first object from the output result.

[0135] In addition, the first feature data may also be obtained in other ways. For example, after the first object is determined, an image of the first object is obtained through third-party software, and the feature data of the first object is extracted.

[0136] Step 205: input the first feature data into a second model to identify the first object, and obtain a second possibility that the first object belongs to the target object; the second model is trained based on the second feature data, and the second feature data is extracted from text feature information corresponding to the features of each object in a second sample image, and the second sample image at least includes the first sample image.

[0137] In the embodiment of the present invention, step 205 may refer to step 104 and will not be described in detail here.

[0138] The above steps 201 to 205 are a process of obtaining the first feature data from the first model and inputting the first feature data into the second model for comprehensive detection. In order to further improve the detection accuracy, the second feature data used in the training of the second model can also be compared with the first feature data output by the first model to obtain a new detection result, i.e., steps 206 and 207.

[0139] Step 206: Obtain the second characteristic data.

[0140] Step 207: Determine a third possibility that the first object belongs to the target object based on the second feature data and the first feature data.

[0141] In step 206 and step 207, each item in the first feature data may be compared with the corresponding item in the second feature data, thereby determining a third possibility that the first object belongs to the target object.

[0142] Optionally, step 207 includes the following steps A1-A4:

[0143] A1. Acquire size information of the first object from the first feature data to obtain a first size;

[0144] A2. Acquire pixel equivalent information of the second sample image from the second feature data;

[0145] A3. determining an actual size of the first object according to the pixel equivalent information and the first size;

[0146] A4. Determine, according to the actual size, a third possibility that the first object belongs to the target object.

[0147] In steps A1-A4, pixel equivalent refers to the actual physical size represented by a pixel in the image, and the first size is the number of pixels of the length, width and other parameters of the first object in the image. The actual physical size of the first object can be calculated through the pixel equivalent information of the sample image. According to the difference between the actual size and the first size, the third possibility can be determined.

[0148] For example, if the first size of the first object in the image is 5*3, and the pixel equivalent of the second sample image is 100 cm:1, that is, the physical size corresponding to one pixel in the image is 100 cm, then the actual physical size of the first object can be calculated to be 500 cm*300 cm. Then, based on the actual size of the ship, it can be determined whether the first object of 500 cm*300 cm is a ship.

[0149] Step 208: Perform a weighted summation on the second possibility and the third possibility to obtain a target possibility.

[0150] According to the experimental test results, the weights of the second possibility and the third possibility can be set in advance, and the weighted sum of the two can be taken. The obtained target possibility combines the second possibility and the third possibility, and the obtained detection result is more comprehensive and accurate.

[0151] In addition, the second possibility and the third possibility can be comprehensively judged using decision methods such as ensemble learning and EMV (Expected Monetary Value) decision comparison to obtain the target possibility.

[0152] Step 209: Determine whether the image to be detected includes the target object based on the relationship between the target possibility and a preset second threshold; the second threshold is greater than the first threshold.

[0153] The second threshold value may be set to a value larger than the first threshold value, and an optimal value may be determined through experiments.

[0154] For example, if the second threshold is 90% and the value of the target possibility is 92%, it can be determined that the first object corresponding to the target possibility is the target object, that is, it is concluded that the image to be detected includes the target object.

[0155] Step 210: If it is determined that the image to be detected contains the target object, then obtain feature data of the target object.

[0156] Step 211: Acquire feature information corresponding to each target object category in the second feature data to obtain category feature information.

[0157] Step 212: Match the feature data of the target object with the category feature information, and determine the category of the target object.

[0158] In step 210-step 212, since the second feature data includes the category information of the target object, the feature data corresponding to different target object categories can be counted to obtain the category feature information. The feature information of the target object is matched with the category feature information to obtain the category to which the target object belongs.

[0159] For example, the category feature information includes the geographical location where the ships of this category are usually located, such as the Liaoning ship will appear in the geographical scope of China, and the Thor will appear in the geographical scope of the United States. From the feature data of the target object, it is obtained that the geographical location of the target object is in China, and according to the category feature information, it can be obtained that the target object is one of the types of ships in China.

[0160] Figure 3 A flowchart of the steps of the training method of the first model provided in an embodiment of the present invention.

[0161] The method comprises the following steps:

[0162] Step 301: perform data cleaning and preprocessing on the original image to obtain a first sample image; the preprocessing at least includes performing linear transformation, spatial transformation, and image enhancement on the original image.

[0163] Data cleaning of manually collected raw images is mainly to remove dirty or erroneous data in the raw images. Preprocessing of raw images, including but not limited to linear transformation, spatial transformation, image enhancement and other image processing, is to make the image features more prominent and make it easier for the algorithm model to learn the image features, so as to speed up the training speed and improve the model detection effect.

[0164] Step 302: determine the usage scenario and usage requirement of target object detection, and select a matching algorithm model from multiple algorithm models according to the usage scenario and usage requirement to obtain a first algorithm model.

[0165] Determine the usage scenarios and requirements of target object detection, and select different algorithm models based on the usage scenarios and requirements. For example, for scenarios with high requirements for effects or high requirements for running stability, you can choose a two-stage target detection model like Faster-RCNN (Faster Region-based Convolutional Neural Networks); for scenarios with high requirements for processing speed, you can choose a one-stage target detection model like YOLO (You Only Look Once).

[0166] Step 303: Use the first sample image to train the first algorithm model to obtain a first model.

[0167] After the first algorithm model is determined, the first model is finally obtained through steps such as feature information extraction, model training, model parameter adjustment, and model evaluation.

[0168] In step 301-step 303, the original image is preprocessed to make the image features more prominent and make it easier for the algorithm model to learn the image features, thereby speeding up the training speed and improving the model detection effect; according to the usage scenarios and usage requirements of the target object detection, a matching algorithm model is selected to make the training results more in line with actual needs and provide users with a better usage experience.

[0169] Figure 4 A flowchart of the steps of the training method of the second model provided in an embodiment of the present invention.

[0170] The method comprises the following steps:

[0171] Step 401: extract text feature information corresponding to the target object to be detected from the second sample image to obtain second feature data.

[0172] First, the manually collected original images are cleaned and preprocessed, including removing logical errors and outliers, supplementing missing values, and correcting format content.

[0173] Then, feature selection technology is used to select the attribute features to be extracted, that is, to select the features that can make the model detection effect best from all the features of the current target object. Different feature selection methods are selected according to needs. The filtering method is used when pursuing training speed, the encapsulation method is used when pursuing model performance, and the embedding method is used when pursuing a balance between the two.

[0174] Step 402: determine the output value type of the target object detection, and select an algorithm model that matches the output value type from multiple algorithm models to obtain a second algorithm model; the output value type includes one of a continuous value and a type value.

[0175] In different usage environments, the output result types of the deep learning model may be different. If a continuous value needs to be fed back, such as feeding back a characteristic value of a ship according to different proportions, it is a regression problem; if a certain type of value needs to be returned, such as returning the ship type code, it is a classification problem.

[0176] According to the output result type, an algorithm model matching the output value type is selected from multiple algorithm models to obtain a second algorithm model.

[0177] Step 403: Use the second feature data to train the second algorithm model to obtain a second model.

[0178] The process of training the second algorithm model using the second feature data includes the step of adjusting the model parameters, and the technicians are required to adjust the parameters according to the model used and the training results.

[0179] After the model training is completed, it is necessary to judge whether the model is good or not. Model evaluation indicators are a method to judge whether the model is good or not. This step uses model evaluation indicators to judge whether the model is good or not. Different model evaluation indicators are used for different types of problems and situations.

[0180] The results of the model are used to determine whether further iterative training is needed. If the results are good, the model training is completed and the model can be used for prediction. If the results are not ideal, further model parameter adjustment and other steps are performed according to the situation to train again.

[0181] In steps 401-403, the output value type of the target object detection is determined, and an algorithm model matching the output value type is selected from multiple algorithm models to obtain a second algorithm model, which can speed up the training of the second algorithm model and obtain more accurate detection results.

[0182] Optionally, after obtaining the second model, the following steps B1-B3 are further included:

[0183] B1. Acquire a third sample image, where an object in the third sample image is an object that is not in the first sample image and the second sample image;

[0184] B2. extracting feature information of the target object to be detected from the third image sample data to obtain newly added sample feature data;

[0185] B3. Retrain the second algorithm model using the newly added sample feature data and the sample feature data obtained according to the second sample image to obtain an updated second model.

[0186] In traditional deep learning algorithms, if you want to add a target object type as a new target object, you need to collect a large amount of training data for this type of target object, manually identify the data set, and then retrain the new training data together with the old data, which takes a lot of time and money.

[0187] However, in this solution, only the second model needs to be trained separately. Specifically, if a new target object type different from the previous one is to be added, only the third sample image of the target object of this type needs to be collected, and the new sample feature data such as the shape, structure, and size of the target object are extracted from the third sample image. Then, the new sample feature data is retrained together with the old second feature data to obtain an updated second model.

[0188] In this way, even if the first model is unable to recognize the type of target object in step 102, the second model has feature data of the type of target object, and the feature data of the type of target object is input into the second model, accurate recognition can be achieved.

[0189] Therefore, when a new target object category is added, this solution only needs to train the second model, which can still achieve the effect of identifying the new target object. The training time is much lower than that of the traditional method, greatly reducing the time and labor costs.

[0190] Figure 5 This is a structural block diagram of a target object detection device provided by an embodiment of the present invention.

[0191] The target detection device 500 comprises:

[0192] An acquisition module 501 is used to acquire an image to be detected;

[0193] A first recognition module 502 is used to input the image to be detected into a first model to recognize objects in the image to be detected, and obtain a first possibility that each object in the image to be detected belongs to a target object; the first model is trained based on a first sample image of the target object;

[0194] A first feature data acquisition module 503 is used to acquire feature data of a first object to obtain first feature data; the first object is an object whose first probability is greater than a preset first threshold;

[0195] A second recognition module 504 is configured to input the first feature data into a second model to recognize the first object, and obtain a second possibility that the first object belongs to the target object; the second model is trained based on the second feature data, the second feature data is extracted from text feature information corresponding to features of each object in a second sample image, and the second sample image at least includes the first sample image;

[0196] The determination module 505 is configured to determine whether the image to be detected contains the target object based on the second possibility.

[0197] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0198] In another embodiment provided by the present invention, a device is also provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the target object detection method described in the embodiment of the present invention.

[0199] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the target object detection method described in the embodiment of the present invention.

[0200] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0201] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0202] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0203] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A target object detection method, characterized in that: The method comprises: Acquire an image to be detected; the image to be detected is an ocean remote sensing image; Inputting the image to be detected into a first model to identify objects in the image to be detected, and obtaining a first possibility that each object in the image to be detected belongs to a target object; the first model is trained based on a first sample image of the target object; the target object is a ship; Acquire feature data of a first object to obtain first feature data; the first object is an object whose first possibility is greater than a preset first threshold; Inputting the first feature data into a second model to identify the first object, and obtaining a second possibility that the first object belongs to the target object; the second model is trained based on the second feature data, the second feature data is extracted from text feature information corresponding to features of each object in a second sample image, and the second sample image includes images of the target object of different categories; Determine whether the image to be detected contains the target object based on the second possibility; After obtaining the second possibility that the first object belongs to the target object, the method further includes: Acquire the second characteristic data; determining, based on the second feature data and the first feature data, a third possibility that the first object belongs to the target object; The determining, based on the second possibility, whether the image to be detected contains the target object includes: Performing a weighted summation on the second possibility and the third possibility to obtain a target possibility; According to the relationship between the target possibility and a preset second threshold, it is determined whether the image to be detected includes the target object; the second threshold is greater than the first threshold.

2. The method according to claim 1, characterized in that The first feature data at least includes: aspect ratio information, color information, shape information, and size information of the first object. The third possibility of determining that the first object belongs to the target object includes: Acquire size information of the first object from the first feature data to obtain a first size; Obtaining pixel equivalent information of the second sample image from the second feature data; the pixel equivalent refers to the actual physical size represented by a pixel point in the image; determining an actual size of the first object according to the pixel equivalent information and the first size; A third possibility is determined, based on the actual size, that the first object belongs to the target object.

3. The method according to claim 1, characterized in that After determining whether the image to be detected contains the target object based on the second possibility, the method further includes: If it is determined that the target object is included in the image to be detected, acquiring feature data of the target object; Acquire feature information corresponding to each target object category in the second feature data to obtain category feature information; The characteristic data of the target object and the category characteristic information are matched to determine the category of the target object.

4. The method according to claim 1, characterized in that Before inputting the image to be detected into the first model to identify the object in the image to be detected, the method further includes: Setting an output result of the first model, wherein the output result includes feature data of each object in the image to be detected; The acquiring feature data of the first object includes: Acquire feature data of the first object from the output result.

5. The method according to any one of claims 1 to 4, characterized in that: The first model is trained by the following method: Preprocessing the original image to obtain a first sample image; the preprocessing at least includes performing linear transformation, spatial transformation, and image enhancement on the original image; Determine a usage scenario and usage requirement for target object detection, and select a matching algorithm model from a plurality of algorithm models according to the usage scenario and usage requirement to obtain a first algorithm model; The first algorithm model is trained using the first sample image to obtain a first model.

6. The method according to any one of claims 1 to 4, characterized in that: The second model is trained by the following method: Extracting text feature information corresponding to the target object to be detected from the second sample image to obtain second feature data; Determine an output value type of the target object detection, and select an algorithm model matching the output value type from a plurality of algorithm models to obtain a second algorithm model; the output value type includes one of a continuous value and a type value; The second algorithm model is trained using the second feature data to obtain a second model.

7. The method according to claim 6, characterized in that After obtaining the second model, it also includes: Acquire a third sample image, where an object in the third sample image is an object that is not in the first sample image and the second sample image; Extracting feature information of the target object to be detected from the third image sample data to obtain newly added sample feature data; The second algorithm model is retrained using the newly added sample feature data and the sample feature data obtained according to the second sample image to obtain an updated second model.

8. The method according to claim 1, characterized in that The second feature data at least includes: category information, aspect ratio information, color information, shape information, geographic location information and pixel equivalent information of the second sample image of the target object.

9. A target object detection device, characterized in that: The device comprises: The image acquisition module to be detected is used to acquire the image to be detected; the image to be detected is an ocean remote sensing image; a first recognition module, configured to input the image to be detected into a first model to recognize objects in the image to be detected, and obtain a first possibility that each object in the image to be detected belongs to a target object; the first model is trained based on a first sample image of the target object; the target object is a ship; A first feature data acquisition module, used to acquire feature data of a first object to obtain first feature data; the first object is an object whose first possibility is greater than a preset first threshold; a second recognition module, configured to input the first feature data into a second model to recognize the first object, and obtain a second possibility that the first object belongs to the target object; the second model is trained based on the second feature data, the second feature data is extracted from text feature information corresponding to features of each object in a second sample image, and the second sample image at least includes the first sample image; a target object determination module, configured to determine whether the image to be detected contains the target object based on the second possibility; The device is also used to Acquire the second characteristic data; determining, based on the second feature data and the first feature data, a third possibility that the first object belongs to the target object; The target object determination module is specifically used for: Performing a weighted summation on the second possibility and the third possibility to obtain a target possibility; According to the relationship between the target possibility and a preset second threshold, it is determined whether the image to be detected includes the target object; the second threshold is greater than the first threshold.

10. A device for detecting a target object, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the target object detection method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the target object detection method as described in any one of claims 1-8.

Images