Target detection method, system and device based on image generation model

Abstract

The embodiment of the invention provides a target detection method, system and device based on an image generation model. In the embodiment, a mode of dynamically optimizing the image generation model is adopted, the finally obtained image generation model can be ensured to be optimal, and then the training image generated by the optimal image generation model and used for training the target detection model is ensured to be optimal, so that the optimal training image generated by the optimal image generation model is utilized, and the target detection efficiency is improved. The target detection model is trained, so that training images for training the detection model are enriched, and the influence on the performance of the detection model, the accuracy of target detection and the like due to the lack of the training images is avoided.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a target detection method, system, and device based on an image generation model. Background Art

[0002] In specific applications, detection models are often used to detect objects in images. For example, a dangerous animal detection model is used to detect whether there are dangerous animals such as poisonous snakes and tigers in an image.

[0003] However, in practice, due to the specific nature of targets such as dangerous animals, it's often difficult to directly collect many real-world training images containing dangerous animals. For example, images of tigers in gardens or bears at gas stations are difficult to collect. This leads to a shortage of training images for training detection models. This lack of training images can affect the performance of the trained detection model and the accuracy of target detection. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a target detection method, system and device based on an image generation model to realize dynamic generation of training images, thereby avoiding the impact on the performance of the detection model, the accuracy of target detection, etc. due to the difficulty in collecting training images containing targets such as dangerous animals in real scenes.

[0005] The present invention provides an object detection method based on an image generation model, the method comprising: generating an object model corresponding to the object based on sample images of the same object in different scenes generated using different image generation methods and a current image generation model; Fine-tune the current image generation model based on the object models corresponding to different objects; Determine whether the current image generation model meets the model iteration requirements. If not, return to the step of generating the object model corresponding to the object; if yes, then: Generate a training image for training the object detection model based on the current image generation model and using a mask area corresponding to an image area in each training scene image; the mask area is used to indicate the area where the object is located; Using the mask area corresponding to the object area in each training image and combining it with the segmentation model, the object contour label and the object area location label are automatically generated; The object detection model is trained using each training image and the contour labels of the objects in the training images and the region location labels where the objects are located.

[0006] The present invention also provides a target detection device, which includes: A first generation module is configured to generate an object model corresponding to the object based on sample images of the same object in different scenes generated using different image generation methods and a current image generation model; A fine-tuning module is used to fine-tune the current image generation model based on the object models corresponding to different objects; The second generation module is configured to determine whether the current image generation model meets the model iteration requirement. If not, the module returns to the step of generating an object model corresponding to the object. If yes, the module then: Generate a training image for training the object detection model based on the current image generation model and using a mask area corresponding to an image area in each training scene image; the mask area is used to indicate the area where the object is located; The third generation module is used to use the mask area corresponding to the object area in each training image and combine it with the segmentation model to automatically generate the object contour label and the area location label where the object is located; The training module is used to train the target detection model by using each training image and the contour labels of the objects and the region position labels of the objects in the training images.

[0007] The present invention also provides a target detection system, which includes: An image acquisition device, used for acquiring scene images; The model device side is a processing device set locally or in the cloud for running the image generation model, which obtains the scene image captured by the image acquisition device and executes the steps in the above method based on the scene image.

[0008] An embodiment of the present application further provides an electronic device, comprising: a processor and a memory for storing computer program instructions, wherein the computer program instructions, when executed by the processor, enable the processor to execute the steps of the above method.

[0009] An embodiment of the present application further provides a machine-readable storage medium, which stores computer program instructions. When the computer program instructions are executed, the steps of the above method can be implemented.

[0010] It can be seen from the above technical solution that in this embodiment, the object model corresponding to the object is first generated with the help of sample images of the same object in different scenes generated by different image generation methods and the current image generation model, and then the current image generation model is fine-tuned based on the object models corresponding to different objects to optimize the current image generation model. If the optimized current image generation model does not meet the model iteration requirements, the step of generating the object model corresponding to the object is returned to the above-mentioned step; if the current image generation model meets the model iteration requirements, the training image for training the target detection model is generated based on the current image generation model and with the help of the mask area corresponding to the image area in each training scene image. This method of dynamically optimizing the image generation model can ensure that the final image generation model is optimal, and then ensure that the training image generated by the optimal image generation model for training the target detection model is optimal, so as to realize the use of the optimal training image generated by the optimal image generation model to train the target detection model, which enriches the training images used to train the detection model and avoids the impact of the lack of training images on the performance of the detection model, the accuracy of target detection, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic flow chart of the method provided in the embodiment of the present application; Figure 2a A schematic diagram of generating image samples provided in an embodiment of the present application; Figure 2b A schematic diagram of generating image samples provided in an embodiment of the present application; Figure 2c A schematic diagram of generating image samples provided in an embodiment of the present application; Figure 2d A schematic diagram of generating image samples provided in an embodiment of the present application; Figure 2e A schematic diagram of generating image samples provided in an embodiment of the present application; Figure 3 A schematic diagram of the device structure provided in an embodiment of the present application; Figure 4 A schematic diagram of the system structure provided in an embodiment of the present application; Figure 5 A schematic diagram of the electronic device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0013] It should be noted that the target detection method provided in the embodiments of the present application is applicable to multiple fields such as the detection of dangerous animals and the detection of target faces. For the sake of convenience, the present application describes the method provided in the present application using the detection of dangerous animals.

[0014] See also Figure 1 , Figure 1 The method flow diagram provided in the embodiment of the present application is applied to an electronic device, which may be a server, and is not specifically limited in the present application.

[0015] like Figure 1 As shown, the process may include the following steps: S101 : generating an object model corresponding to the object based on sample images of the same object in different scenes generated by using different image generation methods and a current image generation model.

[0016] Here, the object is a designated dangerous animal, such as a wild boar or venomous snake, or a designated occluding object, such as a table or chair. Using different image generation methods to generate sample images of the same object in different scenes can effectively enrich the sample images and ensure their diversity, thereby optimizing the current image generation model.

[0017] In this embodiment, the current image generation model is the initial image generation model or the image generation model obtained after a previous fine-tuning using each object model. As an example, the initial image generation model is obtained by training a neural network model (e.g., a diffusion model) using raw images, including dangerous animals, directly captured by an image acquisition device (e.g., a camera) in different scenes.

[0018] In this embodiment, the specific implementation method of generating sample images of the same object in different scenes using different image generation methods and generating the object model corresponding to the object will be explained later and will not be repeated here.

[0019] S102: Fine-tune the current image generation model based on object models corresponding to different objects.

[0020] In this embodiment, the specific implementation of step S102 will be described later and will not be repeated here.

[0021] S103: Determine whether the current image generation model meets the model iteration requirements.

[0022] If the execution result of step S103 is no, then return to the above step S101; if the execution result of step S103 is yes, then execute the following step S104.

[0023] Here, the current image generation model in step S103 is different from the current image generation model mentioned in step S101 and step S102, and is fine-tuned compared with the previous current image generation model.

[0024] The specific implementation of step S103 will be described later and will not be repeated here.

[0025] S104: Generate a training image for training the target detection model based on the current image generation model and with the help of the mask area corresponding to the image area in each training scene image.

[0026] In this embodiment, the mask region is obtained by marking the image region in the scene image or smearing the image region, and the mask region is used to indicate the region where the object is located. The current image generation model here is an image generation model that meets the model iteration requirements.

[0027] The specific implementation of step S104 will be described later and will not be repeated here.

[0028] S105 , using the mask area corresponding to the object area in each training image and combining it with the segmentation model, automatically generate the object's contour label and the object's area position label.

[0029] In this embodiment, the training image is input into the trained segmentation model to automatically generate the contour label of the object and the regional location label of the object. This method of automatically generating contour labels and regional location labels can greatly reduce the workload and cost of manual labeling and improve the generation efficiency of subsequent target detection models.

[0030] S106 , training an object detection model using each training image and the contour labels of the objects and the region position labels where the objects are located in the training images.

[0031] In this embodiment, a large number of training images are generated based on the current image generation model that meets the model iteration requirements, and each training image is input into a neural network model, such as a CNN convolutional model, to obtain the predicted contour and predicted position information output by the model. The predicted contour and predicted position information are used to calculate the loss value with the contour label and position label of the training sample respectively, and the parameters of the neural network model are adjusted according to the loss value until the neural network model meets the conditions for stopping iterative training, and the neural network model that meets the conditions for stopping iterative training is determined as the target detection model.

[0032] So far, completed Figure 1 The process shown.

[0033] pass Figure 1As can be seen from the shown process, the object model corresponding to the object is first generated with the help of sample images of the same object in different scenes generated by different image generation methods and the current image generation model. Then, the current image generation model is fine-tuned based on the object models corresponding to different objects to optimize the current image generation model. If the optimized current image generation model does not meet the model iteration requirements, the step of generating the object model corresponding to the object is returned to the above-mentioned step; if the current image generation model meets the model iteration requirements, the training images for training the target detection model are generated based on the current image generation model and with the help of the mask areas corresponding to the image areas in each training scene image. This method of dynamically optimizing the image generation model can ensure that the final image generation model is optimal, and then ensure that the training images generated by the optimal image generation model for training the target detection model are optimal, so as to realize the use of the optimal training images generated by the optimal image generation model to train the target detection model, which enriches the training images used to train the detection model and avoids the impact of lack of training images on the performance of the detection model and the accuracy of target detection.

[0034] The following combination Figures 2a to 2e The sample images of the same object in different scenes generated by using different image generation methods in the above step S101 are described with examples.

[0035] The first way to generate sample images is: For any object, a scene image and a textual description of the object in any scene are input into the current image generation model to generate a sample image of the object in that scene. The scene image can be a real-world image captured by an image acquisition device in various real-world scenarios, such as a courtyard, a playground, or a desert. The object's textual description is used to describe the generation of the object within the mask region corresponding to the image region in the scene image.

[0036] For example, if Figure 2a As shown, the object is a wild boar, the scene image is an image of a courtyard scene, and the text description information of the object is Figure 2a As shown in the figure, “a wild boar needs to be generated in the mask area in the figure”. The scene image and the text description information of the object are input into the current image generation model to obtain a sample image of “a wild boar in the garden”.

[0037] For example, Figure 2a As shown, the object is a venomous snake, the scene image is an image of a courtyard scene, and the text description information of the object is Figure 2a As shown in the figure, “a poisonous snake needs to be generated in the mask area in the figure”. The scene image and the text description information of the object are input into the current image generation model to obtain a sample image of “a poisonous snake in the garden”.

[0038] For example, Figure 2b As shown, the object is a table, the scene image is an image of a courtyard scene, and the text description information of the object is Figure 2b As shown in the figure, “a table needs to be generated in the mask area in the figure”. The scene image and the text description information of the object are input into the current image generation model to obtain a sample image of “a table in the garden”.

[0039] For example, Figure 2b As shown, the object is a chair, the scene image is an image of a courtyard scene, and the text description information of the object is Figure 2b As shown in the figure, “a chair needs to be generated in the mask area in the figure”. The scene image and the object text description information are input into the current image generation model to obtain a sample image of “a chair in the garden”.

[0040] In this sample image generation method, for any object, by combining scene images in different real scenes with the object, sample images of the object in different scenes can be obtained. In particular, images that are difficult to collect directly can be obtained (for example, images such as "a jaguar in a garden" that are difficult to collect), thereby greatly enriching the diversity of image samples.

[0041] The second way to generate sample images is: For any object, a scene image and the acquired object image are input into the current image generation model. The current image generation model fuses the object image with the mask region corresponding to the image region in the scene image to obtain a sample image of the object in the scene. Here, the object image can be a partial cutout obtained from another image.

[0042] For example, if Figure 2c As shown in the figure, the object image is a "wild boar picture" (for example, a partial cutout of a "wild boar" from another image), and the scene image is an image of a garden. This scene image and the "wild boar picture" are input into the current image generation model, resulting in a sample image of "wild boar in a garden."

[0043] In this sample image generation method, scene images in different real scenes can also be combined with the object, thereby greatly enriching the diversity of image samples.

[0044] The third way to generate sample images is: A first image and a second image are fused. The first image and the second image are images of the same scene. The first image and the second image contain different objects, and the image regions containing the different objects intersect. Preferably, the objects in the first image and the second image are of different categories, with one being a dangerous animal and the other being an occluding object. A sample image of the scene is obtained using the fusion result. In the obtained sample image, an occlusion relationship exists between the objects originally present in the first image and the objects originally present in the second image.

[0045] For example, if Figure 2d As shown, the first and second images obtained in a courtyard scene are fused. The first image contains a "wild boar" and the second image contains a "table." The image areas of the "wild boar" and "table" intersect. After fusion, a sample image of "a table occluding a wild boar in a courtyard" is obtained.

[0046] Optionally, in a specific implementation, the first image and the second image may be generated by the first sample image generation method or the second sample image generation method, and the same scene image is used in the generation process.

[0047] In this sample image generation method, sample images with specific occlusion relationships between different objects are constructed, further enriching the diversity of sample images.

[0048] The fourth sample image generation method is: The depth image and the scene guidance generation condition are input into the current image generation model, so that the current image generation model generates a sample image in the scene indicated by the scene guidance generation condition based on the contour information and depth information of two objects in an occlusion relationship in the depth image, and the scene guidance generation condition; there are two objects in the sample image that meet the occlusion relationship.

[0049] Here, the depth image may be extracted from a sample image obtained by using the third sample image generation method, or may be extracted from an image directly acquired by an image acquisition device and including two objects in an occlusion relationship.

[0050] It should be noted that the occlusion relationship between the two objects in the sample image generated above may be the same as or different from the occlusion relationship in the depth image input previously.

[0051] For example, if Figure 2e As shown, the depth image and the scene guidance generation condition of "the scene where the generated object is located is a courtyard" are input into the current image generation model to generate the following Figure 2e The three sample images shown in the figure have three different occlusion relationships between the “table” and the “wild boar”.

[0052] In this sample image generation method, the two objects in the generated sample image present more than one occlusion relationship, which enriches the diversity of the sample image. Subsequent use of the obtained sample images to fine-tune the image generation model can improve the generation ability of the image generation model.

[0053] The above introduces in detail the sample images of the same object in different scenes generated by using different image generation methods.

[0054] The object model corresponding to the object generated in step S101 is described in detail below: After generating sample images corresponding to the object using at least one of the above-mentioned four image generation methods, for any object, on the premise that the parameter value corresponding to at least one parameter attribute matching the object in the current image generation model remains unchanged, the sample images of the object in different scenes are used to train other variable parameter values ​​in the current image generation model, and the trained current image generation model is used as the object model.

[0055] For example, if the object is a venomous snake, after obtaining sample images of the venomous snake in different scenes, the current image generation model has a total of M parameter attributes, from which N parameter attributes are selected (N is less than M). The parameter values ​​corresponding to these N parameter attributes remain unchanged. The current image generation model is trained using sample images of the venomous snake in different scenes. During the iterative training process, the parameter values ​​corresponding to the remaining MN parameters are changed. The iterative training is performed until the iterative training condition is met (for example, the number of iterations reaches a set threshold, such as 20), and the venomous snake model is obtained, denoted as venomous snake lora. In this way, object models of different objects can be obtained, such as wild boar lora, snake lora, grizzly bear lora, gray wolf lora, kangaroo lora, chair lora, table lora, debris lora, trash can, and various types of obstructions lora.

[0056] The object model corresponding to the object generated in step S101 is described in detail above.

[0057] After obtaining the object models of different objects, the current image generation model is fine-tuned based on the object models corresponding to the different objects. The above step S102 is described in detail below.

[0058] In the specific implementation of step S102, as an embodiment, for any parameter attribute, parameter values ​​belonging to the parameter attribute are found from various object models, and a specified operation (here, the specified operation may be an operation such as weighted averaging) is performed on each of the found parameter values ​​to obtain a target parameter value corresponding to the parameter attribute. Based on the target parameter value corresponding to the parameter attribute, the current parameter value of the parameter attribute in the current image generation model is adjusted, for example, by replacing the current parameter value of the parameter attribute with the target parameter value corresponding to the parameter attribute.

[0059] Through the above-mentioned fine-tuning, the generation accuracy of the current image generation model when generating each object can be improved, thereby optimizing the accuracy of the image generated by the current image generation model.

[0060] After fine-tuning the current image generation model in the above manner, it is necessary to determine whether the current image generation model (i.e., the image generation model obtained after executing step S102 above) meets the model iteration requirements. The following describes in detail how to determine whether the current image generation model meets the model iteration requirements: In a specific implementation, as an embodiment, when sample images of the same object in different scenes are generated using different image generation methods in step S101, the method further includes: for any object, based on the description information of the sample images of the object in each scene and based on the current image generation model, generating a test image corresponding to the object according to the description information. Here, the description information is used to describe the object in the sample image and the environment in which the object is located.

[0061] For example, as shown in Figure 2a The description information corresponding to the first image sample in is "there is a wild boar in the courtyard". The description information of the sample image is output to the current image generation model, so that the current image generation model generates an image based on the scene and object described by the description information, and the generated image is determined as the test image corresponding to the object.

[0062] After obtaining the test images corresponding to each object, the specific implementation method of determining whether the current image generation model meets the model iteration requirements is as follows: based on the test images corresponding to each object, test whether the current image generation model meets the accuracy requirements: if not, determine that the current image generation model does not meet the model iteration requirements; if yes, determine that the current image generation model meets the model iteration requirements.

[0063] The specific implementation method of testing whether the current image generation model meets the accuracy requirements based on the test images corresponding to each object can be: for each sample image, determine whether the difference between the sample image and the test image corresponding to the sample image is less than or equal to the set difference threshold. If so, it is determined that the difference condition is met; otherwise, it is determined that the difference condition is not met. If the number of images that meet the above-mentioned difference condition in the sample images of each object obtained is greater than the set number value, it is determined that the current image generation model meets the accuracy requirements; otherwise, it is determined that the current image generation model does not meet the accuracy requirements.

[0064] The above describes in detail how to determine whether the current image generation model meets the model iteration requirements.

[0065] After determining that the current image generation model meets the model iteration requirements, a training image for training the target detection model will be generated based on the current image generation model and with the help of the mask area corresponding to the image area in each training scene image.

[0066] The following describes in detail the generation of a training image for training the object detection model based on the current image generation model in step S103 using mask regions corresponding to image regions in each training scene image. The specific method of generating training samples is at least one of the following methods: The first training image generation method is: For any training object, the training scene image in any training scene and the text description information of the training object are input into the current image generation model, so that the current image generation model obtains the training image of the training object in the training scene; the text description information of the training object is used to describe the generation of the training object in the mask area corresponding to the image area in the training scene image.

[0067] The second training image generation method is: For any training object, the training scene image in any training scene and the obtained training object image are input into the current image generation model, so that the current image generation model fuses the training object image into the mask area corresponding to the image area in the training scene image to obtain the training image of the training object in the training scene.

[0068] The third training image generation method is: fusing the third image and the fourth image; the third image and the fourth image are images of the same training scene, the training objects in the third image and the fourth image are different, and the image regions where the different training objects are located have an intersection; A training image in the training scene is obtained using the fusion result; wherein, in the training image, there is an occlusion relationship between the training object originally existing in the third image and the training object originally existing in the fourth image.

[0069] The fourth training image generation method is: The depth image and the scene guidance generation condition are input into the current image generation model, so that the current image generation model generates a sample image in the scene indicated by the scene guidance generation condition based on the contour information and depth information of two objects with an occlusion relationship in the depth image, and the scene guidance generation condition; there are two objects with an occlusion relationship in the sample image.

[0070] The generation methods of the above four training images are similar to the generation methods of the four sample images mentioned above, and will not be repeated here.

[0071] This approach generates diverse training samples, significantly enriching the training images for the target detection model and effectively improving its accuracy. Furthermore, by fully accounting for the effects of occlusions and interference from different scenarios, these training samples are used to train the target detection model, enabling it to better learn how to accurately detect dangerous animals in the presence of occlusions and complex scenes. This improves the target detection model's detection capabilities in these conditions, further enhancing its accuracy.

[0072] The above describes in detail the generation of training images for training object detection models.

[0073] After obtaining the training samples in the above manner, it is necessary to perform the above steps S105 and S106 to obtain the target detection model.

[0074] After obtaining the target detection model, the method further includes obtaining performance evaluation parameters for the target detection model in a specified scenario. If the performance evaluation parameters do not meet the set evaluation parameter requirements, optimized images of different training objects in the specified scenario are generated using the current image training model that meets the model iteration requirements (the generation method is similar to the generation method of the four sample images described above and will not be repeated here). The target detection model is optimized using these optimized images.

[0075] In this embodiment, the effect evaluation parameters of the target detection model in the specified scenario are collected, and the effect evaluation parameters are used to guide the generation direction of subsequent training images, so as to continuously obtain high-quality training images, and then the target detection model is optimized in the direction of improving the detection accuracy of the specified scenario, thereby improving the adaptability of the target detection model.

[0076] The above describes the method provided in the embodiment of the present application. The following describes the device provided in the embodiment of the present application: See also Figure 3 , Figure 3 This is a diagram of the structure of the device provided in the embodiment of the present application. The device is applied to electronic equipment, such as Figure 3 As shown, the apparatus may include: a first generating module 301 , a fine-tuning module 302 , a second generating module 303 , a third generating module 304 , and a training module 305 .

[0077] A first generation module 301 is configured to generate an object model corresponding to the object based on sample images of the same object in different scenes generated using different image generation methods and a current image generation model; A fine-tuning module 302 is configured to fine-tune the current image generation model based on object models corresponding to different objects; The second generation module 303 is used to determine whether the current image generation model meets the model iteration requirement. If not, it returns to the step of generating the object model corresponding to the object; if yes, it then: Generate a training image for training the object detection model based on the current image generation model and using a mask area corresponding to the image area in each training scene image; the mask area is used to indicate the area where the object is located; The third generation module 304 is used to automatically generate the object's contour label and the object's location label by using the mask area corresponding to the object area in each training image and combining it with the segmentation model; The training module 305 is used to train the object detection model using the training images and the contour labels of the objects and the region position labels of the objects in the training images.

[0078] As an embodiment, generating sample images of the same object in different scenes using different image generation methods includes: Generate sample images of the object in different scenes using at least one of the following image generation methods: The first image generation method is: For any object, a scene image in any scene and text description information of the object are input into the current image generation model, so that the current image generation model obtains a sample image of the object in the scene; the text description information of the object is used to describe the generation of the object in the mask area corresponding to the image area in the scene image; The second image generation method is: For any object, a scene image in any scene and the obtained object image are input into the current image generation model, so that the current image generation model fuses the object image into the mask area corresponding to the image area in the scene image to obtain a sample image of the object in the scene; The third image generation method is: The first image and the second image are fused; the first image and the second image are images of the same scene, objects in the first image and the second image are different, and image regions where the different objects are located have an intersection; A sample image of the scene is obtained using the fusion result; wherein, in the sample image, an object originally existing in the first image and an object originally existing in the second image have an occlusion relationship; The fourth image generation method is: The depth image and the scene guidance generation condition are input into the current image generation model, so that the current image generation model generates a sample image in the scene indicated by the scene guidance generation condition based on the contour information and depth information of two objects with an occlusion relationship in the depth image, and the scene guidance generation condition; there are two objects with an occlusion relationship in the sample image.

[0079] As an embodiment, based on sample images of the same object in different scenes generated using different image generation methods and the current image generation model, generating an object model corresponding to the object includes: For any object, under the premise that the parameter value corresponding to at least one parameter attribute matching the object in the current image generation model remains unchanged, the sample images of the object in different scenes are used to train other variable parameter values ​​in the current image generation model, and the trained current image generation model is used as the object model.

[0080] As an embodiment, fine-tuning the current image generation model based on object models corresponding to different objects includes: For any parameter attribute, find the parameter value belonging to the parameter attribute from each object model, and perform the specified operation on each parameter value found to obtain the target parameter value corresponding to the parameter attribute; Depending on the target parameter value corresponding to the parameter attribute, the current parameter value under the parameter attribute in the current image generation model is adjusted.

[0081] As an embodiment, generating a training image for training an object detection model based on the current image generation model and using a mask area corresponding to an image area in each training scene image includes: For any training object, a training scene image in any training scene and text description information of the training object are input into the current image generation model, so that the current image generation model obtains a training image of the training object in the training scene; the text description information of the training object is used to describe the generation of the training object in the mask area corresponding to the image area in the training scene image; and / or, For any training object, a training scene image in any training scene and an acquired training object image are input into the current image generation model, so that the current image generation model fuses the training object image into a mask region corresponding to an image region in the training scene image, thereby obtaining a training image of the training object in the training scene; and / or, fusing the third image and the fourth image; the third image and the fourth image are images of the same training scene, the training objects in the third image and the fourth image are different, and the image regions where the different training objects are located have an intersection; A training image of the training scene is obtained using the fusion result; wherein, in the training image, there is an occlusion relationship between the training object originally existing in the third image and the training object originally existing in the fourth image; and / or, The depth image and the training scene guidance generation conditions are input into the current image generation model, so that the current image generation model generates a training image under the training scene indicated by the training scene guidance generation conditions based on the contour information and depth information of two training objects that have an occlusion relationship in the depth image, and the training scene guidance generation conditions; there are two training objects that meet the occlusion relationship in the training image.

[0082] As an embodiment, the object is a designated dangerous animal or a designated obstruction.

[0083] As an embodiment, after obtaining the target detection model, the training module is further used to: Obtain the performance evaluation parameters of the target detection model in a specified scenario; If the effect evaluation parameters do not meet the set evaluation parameter requirements, the current image training model that meets the model iteration requirements is used to generate optimized images of different training objects in the specified scene; Utilize the optimized images to optimize the target detection model.

[0084] As an embodiment, generating sample images of the same object in different scenes using different image generation methods further includes: For any object, based on the description information of the sample image of the object in each scene and based on the current image generation model, the current image generation model generates a test image corresponding to the object according to the description information; the description information is used to describe the object in the sample image and the environment in which the object is located.

[0085] As an embodiment, determining whether the current image generation model meets the model iteration requirement includes: Based on the test images corresponding to each object, it is tested whether the current image generation model meets the accuracy requirements: if not, it is determined that the current image generation model does not meet the model iteration requirements; if yes, it is determined that the current image generation model meets the model iteration requirements.

[0086] So far, completed Figure 3 Structural description of the device shown.

[0087] The following describes the system provided in the embodiment of the present application: See also Figure 4 , Figure 4 This is a diagram of the system structure provided in the embodiment of this application. The system includes: An image acquisition device, used for acquiring scene images; The model device side is a processing device set locally or in the cloud for running the image generation model, which obtains the scene image captured by the image acquisition device and executes the steps in the above method based on the scene image.

[0088] Optionally, there is a communication connection between the image acquisition device and the model device end; The model device side obtains the scene image captured by the image acquisition device, including: receiving the scene image captured by the image acquisition device and sent by the image acquisition device through the communication connection.

[0089] Optionally, the image acquisition device is deployed on the model device end.

[0090] Optionally, the system further includes: a client device; The client device is communicatively connected to the model device end, and is used to send sample images of the object in each scene and corresponding description information to the model device end through the communication connection, so that the model device end generates a test image corresponding to the object based on the description information of the sample images of the object in each scene using the current image generation model according to the description information; the description information is used to describe the object in the sample image and the environment in which the object is located; and the test image is used to test whether the current image generation model meets the model iteration requirements.

[0091] Optionally, the client device is further configured to send text description information of the object to the model device end, so that the model device end inputs the scene image in any scene and the text description information of the object into the current image generation model, so that the current image generation model obtains a sample image of the object in the scene; the text description information of the object is used to describe the generation of the object in the mask area corresponding to the image area in the scene image; and / or, The client device is further configured to send the object image to the model device end, so that the model device end inputs the scene image of any scene and the obtained object image into the current image generation model, so that the current image generation model fuses the object image into the mask area corresponding to the image area in the scene image to obtain a sample image of the object in the scene; and / or, The image acquisition device is also used to acquire depth images; The client is also used to send scene guidance generation conditions to the model device end, so that the model device end inputs the depth image and the scene guidance generation conditions into the current image generation model, so that the current image generation model generates a sample image in the scene indicated by the scene guidance generation conditions based on the contour information and depth information of two objects with an occlusion relationship in the depth image, and the scene guidance generation conditions; there are two objects with an occlusion relationship in the sample image.

[0092] The following examples of the present application provide Figure 3 The hardware structure of the device is described as follows: See Figure 5 , Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0093] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0094] Exemplarily, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0095] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A target detection method based on an image generation model, characterized in that: The method comprises: Based on sample images of the same object in different scenes generated by using different image generation methods and the current image generation model, generate an object model corresponding to the object; Based on the object models corresponding to different objects, fine-tune the current image generation model; Determine whether the current image generation model meets the model iteration requirement, if not, return to the step of generating the object model corresponding to the object; if yes, then: Generate a training image for training a target detection model based on the current image generation model and with the aid of a mask area corresponding to an image area in each training scene image; the mask area is used to indicate an area where an object is located; Using the mask area corresponding to the object area in each training image and combining it with the segmentation model, the contour label of the object and the location label of the area where the object is located are automatically generated; The object detection model is trained using each training image and the contour labels of the objects in the training images and the region location labels where the objects are located.

2. The method according to claim 1, characterized in that The method of generating sample images of the same object in different scenes by using different image generation methods includes: Generate sample images of the object in different scenes according to at least one of the following image generation methods: The first image generation method is: For any object, a scene image in any scene and text description information of the object are input into the current image generation model, so that the current image generation model obtains a sample image of the object in the scene; the text description information of the object is used to describe the generation of the object in the mask area corresponding to the image area in the scene image; The second image generation method is: For any object, a scene image in any scene and an acquired object image are input into the current image generation model, so that the current image generation model fuses the object image into a mask area corresponding to an image area in the scene image, and obtains a sample image of the object in the scene; The third image generation method is: The first image and the second image are fused; the first image and the second image are images of the same scene, the first image and the second image have different objects, and image regions where the different objects are located have an intersection; A sample image of the scene is obtained by using the fusion result; wherein, in the sample image, an object originally existing in the first image and an object originally existing in the second image have an occlusion relationship; The fourth image generation method is: The depth image and the scene guidance generation condition are input into the current image generation model, so that the current image generation model generates a sample image in the scene indicated by the scene guidance generation condition based on the contour information and depth information of two objects in an occlusion relationship in the depth image and the scene guidance generation condition; there are two objects in the sample image that have an occlusion relationship.

3. The method according to claim 1, characterized in that The generating of the object model corresponding to the object based on sample images of the same object in different scenes generated by using different image generation methods and the current image generation model comprises: For any object, under the premise that the parameter value corresponding to at least one parameter attribute matching the object in the current image generation model remains unchanged, sample images of the object in different scenes are used to train other variable parameter values ​​in the current image generation model, and the trained current image generation model is used as the object model.

4. The method according to claim 1 or 3, characterized in that: The fine-tuning of the current image generation model based on the object models corresponding to different objects includes: For any parameter attribute, find the parameter value belonging to the parameter attribute from each object model, and perform the specified operation on each parameter value found to obtain the target parameter value corresponding to the parameter attribute; Depending on the target parameter value corresponding to the parameter attribute, the current parameter value under the parameter attribute in the current image generation model is adjusted.

5. The method according to claim 1, characterized in that The generating a training image for training the target detection model based on the current image generation model and by means of a mask area corresponding to an image area in each training scene image comprises: For any training object, a training scene image in any training scene and text description information of the training object are input into the current image generation model, so that the current image generation model obtains a training image of the training object in the training scene; the text description information of the training object is used to describe the generation of the training object in a mask area corresponding to an image area in the training scene image; and / or, For any training object, a training scene image in any training scene and an acquired training object image are input into a current image generation model, so that the current image generation model fuses the training object image into a mask area corresponding to an image area in the training scene image, thereby obtaining a training image of the training object in the training scene; and / or, The third image and the fourth image are fused; the third image and the fourth image are images of the same training scene, the training objects in the third image and the fourth image are different, and the image regions where the different training objects are located have an intersection; Using the fusion result, a training image in the training scene is obtained; wherein, in the training image, there is an occlusion relationship between the training object originally existing in the third image and the training object originally existing in the fourth image; and / or, The depth image and the training scene guidance generation condition are input into the current image generation model, so that the current image generation model generates a training image under the training scene indicated by the training scene guidance generation condition based on the contour information and depth information of two training objects in an occlusion relationship in the depth image and the training scene guidance generation condition; there are two training objects that meet the occlusion relationship in the training image.

6. The method according to claim 1, characterized in that The object is a designated dangerous animal or a designated obstruction.

7. The method according to claim 1, characterized in that After obtaining the target detection model, the method further comprises: Obtaining effect evaluation parameters of the target detection model in a specified scenario; If the effect evaluation parameter does not meet the set evaluation parameter requirements, then generate optimized images of different training objects in the specified scene by means of the current image training model that meets the model iteration requirements; The object detection model is optimized using each optimized image.

8. The method according to claim 1, characterized in that The method of generating sample images of the same object in different scenes by using different image generation methods further includes: For any object, based on the description information of the sample image of the object in each scene, the current image generation model is used to generate a test image corresponding to the object according to the description information; the description information is used to describe the object in the sample image and the environment in which the object is located.

9. The method according to claim 1 or 8, characterized in that: Determining whether the current image generation model meets the model iteration requirement includes: Based on the test images corresponding to each object, test whether the current image generation model meets the accuracy requirement: if not, determine that the current image generation model does not meet the model iteration requirement; if yes, determine that the current image generation model meets the model iteration requirement.

10. A target detection device, characterized in that: The device comprises: A first generation module, configured to generate an object model corresponding to the object based on sample images of the same object in different scenes generated by using different image generation methods and a current image generation model; A fine-tuning module, used for fine-tuning the current image generation model based on object models corresponding to different objects; The second generation module is used to determine whether the current image generation model meets the model iteration requirement, and if not, return to the step of generating the object model corresponding to the object; if yes, then: Generate a training image for training a target detection model based on the current image generation model and with the aid of a mask area corresponding to an image area in each training scene image; the mask area is used to indicate an area where an object is located; The third generation module is used to use the mask area corresponding to the object area in each training image and combine it with the segmentation model to automatically generate the contour label of the object and the area position label where the object is located; The training module is used to train the target detection model by using each training image and the contour label of the object in the training image and the regional position label of the object.

11. The device according to claim 10, characterized in that The method of generating sample images of the same object in different scenes by using different image generation methods includes: Generate sample images of the object in different scenes according to at least one of the following image generation methods: The first image generation method is: For any object, a scene image in any scene and text description information of the object are input into the current image generation model, so that the current image generation model obtains a sample image of the object in the scene; the text description information of the object is used to describe the generation of the object in the mask area corresponding to the image area in the scene image; The second image generation method is: For any object, a scene image in any scene and an acquired object image are input into the current image generation model, so that the current image generation model fuses the object image into a mask area corresponding to an image area in the scene image, and obtains a sample image of the object in the scene; The third image generation method is: The first image and the second image are fused; the first image and the second image are images of the same scene, the first image and the second image have different objects, and image regions where the different objects are located have an intersection; A sample image of the scene is obtained by using the fusion result; wherein, in the sample image, an object originally existing in the first image and an object originally existing in the second image have an occlusion relationship; The fourth image generation method is: The depth image and the scene guidance generation condition are input into the current image generation model, so that the current image generation model generates a sample image under the scene indicated by the scene guidance generation condition based on the contour information and depth information of two objects in an occlusion relationship in the depth image and the scene guidance generation condition; the sample image contains two objects in an occlusion relationship; and / or, The generating of the object model corresponding to the object based on sample images of the same object in different scenes generated by using different image generation methods and the current image generation model comprises: For any object, under the premise that the parameter value corresponding to at least one parameter attribute matching the object in the current image generation model remains unchanged, sample images of the object in different scenes are used to train other variable parameter values ​​in the current image generation model, and the trained current image generation model is used as the object model; and / or, The fine-tuning of the current image generation model based on the object models corresponding to different objects includes: For any parameter attribute, find the parameter value belonging to the parameter attribute from each object model, and perform the specified operation on each parameter value found to obtain the target parameter value corresponding to the parameter attribute; Depending on the target parameter value corresponding to the parameter attribute, adjusting the current parameter value under the parameter attribute in the current image generation model; and / or, The generating a training image for training the target detection model based on the current image generation model and by means of a mask area corresponding to an image area in each training scene image comprises: For any training object, a training scene image in any training scene and text description information of the training object are input into the current image generation model, so that the current image generation model obtains a training image of the training object in the training scene; the text description information of the training object is used to describe the generation of the training object in a mask area corresponding to an image area in the training scene image; and / or, For any training object, a training scene image in any training scene and an acquired training object image are input into a current image generation model, so that the current image generation model fuses the training object image into a mask area corresponding to an image area in the training scene image, thereby obtaining a training image of the training object in the training scene; and / or, The third image and the fourth image are fused; the third image and the fourth image are images of the same training scene, the training objects in the third image and the fourth image are different, and the image regions where the different training objects are located have an intersection; Using the fusion result, a training image in the training scene is obtained; wherein, in the training image, there is an occlusion relationship between the training object originally existing in the third image and the training object originally existing in the fourth image; and / or, The depth image and the training scene guidance generation condition are input into the current image generation model, so that the current image generation model generates a training image under the training scene indicated by the training scene guidance generation condition based on the contour information and depth information of two training objects in an occlusion relationship in the depth image and the training scene guidance generation condition; the training image contains two training objects that meet the occlusion relationship; and / or, The object is a designated dangerous animal or a designated obstruction; and / or, After obtaining the target detection model, the method further comprises: Obtaining effect evaluation parameters of the target detection model in a specified scenario; If the effect evaluation parameter does not meet the set evaluation parameter requirements, then generate optimized images of different training objects in the specified scene by means of the current image training model that meets the model iteration requirements; Optimizing the target detection model using each optimized image; and / or, The method of generating sample images of the same object in different scenes by using different image generation methods further includes: For any object, based on the description information of the sample image of the object in each scene and based on the current image generation model, the current image generation model generates a test image corresponding to the object according to the description information; the description information is used to describe the object in the sample image and the environment in which the object is located; and / or, Determining whether the current image generation model meets the model iteration requirement includes: Based on the test images corresponding to each object, test whether the current image generation model meets the accuracy requirement: if not, determine that the current image generation model does not meet the model iteration requirement; if yes, determine that the current image generation model meets the model iteration requirement.

12. A target detection system, characterized in that: The system includes: An image acquisition device, used for acquiring scene images; The model device side is a processing device set locally or in the cloud for running the image generation model, which obtains the scene image captured by the image acquisition device and executes the steps in the method described in any one of claims 1 to 9 based on the scene image.

13. The target detection system according to claim 12, characterized in that: There is a communication connection between the image acquisition device and the model device end; The model device side obtains the scene image captured by the image acquisition device, including: receiving the scene image captured by the image acquisition device and sent by the image acquisition device through the communication connection.

14. The target detection system according to claim 12, characterized in that: The image acquisition device is deployed on the model device end.

15. The target detection system according to claim 13 or 14, characterized in that: The system also includes: a client device; The client device is communicatively connected with the model device end, and is used to send sample images of the object in each scene and corresponding description information to the model device end through the communication connection, so that the model device end generates a test image corresponding to the object based on the description information of the sample images of the object in each scene and the description information using the current image generation model; the description information is used to describe the object in the sample image and the environment in which the object is located; and the test image is used to test whether the current image generation model meets the model iteration requirements.

16. The target detection system according to claim 15, characterized in that: The client device is also used to send the text description information of the object to the model device end, so that the model device end inputs the scene image in any scene and the text description information of the object into the current image generation model, so that the current image generation model obtains a sample image of the object in the scene; the text description information of the object is used to describe the generation of the object in the mask area corresponding to the image area in the scene image; and / or, The client device is further used to send the object image to the model device end, so that the model device end inputs the scene image in any scene and the obtained object image into the current image generation model, so that the current image generation model fuses the object image into the mask area corresponding to the image area in the scene image to obtain a sample image of the object in the scene; and / or, The image acquisition device is also used to acquire depth images; The client is also used to send scene guidance generation conditions to the model device end, so that the model device end inputs the depth image and the scene guidance generation conditions into the current image generation model, so that the current image generation model generates a sample image in the scene indicated by the scene guidance generation conditions based on the contour information and depth information of two objects in an occlusion relationship in the depth image, and the scene guidance generation conditions; there are two objects in the sample image that have an occlusion relationship.

17. An electronic device, characterized in that: The electronic device includes: Processor; and A computer-readable storage medium, in which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 9.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, enable the processor to perform the steps of the method according to any one of claims 1 to 9.

Images