A target detection model training method and device, equipment and medium

Abstract

Embodiments of the present application provide a target detection model training method and device, equipment and medium, to solve the problem that the detection accuracy of real sample images is limited in the prior art. In the method, the sample image and the target position of the target object contained therein are obtained; the sample image and the corresponding scene information are input into the target detection model, and the predicted position of the target object in the sample image output by the target detection model is obtained; the sub-loss value is determined according to the predicted position and the target position of the sample image; the sub-loss value is adjusted according to the weight corresponding to the scene information to which the sample image belongs, wherein the weight corresponding to the real scene is greater than the weight corresponding to the default scene; the loss value is determined, and the target detection model is adjusted. The weight corresponding to the real sample image is greater than the weight corresponding to the reference sample image, and more attention is paid to the influence of the real sample image on the target detection model, thereby improving the detection accuracy of the target detection model during detection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for training a target detection model. Background Technology

[0002] Currently, in some scenarios, detection models are used to detect targets, such as... Figure 1 This paper illustrates the target detection process in a traditional application scenario. The acquired image is input into a detection model, which then outputs the target's location and category. This approach avoids manual inspection and improves efficiency. For example, in a kitchen / restaurant setting, to meet safety standards, it's necessary to determine whether chefs are wearing masks and hats. Masks and hats can be used as targets, and the aforementioned detection model can detect them. This allows for early detection of operational errors in the kitchen / restaurant, providing intelligent detection and early warning, improving management efficiency, reducing supervision costs, effectively avoiding manual inspections, and solving the problems of low efficiency and high cost associated with manual inspections.

[0003] To make the output of the above detection model more accurate, currently, methods such as... Figure 2 The current system framework shown involves training the model based on reference sample images to obtain a reference model, inputting real sample images into the reference model, fine-tuning the reference model, and training to obtain the final model.

[0004] However, the number of reference sample images in this method is much larger than that of real sample images, which results in the real sample images not performing well when fine-tuning the reference model. Therefore, the improvement of detection accuracy is limited when the final model is used to detect real sample images in practical applications. Summary of the Invention

[0005] This application provides a training method, apparatus, device, and medium for a target detection model, which addresses the problem of limited improvement in detection accuracy when detecting real sample images in the prior art.

[0006] In a first aspect, embodiments of this application provide a method for training an object detection model, the method comprising:

[0007] Obtain each sample image in the sample set and the target location of the target object contained in each sample image. The sample images include reference sample images and real sample images, and the scene information to which the sample images belong is recorded. The scene information includes the default scene and the real scene.

[0008] Each sample image and its corresponding scene information are input into the target detection model to obtain the predicted location of the target object in each sample image output by the target detection model.

[0009] For each sample image, a sub-loss value is determined based on the predicted location and target location corresponding to the sample image, and the sub-loss value is adjusted according to the weight corresponding to the scene information to which the sample image belongs, wherein the weight corresponding to the real scene is greater than the weight corresponding to the default scene.

[0010] Based on each adjusted sub-loss value, a loss value is determined, and the target detection model is adjusted based on the loss value.

[0011] Secondly, embodiments of this application provide a target detection method, the method comprising:

[0012] Acquire the image to be detected;

[0013] The image to be detected and the real scene information to which the image belongs are input into the trained target detection model to obtain the target detection location of the target object in the image to be detected under the real scene information output by the target detection model.

[0014] Thirdly, embodiments of this application also provide a training apparatus for an object detection model, the apparatus comprising:

[0015] The first acquisition module is used to acquire each sample image contained in the sample set and the target location of the target object contained in each sample image. The sample images include reference sample images and real sample images, and the scene information to which the sample images belong is recorded. The scene information includes the default scene and the real scene.

[0016] The input module is used to input each sample image and its corresponding scene information into the target detection model, and to obtain the predicted location of the target object in each sample image output by the target detection model.

[0017] The determination module is used to determine a sub-loss value for each sample image based on the predicted location and target location corresponding to the sample image, and to adjust the sub-loss value according to the weight corresponding to the scene information to which the sample image belongs, wherein the weight corresponding to the real scene is greater than the weight corresponding to the default scene.

[0018] The training module is used to determine the loss value based on each adjusted sub-loss value and to adjust the object detection model based on the loss value.

[0019] Fourthly, embodiments of this application also provide a target detection device, the device comprising:

[0020] The second acquisition module is used to acquire the image to be detected;

[0021] The detection module is used to input the image to be detected and the real scene information to which the image to be detected belongs into the trained target detection model, and obtain the target detection position of the target object in the image to be detected under the real scene information output by the target detection model.

[0022] Fifthly, embodiments of this application also provide an electronic device, which includes at least a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of training the target detection model as described in any of the preceding claims, or to implement the steps of the target detection method as described in any of the preceding claims.

[0023] Sixthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the training method for the target detection model as described in any of the preceding claims, or performs the steps of the target detection method as described in any of the preceding claims.

[0024] In this embodiment, each sample image in the sample set and the target location of the target object contained in each sample image are obtained. The sample images include reference sample images and real sample images, and the scene information to which the sample images belong is recorded. The scene information includes a default scene and a real scene. Each sample image and its corresponding scene information are input into the target detection model to obtain the predicted location of the target object in each sample image output by the target detection model. For each sample image, a sub-loss value is determined based on the predicted location and target location corresponding to the sample image, and the sub-loss value is adjusted according to the weight corresponding to the scene information to which the sample image belongs, wherein the weight corresponding to the real scene is greater than the weight corresponding to the default scene. Based on each adjusted sub-loss value, a loss value is determined, and the target detection model is adjusted based on the loss value. By training the object detection model using both reference and real sample images, the model takes into account the influence of both images during training. Scene information is used to distinguish between the reference and real sample images. The loss value is adjusted based on the weights corresponding to the default scene in the reference image and the weights corresponding to the real scene in the real sample image. The model is then trained using this adjusted loss value. Furthermore, since the weights corresponding to real sample images are greater than those corresponding to reference image images, the model pays more attention to the influence of real sample images on the object detection model, thus improving the detection accuracy of the trained model when detecting real images. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the target detection process in traditional application scenarios;

[0027] Figure 2 This is a schematic diagram of the current system framework;

[0028] Figure 3 A schematic diagram illustrating the training process of an object detection model provided in an embodiment of this application;

[0029] Figure 4 This is a schematic diagram of a target detection model structure provided in an embodiment of this application;

[0030] Figure 5 This is a schematic diagram of a target detection process provided in an embodiment of this application;

[0031] Figure 6 A schematic diagram of a training device for an object detection model provided in an embodiment of this application;

[0032] Figure 7 This is a schematic diagram of a target detection device provided in an embodiment of this application;

[0033] Figure 8 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, a further detailed description will be provided below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0035] This application provides a training method, apparatus, device, and medium for an object detection model. The method involves acquiring each sample image in a sample set and the target location of the target object contained in each sample image. The sample images include reference sample images and real sample images, and the scene information to which the sample images belong is recorded, including a default scene and a real scene. Each sample image and its corresponding scene information are input into the object detection model to obtain the predicted location of the target object in each sample image output by the object detection model. For each sample image, a sub-loss value is determined based on the predicted location and target location of the sample image, and the sub-loss value is adjusted according to the weight corresponding to the scene information to which the sample image belongs, wherein the weight corresponding to the real scene is greater than the weight corresponding to the default scene. Based on each adjusted sub-loss value, a loss value is determined, and the object detection model is adjusted based on the loss value. By training the object detection model using both reference and real sample images, the model takes into account the influence of both images during training. Scene information is used to distinguish between the reference and real sample images. The loss value is adjusted based on the weights corresponding to the default scene in the reference image and the weights corresponding to the real scene in the real sample image. The model is then trained using this adjusted loss value. Furthermore, since the weights corresponding to real sample images are greater than those corresponding to reference image images, the model pays more attention to the influence of real sample images on the object detection model, thus improving the detection accuracy of the trained model when detecting real images.

[0036] Example 1:

[0037] Figure 3 This application provides a schematic diagram of the training process of an object detection model, which includes:

[0038] S301: Obtain each sample image contained in the sample set and the target location of the target object contained in each sample image. The sample images include reference sample images and real sample images, and the scene information to which the sample images belong is recorded. The scene information includes the default scene and the real scene.

[0039] The target detection model training method provided in this application embodiment is applied to an electronic device, which may be a personal computer (PC), a server, an image acquisition device such as a camera, etc.

[0040] The electronic device acquires each sample image contained in the sample set, wherein the sample set contains one or more sample images, and each sample image is marked with a target object; for each sample image, the target position of the target object in the sample image is obtained based on the target object marked in the sample image.

[0041] The target objects labeled in the sample images include the following two cases:

[0042] In the first scenario, when the target detection model can detect a target object, the target location of the target object is marked in each sample image in the sample set; in this embodiment, the target location of the target object in the sample image is obtained based on the marked sample image.

[0043] Scenario 2: When the target detection model can detect multiple target objects, each sample image in the sample set is marked with the target location and target category of the target object; then, in this embodiment of the application, the target location and target category of the target object in the sample image are obtained based on the marked sample image.

[0044] The sample images in the sample set include reference sample images and real sample images, and the reference sample image can be represented as: A real sample image can be represented as For example, the reference sample image can be a high-resolution image stored on the server, such as a high-resolution image like a wallpaper containing the target object. The real sample image is an image captured by the image acquisition device for the target object in a real-world scene.

[0045] Since the reference sample image is a saved high-resolution image, and the real sample image is an acquired image, it can be understood that in this embodiment, the electronic device acquiring each sample image contained in the sample set includes: if the electronic device is not an image acquisition device, the electronic device can acquire its own saved reference sample image and acquire the real sample image from the image acquisition device to determine the sample images contained in the sample set. If the electronic device is an image acquisition device, the image acquisition device can acquire the reference sample image saved on the server from a server or other device, and acquire the real sample image it has acquired to determine the sample images contained in the sample set.

[0046] Furthermore, when acquiring reference sample images and real sample images from the sample set, considering that the acquisition and annotation of real sample images are more difficult and time-consuming than that of reference sample images, this embodiment of the application uses a large number of reference sample images and a small number of real sample images to reduce the large workload of acquisition and annotation caused by using a large number of real sample images. Here, a small sample typically refers to a sample with a sample size of less than or equal to 30.

[0047] It is understood that the sample images in the sample set can be unprocessed original images, or they can be images obtained after processing the original images. For example, in the embodiments of this application, the size of the original image corresponding to each sample image can be the same or different. Therefore, in order to ensure that the sample images can be input into the target detection model, in this step, for each sample image, the electronic device can process the original image to obtain the sample image in ways including but not limited to the following: Method 1, cropping the original image to a set size to obtain the sample image; Method 2, compressing the original image to a set size to obtain the sample image; Method 3, inputting the original image into a convolutional layer, through which the convolutional layer extracts features and resizes the original image to obtain a sample image of the set size. It is understood that the sample images obtained after processing by this convolutional layer have the same size. Here, the set size is the size of the input image of the target detection model. For example, the width × height of the set size can be 224 × 224, or 256 × 256, etc., without specific limitations here.

[0048] After acquiring sample images from the sample set, the scene information to which the recorded sample images belong is determined in the electronic device. This scene information is determined based on the scene where the sample image is located and the scene information determination rules stored in the electronic device for the sample image. The scene information to which the sample image belongs includes the default scene corresponding to the reference sample image and the real scene corresponding to the real sample image. The scene information determination rules stored in the electronic device for the sample image include: for a reference sample image in the sample set, obtaining the default scene stored in the electronic device for that reference sample image; for a real sample image in the sample set, determining the real scene corresponding to that real sample image based on the scene where the real sample image is located. There can be multiple real scenes. Specifically, based on the scene where the real sample image is located, the type of scene for all real sample images is determined, with each type of scene corresponding to one real scene; for each real sample image, the real scene corresponding to that real sample image is determined based on the scene where the real sample image is located and the correspondence between the scene type and the real scene. It can be understood that, taking a kitchen / restaurant as an example, different real scenes can include, but are not limited to, a five-star hotel kitchen, a three-star hotel kitchen, and a street-side shop kitchen.

[0049] For example, scene information can be represented by location identification information.

[0050] Another example is that scene information can be represented using positional encoding. In one possible implementation, the positional encoding can be a row vector, and the length of this row vector (the number of elements in the row vector) can be the same as the width of the sample image matrix. For example, when the width × height of the sample image is 256 × 224, the number of elements in the positional encoding is 256. In yet another possible implementation, the positional encoding can be a column vector, and the length of this column vector (the number of elements in the column vector) can be the same as the height of the sample image matrix. For example, when the width × height of the sample image is 256 × 224, the number of elements in the positional encoding is 224.

[0051] Taking the representation of scene information by location coding as an example, it can be understood that since location coding represents scene information, and the scene information is determined based on the scene where the sample image is located, the values ​​of each element in the location coding can be determined based on the scene where the sample image is located and the location coding determination rules stored in the electronic device, making it convenient to use this location coding to represent scene information.

[0052] The location encoding determination rules stored in the electronic device may include: when the sample image is a reference sample image, the value of the first element in the location encoding corresponding to the reference sample image is determined as a first preset value, and the values ​​of other elements in the location encoding are determined as second preset values; when the sample image is a real sample image, and the number of scene types in which the real sample image is located is N, each scene is designated as scene 1, scene 2, ..., scene N, and when the scene type of one of the real sample images is the nth (n≤N)th type, i.e., scene n, the value of the (n+1)th element in the location encoding corresponding to the real sample image is determined as the first preset value, and the values ​​of other elements in the location encoding are determined as second preset values. It can be understood that the number of scene types in which the real sample image is located is generally much smaller than the width of the sample image matrix. Here, N and n are positive integers; the first preset value and the second preset value are not the same. For example, the first preset value can be a non-zero value (e.g., the first preset value can be 1), and the second preset value can be 0.

[0053] Taking a positional encoding as a row vector, with a first preset value of 1 and a second preset value of 0 as an example, when the sample image is a reference sample image, the positional encoding corresponding to the reference sample image is [1, 0, ..., 0]. When the sample image is a real sample image, and when N=1, that is, when the number of scene types in which the real sample image is located is one, then the positional encoding corresponding to the real sample image is determined to be [0, 1, 0, ..., 0]. When N=2, that is, when the number of scene types in which the real sample image is located is two, then when the scene in which the real sample image is located is scene 1, the positional encoding corresponding to the real sample image is [0, 1, 0, ..., 0]. When the scene in which the real sample image is located is scene 2, the positional encoding corresponding to the real sample image is [0, 0, 1, 0, ..., 0]. It can be understood that the length of the above positional encoding is the same as the width of the real sample image matrix, which will not be elaborated here.

[0054] S302: Input each sample image and its corresponding scene information into the target detection model to obtain the predicted location of the target object in each sample image output by the target detection model.

[0055] After obtaining the sample images in the sample set and the scene information to which the sample images belong, for case one of the target objects marked in the sample images in S301, that is, when the target detection model can detect the location information of the target object, each sample image in the sample set and the corresponding scene information are input into the target detection model. Based on the target detection model, the predicted position of the target object in each sample image output by the target detection model is determined and obtained.

[0056] For the second case of the target object marked in the sample image in S301, that is, when the target detection model can detect the location information and category of the target object, each sample image in the sample set is input into the target detection model. Based on the target detection model, the predicted location and predicted category of the target object in each sample image output by the target detection model are determined and obtained.

[0057] S303: For each sample image, determine a sub-loss value based on the predicted and target locations corresponding to that sample image, and adjust the sub-loss value according to the weights corresponding to the scene information to which the sample image belongs. The weights corresponding to the real scene are greater than the weights corresponding to the default scene.

[0058] When the target object labeled in the sample image is case one in S301, that is, when the target detection model detects the location information of the target object, for each sample image, the distance between the predicted position and the target position is determined according to the predicted position and the target position corresponding to the sample image, and the sub-loss value of the sample image is determined.

[0059] When the target object labeled in the sample image is Case 2 in S301, that is, when the target detection model detects the location information and category of the target object, for each sample image, based on the predicted location, target location, predicted category and target category corresponding to the sample image, the distance between the predicted location and the target location, the distance between the predicted category and the target category, and the sub-loss value of the sample image are determined.

[0060] After determining the sub-loss value for each sample image, the weight of each sample image and the sub-loss value can be multiplied according to the weight corresponding to the scene information to which each sample image belongs, and the sub-loss value of each sample image can be adjusted.

[0061] In this approach, regardless of whether a real sample image belongs to one or more real-world scenes, the weight corresponding to the real-world scene to which each real sample image belongs is greater than the weight corresponding to the default scene to which the reference sample image belongs. The reference sample image has the largest data volume and the smallest weight, which allows for the preservation of the features of the reference sample image without excessively affecting the detection performance in the real-world scene.

[0062] In this embodiment, when training the object detection model, a large weight is given to a small number of real sample images, focusing most of the attention on the real sample images. This makes the output of the object detection model closer to the real scene. Conversely, a small weight is given to a large number of reference sample images, focusing only a portion of the attention on them. This preserves the information learned from the reference sample images while maintaining sufficient data volume. Furthermore, the use of cross-domain features from both reference and real sample images increases the model's robustness. Here, "cross-domain" means that the reference and real sample images are from different domains.

[0063] S304: Determine the loss value based on each adjusted sub-loss value, and adjust the target detection model based on the loss value.

[0064] Based on the sub-loss values ​​adjusted for each sample image, the loss value corresponding to the object detection model can be determined by summing them. Then, based on this loss value, the object detection model can be trained by adjusting the parameters in the object detection model.

[0065] The loss value satisfies the following formula:

[0066]

[0067] in, Indicates the loss value. This indicates the weight corresponding to the default scenario. This represents the sub-loss value of the reference sample image. ... This represents the weight corresponding to the real-world scenario, and Indicates the first Weights corresponding to real-world scenarios ... Represents the sub-loss value of the real sample image, and The real-world scenario is represented as the first... Sub-loss values ​​of real sample images in a real-world scenario; Indicates the default scenario. ... Represents a real-world scenario, and Indicates the first A number of real-world scenarios. The weights corresponding to the default scenario are as follows. Less than the weight corresponding to the real scene ... Any one of them, and in one possible implementation, , ... The sum of is 1, where It is a positive integer greater than 1.

[0068] In this embodiment, the target detection model is trained using both reference sample images and real sample images. This ensures that the target detection model considers the influence of both reference and real sample images during training. The model distinguishes between reference and real sample images using scene information. Based on the weights corresponding to the default scene of the reference sample images and the weights corresponding to the real scene of the real sample images, the loss value is adjusted. The target detection model is then trained based on the adjusted loss value. Since the weights corresponding to real sample images are greater than those corresponding to reference sample images, the model pays more attention to the influence of real sample images on the target detection model, thereby improving the detection accuracy of the trained target detection model when detecting real images to be detected.

[0069] Example 2:

[0070] To further improve the detection accuracy of the target detection model, based on the above embodiments, in this embodiment, each sample image and its corresponding scene information are input into the target detection model to obtain the predicted location of the target object in each sample image output by the target detection model, including:

[0071] For any sample image, the object detection model determines the first shallow features of the sample image based on the sample image and the corresponding scene information, wherein the first shallow features include the color features of the sample image under the scene information; and determines the first deep features of the sample image, wherein the first deep features include the spatial features of the sample image; based on the first shallow features and the first deep features, determines the first relational features corresponding to the color features and spatial features of the sample image under the scene information; and based on the first relational features and the first deep features, determines the predicted position of the target object in the sample image under the scene information.

[0072] Since the object detection model processes any sample image in the same way, the processing procedure for one sample image will be described in this embodiment.

[0073] Because the color distribution difference between the reference sample image and the real sample image is too large, the detection accuracy of the trained object detection model is limited. In this embodiment, considering the significant differences in color distribution under different scenes, scene information and shallow features (including color features) can be combined. This not only better expresses the color features of the sample image, but also distinguishes the reference sample image and the real sample image through scene information. This makes it easier to assign different weights to the reference sample image and the real sample image according to different scene information when training the object detection model. This reduces the problem that the detection accuracy of the trained object detection model is limited due to the large color distribution difference between the reference sample image and the real sample image.

[0074] Therefore, refer to Figure 4 The illustrated embodiment of this application provides a schematic diagram of an object detection model structure. A sample image and corresponding scene information are input into the M1 convolutional layer (first convolutional layer) of the object detection model to determine the first shallow features of the sample image. These first shallow features include the color features of the sample image. Since the sample image and corresponding scene information are used as input to the first convolutional layer in the object detection model, the first shallow features output by the first convolutional layer also carry the scene information corresponding to the sample image. In other words, the first shallow features include the color features of the sample image under the scene information corresponding to the sample image. These first shallow features can be represented as F. shallow .

[0075] For example, when the scene information is a location-encoded vector, and the location encoding is a row vector, feature extraction is performed on the sample image based on the sample image to determine the sample image matrix features corresponding to the sample image. The location encoding is then added above the first row of the sample image matrix features to determine the sample image matrix features with coded locations, such as... Figure 4The sample image matrix features with encoded positions are input into the first convolutional layer. Based on the first convolutional layer, the first shallow features corresponding to the sample image are determined.

[0076] like Figure 4 As shown, the sample image is input into the M2 layer (M2 > M1) convolutional layer (second convolutional layer) of the object detection model to determine the first deep feature of the sample image. The first deep feature includes the spatial features of the sample image, which can be represented as F. deep .

[0077] For example, when the scene information is location-encoded, and the location encoding is a row vector, such as... Figure 4 The sample image matrix features are input into the second convolutional layer, and the first deep features corresponding to the sample image are determined based on the second convolutional layer.

[0078] After obtaining the first shallow layer features and the first deep layer features of the sample image, based on these features, such as... Figure 4 As shown, the first shallow layer feature and the first deep layer feature are multiplied by F. shallow F deep The first shallow feature and the first deep feature of the sample image are determined to correspond to the first relational feature, which can be represented as F. shallow-deep-realtion Among them, the first relational feature can represent the spatial relationship corresponding to different color distribution features, that is, the spatial relationship corresponding to different scenes.

[0079] After obtaining the first relational feature, since the first deep feature can represent the location information of the target object and the first relational feature can represent the spatial relationship corresponding to different color distribution features, feature extraction can be performed on the first deep feature and the first relational feature through an attention mechanism to determine the first attention feature F. atten The first attention feature can represent the position of the target object in the spatial relationship corresponding to different color distribution features.

[0080] In one possible implementation, regarding case one in S301, i.e., the case where the target detection model can detect the positional information of the target object, since the first deep feature can characterize the positional information of the target object and the first relational feature can represent the spatial relationship corresponding to different color distribution features, the first attention feature F determined based on the first deep feature and the first relational feature is... atten It can represent the position of the target object in the spatial relationship corresponding to different color distribution features. Then, based on this first attention feature F... atten The predicted location of the target object in the sample image is determined based on the scene information corresponding to the sample image.

[0081] In another possible implementation, regarding case two in S301, i.e., when the object detection model can detect the location information and category of the target object, both the first shallow feature and the first deep feature can characterize the category information of the target object in the sample image. Therefore, the first relation feature determined based on the first shallow feature and the first deep feature can also characterize the category information of the target object, and thus the first attention feature F determined based on the first relation feature and the first deep feature... atten Besides representing the location of the target object in the spatial relationship corresponding to different color distribution features, it can also characterize the category information of the target object. Therefore, based on this first attention feature F... atten This determines the target category of the target object and the predicted location of the target object in the sample image under the scene information corresponding to the sample image.

[0082] In this embodiment, by combining scene information and shallow features, not only can the color features of the sample image be better expressed, but the reference sample image and the real sample image can also be distinguished by the scene information. This makes it easier to assign different weights to the reference sample image and the real sample image according to different scene information when training the target detection model. This reduces the problem that the target detection model's detection accuracy is limited due to the large difference in color distribution between the reference sample image and the real sample image, and further improves the detection accuracy of the target detection model.

[0083] Example 3:

[0084] To further improve the detection accuracy of the target detection model, based on the above embodiments, in this embodiment, the predicted location of the target object in the sample image under scene information is determined according to the first relational feature and the first deep feature, including:

[0085] Based on the first deep feature, obtain the first attention weight; based on the first shallow feature, the first deep feature, the first attention weight, and the attention mechanism algorithm, determine the value corresponding to the first relation feature;

[0086] Based on the first relation feature, the value corresponding to the first relation feature, and the first deep feature, the predicted position under the scene information is determined.

[0087] In this embodiment of the application, in order to make the object detection model pay more attention to color features and spatial features, an attention mechanism is introduced into the object detection model, so that the object detection model pays more attention to the color features and spatial features of the target object and reduces the influence of other unnecessary information on the object detection model.

[0088] After determining the first deep features of the sample image, see Figure 4The first deep features are then passed through another convolutional layer to obtain the first attention weights.

[0089] After obtaining the first attention weight, the first shallow feature is used as the query in the attention mechanism, the first deep feature is used as the key in the attention mechanism, and the first attention weight is used as the value in the attention mechanism. Based on the first shallow feature, the first deep feature, the first attention weight, and the attention mechanism algorithm, the value corresponding to the first relation feature is determined. The value corresponding to the first relation feature can represent the tightness of the relationship between the first shallow feature and the first deep feature. The larger the value corresponding to the first relation feature, the more matched and closely related the first shallow feature and the first deep feature are.

[0090] The attention mechanism algorithm can satisfy the following formula: ,in This represents a query in the attention mechanism. This represents a key in the attention mechanism. Represents the weights in the attention mechanism. Represents the classification function, To represent the transpose of a matrix, It is a constant.

[0091] The value corresponding to the first relation feature can satisfy the following formula:

[0092]

[0093] in, This represents the value corresponding to the first relation feature. This represents the first shallow layer feature. Indicates the first deep feature. This represents the first attention weight obtained by passing the first deep feature through a convolutional layer.

[0094] After obtaining the value corresponding to the first relation feature, the first attention feature is determined based on the first relation feature, the value corresponding to the first relation feature, and the first deep feature; based on this first attention feature F... atten This determines the predicted location of the target object based on the scene information corresponding to the sample image.

[0095] In this embodiment, the attention mechanism enables the target detection model to focus more on the first shallow features and the first deep features, reducing the impact of other unnecessary information on the target detection model and further improving the detection accuracy of the target detection model.

[0096] Example 4:

[0097] To further improve the detection accuracy of the target detection model, based on the above embodiments, in this embodiment, after determining the loss value and before adjusting the target detection model based on the loss value, the method further includes:

[0098] Determine whether the loss value is less than the preset loss value;

[0099] If not, decrease the weight corresponding to the default scene by a first preset value, increase the weight corresponding to each real scene by a second preset value, and determine the sub-loss value corresponding to each sample image based on the adjusted weights, and update the loss value based on each determined sub-loss value.

[0100] After determining the loss value corresponding to the target detection model, it is determined whether the loss value is less than the preset loss value stored in the electronic device. If so, it is determined that the target detection model has a good detection effect, and the training of the target detection model can be terminated.

[0101] If the loss value is not less than the preset loss value, based on the scene information corresponding to the sample image, reduce the weight corresponding to the default scene to which the reference sample image belongs, and reduce the weight corresponding to the default scene by a first preset value to determine the adjusted weight corresponding to the default scene; increase the weight corresponding to the real scene to which the real sample image belongs, and increase the weight corresponding to the real scene by a second preset value to determine the adjusted weight corresponding to the real scene.

[0102] In one possible implementation, the first preset value and the second preset value are values ​​stored in the electronic device.

[0103] In another possible implementation, the electronic device stores a rule for determining a first preset value and a second preset value, and the values ​​of the first preset value and the second preset value can be determined according to the rule for determining the first preset value and the second preset value.

[0104] For example, the rules for determining the first and second preset values ​​stored in the electronic device can be as follows: when the sample image is a reference sample image, the first preset value corresponding to the default scene to which the reference sample image belongs is a specific numerical value; when the sample image is a real sample image, the second preset value is determined based on the sub-loss value corresponding to the real scene to which the real sample image belongs, and based on the magnitude of each sub-loss value; wherein, the larger the sub-loss value, the lower the detection accuracy of the target detection model for the real scene, that is, the higher the requirement for detection accuracy for the real scene, and the higher the requirement for detection accuracy, the larger the weight can be assigned to the scene. Therefore, when the sub-loss value is larger in the real scene, the second preset value stored in the electronic device for the weight corresponding to the sub-loss value in the real scene is larger.

[0105] Since the magnitude of the sub-loss value can represent the detection accuracy corresponding to different scenarios, the higher the detection accuracy requirement of the real scene, the higher the importance of the scene. Therefore, in one possible implementation, the electronic device determines the second preset value of the weight corresponding to each sub-loss value based on its magnitude. This includes: the electronic device determines the ratio of the weights corresponding to the real scenes based on the magnitude of each sub-loss value (i.e., based on the importance of the real scenes), according to the ratio of the magnitudes of the sub-loss values, thereby determining the adjusted weights corresponding to each real scene. Then, based on the adjusted weights and the original weights for each real scene, the second preset value for increasing the weights corresponding to each real scene is determined. For example, considering a specific real-world scenario, when using the same object detection model to detect a five-star hotel kitchen and a street-side shop kitchen, since the detection accuracy requirement for the five-star hotel kitchen is relatively high, a larger weight can be assigned to the five-star hotel kitchen scene, resulting in higher detection accuracy for the trained object detection model when detecting objects in the five-star hotel kitchen scene.

[0106] After determining the adjusted weights, in one possible implementation, for case one in S301, for each sample image, the sub-loss value corresponding to the sample image is determined based on the adjusted weights corresponding to the sample image and the distance between the target position and the predicted position of the sample image.

[0107] In another possible implementation, for case two in S301, for each sample image, the sub-loss value corresponding to the sample image is determined based on the adjusted weights corresponding to the sample image, the distance between the target position and the predicted position of the sample image, and the distance between the target category and the predicted category of the sample image.

[0108] After obtaining the sub-loss value for each sample image, these sub-loss values ​​are summed to determine the updated loss value. The object detection model is then trained based on this updated loss value, and the model parameters are adjusted.

[0109] In this embodiment, the weight ratio corresponding to the real scene is determined according to the importance of the real scene, which facilitates the improvement of the detection accuracy of the target detection model for sample images in real scenes with high importance, and further improves the detection accuracy of the target detection model.

[0110] Based on the above embodiments, the training process of the object detection model will be illustrated with two specific examples:

[0111] Example 1: All real sample images in the sample set belong to the same real scene. Obtain a large number of reference sample images from the sample set. and small sample real sample images Determine the position code for each sample image, where the position code for the reference sample image is [1, 0, ..., 0], and the position code for the real sample image is [0, 1, ..., 0], where the length of the position code is the same as the width of the sample image matrix.

[0112] The sample image matrix features with position encoding are input into two convolutional layers in the object detection model to obtain the first shallow feature with position encoding, which can express scene information and color features; the sample image matrix features without position encoding are input into three convolutional layers in the object detection model to obtain the first deep feature, which can express spatial features.

[0113] An attention mechanism is implemented using the first shallow feature and the first deep feature. The first shallow feature serves as the query in the attention mechanism, and the first deep feature serves as the key. The first shallow feature and the first deep feature are then multiplied. The first relation feature is obtained. The first deep feature is then passed through another convolutional layer to obtain the value in the attention mechanism. Based on the attention mechanism, the first relation feature and the first deep feature are used for feature extraction to obtain the first attention feature. The first attention feature represents the position of the target in the spatial relationship corresponding to different color distribution features; the predicted position of the target object in the sample image is determined based on the first attention feature; a sub-loss value is determined based on the predicted position and the target position; the attention loss is determined based on each sub-loss value and its corresponding weight, and the target detection model is trained. .

[0114] Example 2: All real sample images in the sample set belong to two different real-world scenarios. Obtain a large number of reference sample images from the sample set. and small sample real sample images Determine the position code for each sample image, where the position code for the reference sample image is [1, 0, ..., 0], the position code for the real sample image in real scene 1 is [0, 1, ..., 0], and the position code for the real sample image in real scene 2 is [0, 0, 1, ..., 0]. The length of the position code is the same as the width of the sample image matrix.

[0115] The sample image matrix features with position encoding are input into a single convolutional layer in the object detection model to obtain the first shallow feature with position encoding, which can express scene information and color features. The sample image matrix features without position encoding are input into a four-layer convolutional layer in the object detection model to obtain the first deep feature, which can express spatial features.

[0116] An attention mechanism is implemented using the first shallow feature and the first deep feature. The first shallow feature serves as the query in the attention mechanism, and the first deep feature serves as the key. The first shallow feature and the first deep feature are then multiplied. The first relation feature is obtained. The first deep feature is then passed through another convolutional layer to obtain the value in the attention mechanism. Based on the attention mechanism, the first relation feature and the first deep feature are used for feature extraction to obtain the first attention feature. The first attention feature represents the position of the target in the spatial relationship corresponding to different color distribution features; the predicted position of the target object in the sample image is determined based on the first attention feature; a sub-loss value is determined based on the predicted position and the target position; the attention loss is determined based on each sub-loss value and its corresponding weight, and the target detection model is trained. .

[0117] Example 5:

[0118] Figure 5 This application provides a schematic diagram of a target detection process, which includes:

[0119] S501: Acquire the image to be detected.

[0120] The target detection method provided in this application is applied to an electronic device, which may be a personal computer (PC), a server, or an image acquisition device such as a camera. If the electronic device is not an image acquisition device, it can acquire the image to be detected from an image acquisition device. If the electronic device is an image acquisition device, it can acquire the image to be detected that it has acquired itself.

[0121] The image to be detected can be an unprocessed original image acquired in real time, or it can be an image obtained after processing the original image. The processing method for the original image is the same as that of Method 1, Method 2, and Method 3 in S301, and will not be repeated here. The processed image to be detected is the size of the input image of the target detection model.

[0122] After acquiring the image to be detected, the real scene information to which the image belongs is determined. The process of determining the real scene information to which the image belongs is the same as the process of determining the real scene corresponding to the real sample image in the above embodiment, and will not be described again here.

[0123] For example, real-world scene information can be represented using location identification information. Another example is that real-world scene information can be represented using location encoding. The location encoding can be a row vector or a column vector. When the location encoding is a row vector, its length is the same as the width of the image matrix to be detected; when the location encoding is a column vector, its length is the same as the height of the image matrix to be detected.

[0124] Taking the representation of real-world scene information through location encoding as an example, the process involves determining the scene containing the image to be detected, and then classifying all scenes containing the images to be detected. For instance, if there are N types of scenes, each scene is designated as Scene 1, Scene 2, ..., Scene N. For each image to be detected, the location encoding corresponding to that image is determined based on the type of scene it belongs to. For example, when the scene type of the image to be detected is the nth (n≤N)th type, i.e., Scene n, the value of the (n+1)th element in the location encoding corresponding to that image is determined as the first preset value, and the values ​​of the other elements in the location encoding are determined as the second preset values. It is understandable that the number of scene types containing the images to be detected is much smaller than the width of the image matrix.

[0125] S502: Input the image to be detected and the real scene information to which the image to be detected belongs into the trained target detection model, and obtain the target detection position of the target object in the image to be detected under the real scene information output by the target detection model.

[0126] The target detection model is a model pre-trained according to the target detection model training method provided in the embodiments of this application, and the target detection model can be a model for detecting one type of target object or a model for detecting multiple target objects.

[0127] When the target detection model is designed to detect a specific target object, the image to be detected and the real-world scene information to which the image belongs are input into the trained target detection model. The target detection model then determines the color and spatial features of the image to be detected in combination with the real-world scene information. Based on these color and spatial features, the target detection location of the target object in the image to be detected, as output by the target detection model and given the real-world scene information to which the image belongs, can be determined and obtained.

[0128] When the target detection model is designed to detect multiple target objects, the image to be detected and the real scene information to which the image belongs are input into the trained target detection model. The target detection model determines the color features and spatial features of the image to be detected that are combined with the real scene information. Based on the color features and spatial features combined with the real scene information, the target detection model can determine and obtain the target detection location and target detection type of the target object in the image to be detected, given the real scene information to which the image belongs.

[0129] In this embodiment, the images to be detected are distinguished by real scene information, which makes it easier to determine the target object under different real scene information and improves the detection accuracy of the target object under different scenes.

[0130] Example 6:

[0131] To further improve the detection accuracy of target objects in different scenarios, based on the above embodiments, in this application embodiment, the target detection model outputs the target detection location of the target object in the image to be detected under real scene information, including:

[0132] The object detection model determines the second shallow features of the image to be detected based on the image to be detected and the corresponding real scene information. The second shallow features include the color features of the image to be detected under the real scene information. It also determines the second deep features of the image to be detected, which include the spatial features of the image to be detected.

[0133] Based on the second shallow layer features and the second deep layer, the second relationship features corresponding to the color features and spatial features of the image to be detected under real scene information are determined;

[0134] Based on the second relation feature and the second deep feature, the target detection location information of the target object in the image to be detected is determined under the real scene information.

[0135] Based on the target detection location information, determine the target detection location output by the trained target detection model.

[0136] The image to be detected and its corresponding real-world scene information are input into the M1 convolutional layer (the first convolutional layer) of the object detection model to determine the second shallow features of the image to be detected. These second shallow features include the color features of the image to be detected. Since the image to be detected and its corresponding real-world scene information are used as input to the first convolutional layer of the object detection model, the output of the second shallow features from the first convolutional layer also carries the real-world scene information corresponding to the image to be detected. In other words, the second shallow features include the color features of the image to be detected based on the real-world scene information to which the image belongs. Color features can characterize the approximate location and category information of the target object in the image to be detected.

[0137] For example, when the scene information is a positional encoding and the positional encoding is a row vector, feature extraction is performed on the image to be detected based on the image to be detected, the corresponding image matrix features to be detected are determined, the positional encoding is added above the first row of the image matrix features to be detected, the image matrix features to be detected with encoded positions are determined, the image matrix features to be detected with encoded positions are input into the first convolutional layer, and based on the first convolutional layer, the second shallow features corresponding to the image to be detected are determined.

[0138] like Figure 4 As shown, the image to be detected is input into the M2 layer (M2 > M1) convolutional layer (second convolutional layer) of the object detection model to determine the second deep features of the image. These second deep features include the spatial features of the image. Spatial features can characterize the approximate location and category information of the target object in the image.

[0139] For example, when the scene information is a positional encoding and the positional encoding is a row vector, the feature matrix of the image to be detected is input into the second convolutional layer, and based on the second convolutional layer, the second deep feature corresponding to the image to be detected is determined.

[0140] The processes of determining the second shallow layer features and the second deep layer features can be performed in parallel.

[0141] After obtaining the second shallow feature and the second deep feature of the image to be detected, the second shallow feature and the second deep feature are multiplied together to determine the second relation feature corresponding to the second shallow feature and the second deep feature of the image to be detected. The second relation feature can represent the spatial relationship corresponding to different color distribution features, that is, the spatial relationship corresponding to different real scenes.

[0142] After obtaining the second relational feature, since the second deep feature can characterize the location information of the target object and the second relational feature can represent the spatial relationship corresponding to different color distribution features, the second deep feature and the second relational feature can be used to extract features through the attention mechanism to determine the second attention feature. This second attention feature can represent the location of the target object in the spatial relationship corresponding to different color distribution features.

[0143] In one possible implementation, when the object detection model is designed to detect a specific target object, the second deep feature can characterize the target object's location information, and the second relational feature can represent the spatial relationships corresponding to different color distribution features. Therefore, the second attention feature determined based on the second deep feature and the second relational feature can represent the target object's location within the spatial relationships corresponding to different color distribution features. This second attention feature is then input into the third convolutional layer to determine the target detection location of the target object in the image under the scene information corresponding to the image to be detected. The third convolutional layer can be a single convolutional layer.

[0144] In another possible implementation, when the object detection model is designed to detect multiple target objects, both the second shallow feature and the second deep feature can characterize the category information of the target objects in the image to be detected. Therefore, the second relation feature determined based on the second shallow feature and the second deep feature can also characterize the category information of the target objects. Thus, the attention feature determined based on the second relation feature and the second deep feature can not only represent the position of the target objects in the spatial relationships corresponding to different color distribution features, but also characterize the category information of the target objects. Based on this second attention feature, it is input into the third convolutional layer to determine the target category of the target object and the target detection position of the target object in the image to be detected under the scene information corresponding to the image to be detected.

[0145] The second attention feature mentioned above can be a feature that marks the location of the target object; then, after inputting the second attention feature into the convolutional layer, the target detection location output by the target detection model can be the coordinate point of the target object.

[0146] In this embodiment of the application, by combining scene information and shallow features, not only can the color features of the image to be detected be better expressed, but the scene information can also be used to distinguish the images to be detected under different real scene information, which is conducive to further improving the detection accuracy of the target object by the target detection model under different scenes.

[0147] Example 7:

[0148] To further improve the detection accuracy of the target detection model, based on the above embodiments, in this embodiment, the target detection location of the target object in the image to be detected under real scene information is determined according to the second relation feature and the second deep feature, including:

[0149] Based on the second deep feature, obtain the second attention weight; based on the second shallow feature, the second deep feature, the second attention weight, and the attention mechanism algorithm, determine the value corresponding to the second relation feature;

[0150] Based on the second relation feature, the value corresponding to the second relation feature, and the second deep feature, the target detection location in the real scene information is determined.

[0151] After determining the second deep features of the image to be detected, the second deep features are passed through a convolutional layer to obtain the second attention weights.

[0152] After obtaining the second attention weight, based on the attention mechanism introduced in the object detection model, the second shallow feature is used as the Query in the attention mechanism, the second deep feature is used as the Key in the attention mechanism, and the second attention weight is used as the value in the attention mechanism. According to the second shallow feature, the second deep feature, the second attention weight, and the attention mechanism algorithm, the value corresponding to the second relationship feature is determined. The value corresponding to the second relationship feature can represent the tightness of the relationship between the second shallow feature and the second deep feature. The larger the value corresponding to the second relationship feature, the more matched and closely related the second shallow feature and the second deep feature are, thus the location information of the target object determined based on the second relationship feature is more accurate.

[0153] The value corresponding to the second relation feature can satisfy the following formula:

[0154]

[0155] in, This represents the value corresponding to the second relation feature. This represents the second shallow layer feature. This indicates the second deep feature. This represents the second attention weight obtained by passing the second deep feature through a convolutional layer.

[0156] After obtaining the value corresponding to the second relation feature, a second attention mechanism is determined based on the second relation feature, the value corresponding to the second relation feature, and the second deep feature; based on the second attention mechanism, the target detection position of the target object is determined under the real scene information to which the image to be detected belongs.

[0157] In this embodiment, the attention mechanism enables the target detection model to focus more on the second shallow features and the second deep features, reducing the impact of other unnecessary information on the target detection model when performing target detection, thereby improving the detection accuracy of the target detection model.

[0158] Based on the above embodiments, the target detection process is illustrated with two specific examples:

[0159] Example 1: If the real scene information to which the image to be detected belongs is one type, then the position code of the image to be detected is determined to be [0, 1, ..., 0], where the length of the position code is the same as the width of the matrix of the image to be detected.

[0160] The location-encoded image matrix features are input into two convolutional layers in the object detection model to obtain the second shallow feature layer with location encoding, which can express scene information and color features. The location-unencoded image matrix features are input into three convolutional layers in the object detection model to obtain the second deep feature layer, which can express spatial features.

[0161] An attention mechanism is implemented using the second shallow feature and the second deep feature. The second shallow feature serves as the query in the attention mechanism, and the second deep feature serves as the key. The second shallow feature and the second deep feature are then multiplied. The second relation feature is obtained. The second deep feature is then passed through another convolutional layer to obtain the value in the attention mechanism. Based on the attention mechanism, the second relation feature and the second deep feature are used for feature extraction to obtain the second attention feature. The second attention feature represents the position of the target in the spatial relationship corresponding to different color distribution features; based on the second attention feature, the target detection position corresponding to the detection coordinate point of the target object is obtained by convolution.

[0162] Example 2: If the real scene information to which the image to be detected belongs is given to the target detection model, then the position code of the image to be detected under real scene information 1 is determined to be [0, 1, ..., 0], and the position code of the image to be detected under real scene information 2 is determined to be [0, 0, 1, ..., 0], where the length of the position code is the same as the width of the image matrix to be detected.

[0163] The location-encoded image matrix features are input into a single convolutional layer in the object detection model to obtain a second shallow feature layer with location encoding, which can express scene information and color features. The location-unencoded image matrix features are input into a four-layer convolutional layer in the object detection model to obtain a second deep feature layer, which can express spatial features.

[0164] An attention mechanism is implemented using the second shallow feature and the second deep feature. The second shallow feature serves as the query in the attention mechanism, and the second deep feature serves as the key. The second shallow feature and the second deep feature are then multiplied. The second relation feature is obtained. The second deep feature is then passed through another convolutional layer to obtain the value in the attention mechanism. Based on the attention mechanism, the second relation feature and the second deep feature are used for feature extraction to obtain the second attention feature. The second attention feature represents the position of the target in the spatial relationship corresponding to different color distribution features; based on the second attention feature, the target detection position corresponding to the detection coordinate point of the target object is obtained by convolution.

[0165] Example 8:

[0166] Based on the same technical concept and the above embodiments, this application provides a training device for an object detection model. Figure 6 This is a schematic diagram of a training device for an object detection model provided in an embodiment of this application, as shown below. Figure 6 As shown, the device includes:

[0167] The first acquisition module 601 is used to acquire each sample image contained in the sample set and the target location of the target object contained in each sample image. The sample images include reference sample images and real sample images, and the scene information to which the sample images belong is recorded. The scene information includes the default scene and the real scene.

[0168] The input module 602 is used to input each sample image and the corresponding scene information into the target detection model, and obtain the predicted position of the target object in each sample image output by the target detection model;

[0169] The determination module 603 is used to determine a sub-loss value for each sample image based on the predicted position and target position corresponding to the sample image, and to adjust the sub-loss value according to the weight corresponding to the scene information to which the sample image belongs, wherein the weight corresponding to the real scene is greater than the weight corresponding to the default scene.

[0170] Training module 604 is used to determine the loss value based on each adjusted sub-loss value and to adjust the object detection model based on the loss value.

[0171] In one possible implementation, the input module 602 is specifically configured to, for any sample image, determine, based on the sample image and corresponding scene information, a first shallow feature of the sample image, wherein the first shallow feature includes the color feature of the sample image under the scene information; and determine a first deep feature of the sample image, wherein the first deep feature includes the spatial feature of the sample image; determine a first relational feature corresponding to the color feature and spatial feature of the sample image under the scene information based on the first shallow feature and the first deep feature; and determine the predicted position of the target object in the sample image under the scene information based on the first relational feature and the first deep feature.

[0172] In one possible implementation, the input module 602 is specifically used to obtain a first attention weight based on a first deep feature; determine the value corresponding to a first relation feature based on a first shallow feature, a first deep feature, a first attention weight, and an attention mechanism algorithm; and determine the predicted position under scene information based on the first relation feature, the value corresponding to the first relation feature, and the first deep feature.

[0173] In one possible implementation, the training module 604 is further configured to determine whether the loss value is less than a preset loss value after determining the loss value and before adjusting the target detection model based on the loss value; if not, decrease the weight corresponding to the default scene by a first preset value, increase the weight corresponding to each real scene by a second preset value, and determine the sub-loss value corresponding to each sample image based on the adjusted weights, and update the loss value based on each determined sub-loss value.

[0174] Example 9:

[0175] Based on the same technical concept and the above embodiments, this application provides a target detection device. Figure 7 This is a schematic diagram of a target detection device provided in an embodiment of this application, as shown below. Figure 7 As shown, the device includes:

[0176] The second acquisition module 701 is used to acquire the image to be detected;

[0177] The detection module 702 is used to input the image to be detected and the real scene information to which the image to be detected belongs into the trained target detection model, and obtain the target detection position of the target object in the image to be detected under the real scene information output by the target detection model.

[0178] In one possible implementation, the detection module 702 is specifically configured to: determine a second shallow feature of the image to be detected based on the image to be detected and the corresponding real scene information, wherein the second shallow feature includes the color feature of the image to be detected under the real scene information; and determine a second deep feature of the image to be detected, wherein the second deep feature includes the spatial feature of the image to be detected; determine a second relational feature corresponding to the color feature and spatial feature of the image to be detected under the real scene information based on the second shallow feature and the second deep feature; determine the target detection location information of the target object in the image to be detected under the real scene information based on the second relational feature and the second deep feature; and determine the target detection location output by the trained target detection model based on the target detection location information.

[0179] In one possible implementation, the detection module 702 is specifically used to obtain a second attention weight based on the second deep feature; determine the value corresponding to the second relation feature based on the second shallow feature, the second deep feature, the second attention weight, and the attention mechanism algorithm; and determine the target detection position in the real scene information based on the second relation feature, the value corresponding to the second relation feature, and the second deep feature.

[0180] Example 10:

[0181] Based on the same technical concept, this application also provides an electronic device. Figure 8 This application provides a schematic diagram of an electronic device structure, such as... Figure 8 As shown, it includes: processor 801, communication interface 802, memory 803 and communication bus 804, wherein processor 801, communication interface 802 and memory 803 communicate with each other through communication bus 804.

[0182] In one embodiment, memory 803 stores a computer program that, when executed by processor 801, causes processor 801 to perform an embodiment of training any of the above-described target detection models.

[0183] In yet another embodiment, memory 803 stores a computer program that, when executed by processor 801, causes processor 801 to perform any of the above-described embodiments of target detection.

[0184] The communication bus mentioned in the aforementioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 802 is used for communication between the aforementioned electronic device and other devices. The memory can include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0185] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0186] Based on the same technical concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by an electronic device. When the program is run on the electronic device, it causes the electronic device to execute an embodiment that implements training of any of the above-mentioned target detection models, or an embodiment that implements target detection when executed.

[0187] The aforementioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in an electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), optical storage such as CDs, DVDs, BDs, HVDs, etc., and semiconductor storage such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs), etc.

[0188] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A training method for an object detection model, characterized in that, The method includes: Obtain each sample image contained in the sample set and the target location of the target object contained in each sample image, wherein the sample images include reference sample images and real sample images, and record the scene information to which the sample images belong, wherein the scene information includes default scene and real scene; Each sample image and its corresponding scene information are input into the target detection model to obtain the predicted location of the target object in each sample image output by the target detection model. For each sample image, a sub-loss value is determined based on the predicted location and target location corresponding to the sample image, and the sub-loss value is adjusted according to the weight corresponding to the scene information to which the sample image belongs, wherein the weight corresponding to the real scene is greater than the weight corresponding to the default scene. Based on each adjusted sub-loss value, a loss value is determined, and the target detection model is adjusted based on the loss value.

2. The method according to claim 1, characterized in that, The step of inputting each sample image and its corresponding scene information into the target detection model to obtain the predicted location of the target object in each sample image output by the target detection model includes: For any sample image, the target detection model determines a first shallow feature of the sample image based on the sample image and the corresponding scene information, wherein the first shallow feature includes the color feature of the sample image under the scene information; and determines a first deep feature of the sample image, wherein the first deep feature includes the spatial feature of the sample image; based on the first shallow feature and the first deep feature, determines a first relational feature corresponding to the color feature and the spatial feature of the sample image under the scene information; and based on the first relational feature and the first deep feature, determines the predicted position of the target object in the sample image under the scene information.

3. The method according to claim 2, characterized in that, Determining the predicted location of the target object in the sample image under the scene information based on the first relational feature and the first deep feature includes: Based on the first deep feature, obtain the first attention weight; based on the first shallow feature, the first deep feature, the first attention weight, and the attention mechanism algorithm, determine the value corresponding to the first relationship feature; The predicted position under the scene information is determined based on the first relation feature, the value corresponding to the first relation feature, and the first deep feature.

4. The method according to claim 1, characterized in that, After determining the loss value and before adjusting the target detection model based on the loss value, the method further includes: Determine whether the loss value is less than a preset loss value; If not, decrease the weight corresponding to the default scene by a first preset value, increase the weight corresponding to each real scene by a second preset value, and determine the sub-loss value corresponding to each sample image based on the adjusted weights, and update the loss value based on each determined sub-loss value.

5. A target detection method, characterized in that, The method includes: Acquire the image to be detected; The image to be detected and the real scene information to which the image to be detected belong are input into the trained target detection model to obtain the target detection position of the target object in the image to be detected under the real scene information output by the target detection model. The trained target detection model is obtained by inputting each sample image and its corresponding scene information from the sample set into the target detection model, obtaining the predicted position of the target object in each sample image output by the target detection model; determining a sub-loss value based on the predicted position of each sample image and the target position of the target object contained in the sample image, and adjusting the sub-loss value according to the weight corresponding to the scene information to which the corresponding sample image belongs; and adjusting the target detection model based on each adjusted sub-loss value; wherein the sample image includes a reference sample image and a real sample image, and records the scene information to which the sample image belongs, the scene information including a default scene and a real scene, and the weight corresponding to the real scene is greater than the weight corresponding to the default scene.

6. The method according to claim 5, characterized in that, The target detection model outputs the target detection location of the target object in the image to be detected under the real scene information, including: The target detection model determines a second shallow feature of the image to be detected based on the image to be detected and the corresponding real scene information, wherein the second shallow feature includes the color feature of the image to be detected under the real scene information; and determines a second deep feature of the image to be detected, wherein the second deep feature includes the spatial feature of the image to be detected. Based on the second shallow layer features and the second deep layer, a second relationship feature corresponding to the color features and spatial features of the image to be detected under the real scene information is determined; Based on the second relational feature and the second deep feature, the target detection location information of the target object in the image to be detected under the real scene information is determined; Based on the target detection location information, the target detection location output by the trained target detection model is determined.

7. The method according to claim 6, characterized in that, Determining the target detection location of the target object in the image to be detected under the real scene information based on the second relational feature and the second deep feature includes: Based on the second deep feature, obtain the second attention weight; based on the second shallow feature, the second deep feature, the second attention weight, and the attention mechanism algorithm, determine the value corresponding to the second relation feature; The target detection location in the real scene information is determined based on the second relation feature, the value corresponding to the second relation feature, and the second deep feature.

8. A training device for an object detection model, characterized in that, The device includes: The first acquisition module is used to acquire each sample image contained in the sample set and the target location of the target object contained in each sample image. The sample images include reference sample images and real sample images, and the scene information to which the sample images belong is recorded. The scene information includes a default scene and a real scene. The input module is used to input each sample image and the corresponding scene information into the target detection model to obtain the predicted position of the target object in each sample image output by the target detection model; The determination module is used to determine a sub-loss value for each sample image based on the predicted position and target position corresponding to the sample image, and to adjust the sub-loss value according to the weight corresponding to the scene information to which the sample image belongs, wherein the weight corresponding to the real scene is greater than the weight corresponding to the default scene. The training module is used to determine a loss value based on each adjusted sub-loss value, and to adjust the target detection model based on the loss value.

9. A target detection device, characterized in that, The device includes: The second acquisition module is used to acquire the image to be detected; A detection module is used to input the image to be detected and the real scene information to which the image to be detected belongs into a trained target detection model, and obtain the target detection position of the target object in the image to be detected under the real scene information output by the target detection model; wherein, the trained target detection model is obtained by inputting each sample image contained in the sample set and the corresponding scene information into the target detection model, and obtaining the predicted position of the target object in each sample image output by the target detection model; determining a sub-loss value based on the predicted position corresponding to each sample image and the target position of the target object contained in the sample image, and adjusting the sub-loss value according to the weight corresponding to the scene information to which the corresponding sample image belongs; and adjusting the target detection model based on each adjusted sub-loss value; wherein, the sample image includes a reference sample image and a real sample image, and records the scene information to which the sample image belongs, the scene information including a default scene and a real scene, and the weight corresponding to the real scene is greater than the weight corresponding to the default scene.

10. An electronic device, characterized in that, The electronic device includes at least a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the step of training a target detection model as described in any one of claims 1-4, or to implement the step of a target detection method as described in any one of claims 5-7.

Images