Model training method and device, target detection method and device, equipment and storage medium

Abstract

The invention discloses a model training method and device, a target detection method and device, equipment and a storage medium. The model training method comprises the following steps: acquiring an initial training image, and generating an object frame of a target object in the initial training image; segmenting the object frame to obtain at least two segmented image blocks, and generating a label of the object frame according to the area of the target object in the segmented image blocks to obtain a training sample image; and training an initial detection model by using the training sample image to obtain a target detection model. According to the technical scheme provided by the embodiment of the invention, the label of the training sample image can be quickly determined by utilizing the semantic segmentation area without inputting a large amount of labor and economic cost, so that the accuracy of the sample label is ensured, and the training efficiency of the model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and in particular, to a model training method, an object detection method, a device, a device and a storage medium. Background Art

[0002] Autonomous driving technology is one of the high-tech fields that have developed rapidly in recent years. It combines the research results of multiple disciplines such as artificial intelligence, machine learning, sensor technology, big data processing, and advanced control strategies, aiming to achieve the autonomous driving of vehicles, reduce traffic accidents, and improve road safety and traffic efficiency.

[0003] Currently, in the field of autonomous driving, object detection algorithms are one of the core technologies, which are used to identify and track objects around the vehicle, such as other vehicles, pedestrians, traffic signs, and obstacles. These algorithms are crucial for ensuring the safe and effective driving of autonomous vehicles in complex and changing traffic environments.

[0004] However, the training of object detection models usually requires a large amount of data, and often requires a large amount of manual data annotation. The economic cost of manual annotation is relatively large, and the accuracy cannot be guaranteed, which affects the training efficiency of the model. Summary of the Invention

[0005] The present invention provides a model training method, an object detection method, a device, a device and a storage medium to solve the problems of large training efficiency and economic cost of the model.

[0006] In a first aspect, the present invention provides a model training method, including:

[0007] Obtain an initial training image, and generate an object box of a target object in the initial training image;

[0008] Segment the object box to obtain at least two segmented tiles, and generate a label of the object box according to the area of the target object in the segmented tiles to obtain a training sample image, where the label of the object box at least includes the true ground contact point category of the target object, and the true ground contact point category is used to characterize the category of the grounded part of the target object;

[0009] Train an initial detection model using the training sample image to obtain an object detection model.

[0010] In a second aspect, the present invention provides an object detection method, including:

[0011] Obtain an image to be detected;

[0012] Input the image to be detected into the target detection model to obtain the model detection result. Among them, the image to be detected includes a first target object containing a second target object. The model detection result includes the object box, moving direction, and category of the target object, as well as the grounding point position and grounding point category of the second target object. The target detection model is obtained by the model training method described in any one of claims 1-5.

[0013] In a third aspect, the present invention provides a model training device, including:

[0014] An object box generation module, configured to obtain an initial training image and generate an object box of a target object in the initial training image;

[0015] A label generation module, configured to segment the object box to obtain at least two segmented tiles, and generate a label of the object box according to the area of the target object in the segmented tiles to obtain a training sample image. Among them, the label of the object box includes at least the true grounding point category of the target object, and the true grounding point category is used to characterize the category of the grounded part of the target object;

[0016] A model training module, configured to train an initial detection model using the training sample image to obtain a target detection model.

[0017] In a fourth aspect, the present invention provides a target detection device, including:

[0018] An image acquisition module, configured to acquire an image to be detected;

[0019] A target detection module, configured to input the image to be detected into the target detection model to obtain the model detection result. Among them, the image to be detected includes a first target object containing a second target object. The model detection result includes the object box, moving direction, and category of the target object, as well as the grounding point position and grounding point category of the second target object. The target detection model is obtained by the model training method described in the first aspect above.

[0020] In a fifth aspect, the present invention provides an electronic device, which includes:

[0021] At least one processor;

[0022] And a memory communicatively connected to at least one processor;

[0023] Wherein, the memory stores a computer program executable by at least one processor. The computer program is executed by at least one processor so that at least one processor can execute the model training method described in the first aspect above.

[0024] Sixthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a processor to implement the model training method of the first aspect when executed.

[0025] In the model training solution provided by the present invention, an object frame of a target object is first generated in an initial training image, then the object frame is segmented, the label of the training sample image is accurately determined according to the area of the target object in the segmented tile obtained after segmentation, and finally the model can be trained using the training sample image to obtain a target detection model. This training solution does not require a large amount of labor and economic costs, quickly determines the label of the training sample image using the semantic segmentation area, and at the same time ensures the accuracy of the sample label, improving the training efficiency of the model. The target detection solution provided by the present invention can accurately identify the object frame, moving direction, category, grounding point position, and grounding point category of the target object.

[0026] It should be understood that the content described in this part is not intended to identify the key or important features of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0028] Figure 1 is a flowchart of a model training method according to Embodiment 1 of the present invention;

[0029] Figure 2 is a schematic diagram of a segmentation method according to Embodiment 1 of the present invention;

[0030] Figure 3 is a schematic diagram of a wheel object frame according to Embodiment 1 of the present invention;

[0031] Figure 4 is a schematic diagram of the network structure of a vision large model according to Embodiment 1 of the present invention;

[0032] Figure 5 is a flowchart of a model training method according to Embodiment 2 of the present invention;

[0033] Figure 6 is a flowchart of a target detection method according to Embodiment 3 of the present invention;

[0034] Figure 7 It is a schematic structural diagram of a model training device provided in Embodiment 4 of the present invention;

[0035] Figure 8 It is a schematic structural diagram of an object detection device provided in Embodiment 5 of the present invention;

[0036] Figure 9 It is a schematic structural diagram of an electronic device provided in Embodiment 6 of the present invention. Specific embodiments

[0037] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In the description of the present invention, unless otherwise specified, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after. In addition, the terms "including" and "having" and any of their deformations are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0039] Embodiment 1

[0040] Figure 1 This is a flowchart of a model training method provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of training an object detection model. This method can be executed by a model training device, which can be implemented in the form of hardware and / or software. The model training device can be configured in an electronic device, and the electronic device can be composed of two or more physical entities or one physical entity.

[0041] As shown Figure 1 in FIG. Figure 1 , a model training method provided in Embodiment 1 of the present invention specifically includes the following steps:

[0042] S101. Obtain an initial training image, and generate an object box of a target object in the initial training image.

[0043] In this embodiment, an initial training image can be obtained first, and then the target object in the initial training image, such as a vehicle, etc., can be recognized to generate an object box of the target object.

[0044] S102. Segment the object box to obtain at least two segmented tiles, and generate a label of the object box according to the area of the target object in the segmented tiles, so as to obtain a training sample image, wherein the label of the object box at least includes the true ground contact point category of the target object, and the true ground contact point category is used to represent the category of the grounded part of the target object.

[0045] In this embodiment, each obtained object box can be segmented to obtain a plurality of segmented tiles. Usually, the shape of the target object is an irregular shape. Therefore, according to the semantic segmentation area (i.e., the area of the target object) in each segmented tile, a label of the object box including the true ground contact point category can be generated. The label paper can also include, for example, the moving direction, part or type of the target object, etc. This label is the label of the training sample image, and the areas of the target object in each segmented tile are usually not exactly the same. For example, when the target object is an animal, the head direction of the target object and the front leg in contact with the ground can be determined according to the size of the semantic segmentation area in each segmented tile, so as to determine the moving direction of the target object and the label of the ground contact point category (i.e., the front leg).

[0046] S103. Use the training sample image to train an initial detection model to obtain a target detection model.

[0047] In this embodiment, the initial detection model can be trained in multiple rounds by using the training sample image to obtain a target detection model. Among them, the target detection model can be a convolutional neural network model.

[0048] For the model training method provided in the embodiment of the present invention, an object box of a target object is first generated in the initial training image, then the object box is segmented, and the label of the training sample image is accurately determined according to the area of the target object in the segmented tiles obtained after segmentation. Finally, the model can be trained by using the training sample image to obtain a target detection model. This training method does not require a large amount of labor and economic costs, quickly determines the label of the training sample image by using the semantic segmentation area, and at the same time ensures the accuracy of the sample label, improving the training efficiency of the model.

[0049] Optionally, segmenting the object bounding box to obtain at least two segmented tiles, and generating a label for the object bounding box based on the area of the target object in the segmented tiles to obtain a training sample image, includes: determining the midpoints of the sides parallel to the ground in the rectangular object bounding box of the first target object, and using the midpoints to segment the rectangular object bounding box into two segmented tiles; determining the direction corresponding to the target object bounding box in the initial training image as the moving direction of the second target object, and generating a label for the second object bounding box of the second target object based on the moving direction to obtain a training sample image, where the target object bounding box is the segmented tile with a smaller area of the first target object contained in the two segmented tiles, and the second target object is contained in the first target object.

[0050] Optionally, the first target object includes a vehicle, and the second target object includes a wheel.

[0051] Exemplarily, generally the direction of the wheel depends on the orientation of the vehicle, and as long as the orientation of the vehicle head is determined, the direction of the wheel can be confirmed. Figure 2 It is a schematic diagram of a segmentation method. Figure 2 The red rectangular box in it is the rectangular object bounding box, and the dashed line in the red rectangular box is the segmentation line. On the left and right sides of the segmentation line are respectively a segmented tile. As Figure 2 shown, when the first target object is a vehicle and the second target object is a wheel, a segmentation line can be determined using the midpoints of the two sides parallel to the ground in the rectangular object bounding box, and the rectangular object bounding box is segmented into two segmented tiles according to the segmentation line. The direction corresponding to the target object bounding box in the initial training image (i.e., Figure 2 the direction of the blue area in it) is determined as the moving direction of the vehicle and the wheel. According to this moving direction, the ground contact point category of the wheel can be determined, such as the left front wheel, right front wheel, left rear wheel or right rear wheel, and thus the label of the object bounding box of the wheel can be generated. The obtained training sample image can include the category labels and object bounding box coordinates of the vehicle and the wheel, as well as the ground contact point category label and ground contact point position label of the wheel. Among them, Figure 3 It is a schematic diagram of a wheel object bounding box. Figure 3 The red rectangular box in it is the object bounding box of the wheel. As Figure 3 shown, the coordinates of the ground contact point of the wheel can be obtained through the object bounding box of the wheel. For example, the coordinates of the midpoint of the bottom side of the object bounding box of the wheel can be used as the ground contact point position of the wheel.

[0052] Optionally, generating an object bounding box of a target object in the initial training image includes: inputting the initial training image and a preset prompt word into a vision-language model to generate an object bounding box of the target object in the initial training image.

[0053] Exemplarily, Figure 4Schematic diagram of the network structure of a vision large model. As Figure 4 shown, after inputting the initial training image and the prompt into the vision large model, object bounding boxes of the vehicle and their position coordinates, object bounding boxes of the wheels and their position coordinates, class labels of the vehicle, and class labels of the wheels can be generated in the initial training image.

[0054] Embodiment 2

[0055] Figure 5 Flowchart of a model training method provided in Embodiment 2 of the present invention. The technical solution of the embodiment of the present invention is further optimized on the basis of the above-mentioned optional technical solutions, and a specific method for training the object detection model is given.

[0056] Optionally, generating the object bounding box of the target object in the initial training image includes: inputting the initial training image and a preset prompt into the vision large language model to generate the object bounding box of the target object in the initial training image. The advantage of this setting is that a vision large model is introduced for supervised training during the model training process. By utilizing the characteristics of high accuracy and good generalization of the large model, it assists in the model training work, quickly realizes the model training work, and shortens the training cycle.

[0057] Optionally, training the initial detection model with the training sample image to obtain the target detection model includes: inputting the training sample image into the initial detection model to obtain the detection result of the target object, where the target object includes a first target object containing a second target object, and the detection result includes the moving direction of the target object, the object bounding box of the target object, the class of the target object, the position of the grounding point of the second target object, and the class of the grounding point of the second target object. The label of the bounding box includes the true moving direction and true class of the target object and the true grounding point position and true grounding point class of the second target object; determining a first loss function value according to the difference between the object bounding box of the target object and the object bounding box in the training sample image, and determining a second loss function value according to the difference between the class of the target object and the true class; determining a third loss function value according to the differences between the class of the second target object, the class of the grounding point of the second target object, and the position of the grounding point of the second target object and their corresponding labels, where the corresponding labels include the true class and true grounding point class of the second target object and the true grounding point position; obtaining the loss function value by performing weighted summation on the first loss function value, the second loss function value, and the third loss function value, and determining the target detection model according to the loss function value. The advantage of this setting is that by determining the third loss function value, the moving direction and grounding position of the second target object in the first target object are used as the supervision items for model training, further ensuring the accuracy of the model.

[0058] As Figure 5 shown, a model training method provided in Embodiment 2 of the present invention specifically includes the following steps:

[0059] S201. Obtain an initial training image, and input the initial training image and a preset prompt word into a vision-language model to generate an object box of a target object in the initial training image, where the target object includes a first target object containing a second target object, the first target object includes a vehicle, and the second target object includes a wheel.

[0060] Specifically, the second target object touches the ground. The target object is the first target object containing the second target object.

[0061] S202. Determine the midpoint of the side parallel to the ground in the rectangular object box of the vehicle, and use the midpoint to divide the rectangular object box into two divided tiles.

[0062] S203. Determine the direction corresponding to the target object box in the initial training image as the moving direction of the wheel, and generate a label of the second object box of the wheel according to the moving direction to obtain a training sample image, where the target object box is the divided tile with a smaller area of the vehicle included in the two divided tiles.

[0063] S204. Input the training sample image into an initial detection model to obtain a detection result of the target object, where the detection result includes the moving directions of the vehicle and the wheel, the object boxes of the vehicle and the wheel, the categories of the vehicle and the wheel, the position of the wheel contact point on the ground and the category of the wheel contact point on the ground, and the label of the object box includes the true moving direction and true category of the vehicle and the wheel and the true position and true category of the wheel contact point on the ground.

[0064] Specifically, with the development of the Transformer model, 3D object detection algorithms based on Transformer emerge in an endless stream and stand out among autonomous driving perception algorithms. However, in the case of limited vehicle-side computing power, although the Transformer-based model may be more powerful in performance, due to its high computational complexity and dependence on a large amount of computing resources, it often cannot be actually deployed on vehicle-side devices. Therefore, vehicle-side intelligent systems usually choose to adopt traditional convolutional small model solutions, such as the initial detection model in this solution. These models are relatively lightweight, can run on the processors or embedded systems carried by vehicles, and can meet the requirements of real-time performance and practicality.

[0065] Traditional convolution-based object detection mainly focuses on detecting the 2D rectangular bounding box of the object. Then, the midpoint of the bottom edge of the object bounding box is taken as an approximation of the object's ground contact point, and through the coordinate transformation relationship, it is converted into coordinates in the 3D space to obtain the position of the object in the 3D space. The 3D space position is beneficial for the subsequent decision-making and planning of the vehicle. However, when the object is relatively large, such as a vehicle object, the midpoint of its bottom edge does not represent the true position of the vehicle. Therefore, the ground contact point of the vehicle's wheels can be used to reflect the true position information of the vehicle.

[0066] Specifically, the initial detection model may include a backbone network, a feature fusion module, a multi-scale feature module, and multiple detection heads. The initial detection model can realize the prediction of the object bounding box of the target object, the ground contact point coordinates of the target object, the prediction of the moving direction of the target object, and the prediction of the category of the target object. Among them, the backbone network is used to extract image features, the feature fusion module is used to aggregate features at different stages in the backbone network to enhance feature expression at different resolutions, the multi-scale feature module is used to further extract features of target objects of different sizes, and different detection heads are used to implement different detection tasks. For example, the rectangular object bounding box detection head is used to detect the rectangular object bounding box of the target, the ground contact point detection head is used to detect the ground contact point of the target object, and the detection head for category prediction is used to predict the category of the target object. During the model training process, according to the gap between the detection result and the label of the object bounding box, the loss function values in terms of the object bounding box, category, ground contact point position, and ground contact point category of the model can be determined.

[0067] S205. Determine the first loss function value according to the gap between the object bounding box of the target object and the object bounding box in the training sample image.

[0068] S206. Determine the second loss function value according to the gap between the category of the target object and the true category.

[0069] S207. Determine the third loss function value according to the gaps between the wheel category, the ground contact point category of the wheel, and the position of the second target object's ground contact point and their corresponding labels, where the corresponding labels include the true category and true ground contact point category of the wheel and the true ground contact point position.

[0070] S208. Obtain the loss function value by performing a weighted sum of the first loss function value, the second loss function value, and the third loss function value.

[0071] Exemplarily, the determination method of the loss function value L can be:

[0072] L = αL box + βL class + γ all L point

[0073] Among them, L box is the loss function of the object bounding box, that is, the first loss function, which can be the IOU loss function. L class is the class loss function of the first target object, that is, the second loss function, which can be the Focal Loss function. L point is the ground contact point loss function, that is, the third loss function.

[0074] L point = γ 1 L point_mse + γ 2 L point_bce

[0075] Among them, L pointmse is the mean square error loss value of the ground contact point position of the wheel, L pointbce is the binary classification loss value of the second target object class and the second target object ground contact point class, that is, the binary classification loss value of the wheel class and the wheel ground contact point class. α, β, γ all , γ 1 and γ 2 are all weight parameters between different losses.

[0076] S209. Determine the target detection model according to the loss function value.

[0077] In the model training method provided by the embodiments of the present invention, a vision large model is introduced for supervised training during the model training process. Utilizing the characteristics of high accuracy and good generalization of the large model, it assists in the model training work, quickly realizes the model training work, shortens the training cycle, and by determining the third loss function value, taking the ground contact point class and the ground contact position of the second target object in the first target object as the supervision items for model training, further ensures the accuracy of the model.

[0078] Embodiment Three

[0079] Figure 6 The flowchart of a target detection method provided by Embodiment Three of the present invention is shown. This embodiment is applicable to the situation of detecting target objects in an image. This method can be executed by a target detection device, which can be implemented in the form of hardware and / or software. The target detection device can be configured in an electronic device, and the electronic device can be composed of two or more physical entities or one physical entity.

[0080] As Figure 6 shown, the target detection method provided by Embodiment Three of the present invention specifically includes the following steps:

[0081] S301. Obtain the image to be detected.

[0082] S302. Input the image to be detected into the target detection model to obtain the model detection result. The image to be detected includes a first target object containing a second target object. The model detection result includes the object box, moving direction, and category of the target object, as well as the grounding point position and grounding point category of the second target object. The target detection model is obtained through the model training method described above.

[0083] Specifically, if the target objects are a vehicle and a wheel, after inputting the image to be detected into the target detection model, the object boxes and categories of the vehicle and the wheel in the image to be detected, as well as the grounding point category and grounding point position of the wheel, can be obtained.

[0084] The target detection method provided by the embodiments of the present invention acquires an image to be detected, inputs the image to be detected into the target detection model to obtain the model detection result. The image to be detected includes a first target object containing a second target object. The model detection result includes the object box, moving direction, and category of the target object, as well as the grounding point position and grounding point category of the second target object. The target detection model is obtained through the model training method described above. The technical solution of the embodiments of the present invention can accurately identify the object box, moving direction, category of the target object, as well as the grounding point position and grounding point category.

[0085] Embodiment 4

[0086] Figure 7 FIG. is a schematic structural diagram of a model training device provided in Embodiment 4 of the present invention. As Figure 7 shown, the device includes: an object box generation module 401, a label generation module 402, and a model training module 403, where:

[0087] The object box generation module is configured to acquire an initial training image and generate an object box of a target object in the initial training image;

[0088] The label generation module is configured to segment the object box to obtain at least two segmented tiles, and generate a label of the object box according to the area of the target object in the segmented tiles to obtain a training sample image. The label of the object box includes at least the true grounding point category of the target object, and the true grounding point category is used to characterize the category of the grounded part of the target object;

[0089] The model training module is configured to train an initial detection model using the training sample image to obtain a target detection model.

[0090] The model training device provided by the embodiments of the present invention does not require a large amount of labor and economic costs. It quickly determines the labels of training sample images using the semantic segmentation area, while ensuring the accuracy of the sample labels and improving the training efficiency of the model.

[0091] Optionally, the label generation module includes:

[0092] A segmentation unit, configured to determine the midpoints of the sides parallel to the ground in the rectangular object frame of the first target object, and use the midpoints to divide the rectangular object frame into two segmented tiles;

[0093] A training sample image generation unit, configured to determine the moving direction of the second target object as the direction corresponding to the target object frame in the initial training image, and generate the label of the second object frame of the second target object according to the moving direction to obtain a training sample image, where the target object frame is the segmented tile with a smaller area of the first target object included in the two segmented tiles, and the first target object includes the second target object.

[0094] Further, the first target object includes a vehicle, and the second target object includes a wheel.

[0095] Optionally, the model training module includes:

[0096] A detection result determination unit, configured to input the training sample image into an initial detection model to obtain the detection result of the target object, where the target object includes the first target object containing the second target object, and the detection result includes the moving direction of the target object, the object frame of the target object, the category of the target object, and the grounding point position and grounding point category of the second target object. The label of the object frame includes the true moving direction and true category of the target object and the true grounding point position and true grounding point category of the second target object;

[0097] A first loss value determination unit, configured to determine a first loss function value according to the gap between the object frame of the target object and the object frame in the training sample image, and determine a second loss function value according to the gap between the category of the target object and the true category, where the target object includes the first target object and the second target object, and the first target object includes the second target object;

[0098] A second loss value determination unit, configured to determine a third loss function value according to the gaps between the category of the second target object, the grounding point category of the second target object, and the grounding point position of the second target object and the corresponding labels, where the corresponding labels include the true category and true grounding point category of the second target object and the true grounding point position;

[0099] A detection model determination unit, configured to obtain a loss function value by performing weighted summation on the first loss function value, the second loss function value, and the third loss function value, and determine a target detection model according to the loss function value.

[0100] Optionally, the object bounding box generation module includes:

[0101] An object bounding box generation unit, configured to input the initial training image and a preset prompt word into a vision-language model to generate an object bounding box of a target object in the initial training image.

[0102] The model training device provided by an embodiment of the present invention can execute the model training method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0103] Embodiment Five

[0104] Figure 8 FIG. is a schematic structural diagram of a target detection device provided by Embodiment Five of the present invention. As Figure 8 shown, the device includes: an image acquisition module 501 and a target detection module 502, wherein:

[0105] The image acquisition module is configured to acquire an image to be detected;

[0106] The target detection module is configured to input the image to be detected into a target detection model to obtain a model detection result, wherein the image to be detected includes a first target object containing a second target object, and the model detection result includes an object bounding box, a moving direction, and a category of the target object, and a grounding point position and a grounding point category of the second target object, and the target detection model is obtained by the model training method described in the above embodiment.

[0107] The target detection device provided by an embodiment of the present invention can accurately identify an object bounding box, a moving direction, a category of a target object, as well as a grounding point position and a grounding point category.

[0108] The target detection device provided by an embodiment of the present invention can execute the model training method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0109] Embodiment Six

[0110] Figure 9FIG. 0 shows a schematic structural diagram of an electronic device 60 that can be used to implement an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0111] As Figure 9 shown, the electronic device 60 includes at least one processor 61 and a memory communicatively connected to the at least one processor 61, such as a read-only memory (ROM) 62, a random access memory (RAM) 63, etc. The memory stores a computer program executable by the at least one processor. The processor 61 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 62 or the computer program loaded from the storage unit 68 into the random access memory (RAM) 63. In the RAM 63, various programs and data required for the operation of the electronic device 60 can also be stored. The processor 61, the ROM 62, and the RAM 63 are connected to each other via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0112] Multiple components in the electronic device 60 are connected to the I / O interface 65, including: an input unit 66, such as a keyboard, a mouse, etc.; an output unit 67, such as various types of displays, speakers, etc.; a storage unit 68, such as a magnetic disk, an optical disc, etc.; and a communication unit 69, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 69 allows the electronic device 60 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0113] The processor 61 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 61 executes the various methods and processes described above, such as the model training method.

[0114] In some embodiments, the model training method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 68. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 60 via the ROM 62 and / or the communication unit 69. When the computer program is loaded into the RAM 63 and executed by the processor 61, one or more steps of the model training method described above may be performed. Alternatively, in other embodiments, the processor 61 may be configured to execute the model training method by any other suitable means (e.g., by means of firmware).

[0115] The various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on a chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0116] The computer program for implementing the method of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that the computer program, when executed by the processor, causes the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0117] The computer device provided above may be used to execute the model training method provided in any of the above embodiments, and has the corresponding functions and beneficial effects.

[0118] Embodiment Seven

[0119] In the context of the present invention, a computer-readable storage medium may be a tangible medium, and the computer-executable instructions are used to execute a model training method when executed by a computer processor. The method includes:

[0120] Obtaining an initial training image and generating an object box of a target object in the initial training image;

[0121] Segment the object bounding box to obtain at least two segmented tiles, and generate a label for the object bounding box according to the area of the target object in the segmented tiles to obtain a training sample image, wherein the label of the object bounding box includes at least the true ground contact point category of the target object, and the true ground contact point category is used to characterize the category of the grounded part of the target object;

[0122] Use the training sample image to train an initial detection model to obtain a target detection model.

[0123] It is also possible to execute a target detection method, which includes:

[0124] Obtain an image to be detected;

[0125] Input the image to be detected into the target detection model to obtain a model detection result, wherein the image to be detected includes a first target object containing a second target object, and the model detection result includes the object bounding box, moving direction and category of the target object, and the ground contact point position and ground contact point category of the second target object. The target detection model is obtained by the model training method described in the above embodiments.

[0126] In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium may be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] The computer device provided above can be used to execute the model training method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0128] It should be noted that in the embodiments of the above model training device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0129] Note that the above are only the preferred embodiments of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments only. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A model training method, characterized in that: include: Acquire an initial training image, and generate an object frame of a target object in the initial training image; Segmenting the object frame to obtain at least two segmented image blocks, and generating a label of the object frame according to the area of ​​the target object in the segmented image blocks to obtain a training sample image, wherein the label of the object frame includes at least a real grounding point category of the target object, and the real grounding point category is used to characterize the category of the grounded part of the target object; The initial detection model is trained using the training sample images to obtain a target detection model.

2. The method according to claim 1, characterized in that The step of segmenting the object frame to obtain at least two segmented image blocks, and generating a label of the object frame according to the area of ​​the target object in the segmented image blocks to obtain a training sample image includes: Determine the midpoint of the side parallel to the ground in the rectangular object frame of the first target object, and use the midpoint to segment the rectangular object frame into two segmented blocks; The direction corresponding to the target object frame in the initial training image is determined as the moving direction of the second target object, and a label of the second object frame of the second target object is generated according to the moving direction to obtain a training sample image, wherein the target object frame is a smaller segmentation block of the first target object contained in the two segmentation blocks, and the first target object contains the second target object.

3. The method according to claim 1, characterized in that The using the training sample images to train the initial detection model to obtain the target detection model includes: Inputting the training sample image into an initial detection model to obtain a detection result of the target object, wherein the target object includes a first target object including a second target object, and the detection result includes a moving direction of the target object, an object frame of the target object, a target object category, and a grounding point position of the second target object and a grounding point category of the second target object, and the label of the object frame includes a real moving direction and a real category of the target object and a real grounding point position and a real grounding point category of the second target object; Determine a first loss function value according to the difference between the object frame of the target object and the object frame in the training sample image, and determine a second loss function value according to the difference between the target object category and the true category; Determine a third loss function value according to the difference between the second target object category, the second target object grounding point category, and the second target object grounding point position and the corresponding labels, wherein the corresponding labels include the true category and true grounding point category of the second target object and the true grounding point position; A loss function value is obtained by performing weighted summation on the first loss function value, the second loss function value and the third loss function value, and a target detection model is determined according to the loss function value.

4. The method according to claim 1, characterized in that: The generating an object frame of the target object in the initial training image comprises: The initial training image and the preset prompt word are input into a visual large language model to generate an object frame of the target object in the initial training image.

5. The method according to claim 2 or 3, characterized in that: The first target object includes a vehicle, and the second target object includes a wheel.

6. A target detection method, characterized in that: include: Acquire the image to be detected; The image to be detected is input into a target detection model to obtain a model detection result, wherein the image to be detected includes a first target object including a second target object, and the model detection result includes an object frame, a moving direction and a category of the target object and a grounding point position and a grounding point category of the second target object, and the target detection model is obtained by the model training method described in any one of claims 1 to 5.

7. A model training device, characterized in that: include: An object frame generation module, used to obtain an initial training image and generate an object frame of a target object in the initial training image; a label generation module, configured to segment the object frame to obtain at least two segmented image blocks, and generate a label of the object frame according to the area of ​​the target object in the segmented image blocks to obtain a training sample image, wherein the label of the object frame includes at least a real grounding point category of the target object, and the real grounding point category is used to characterize the category of the grounding part of the target object; The model training module is used to train the initial detection model using the training sample images to obtain the target detection model.

8. A target detection device, characterized in that: include: An image acquisition module, used for acquiring an image to be detected; A target detection module, used for inputting the image to be detected into a target detection model to obtain a model detection result, wherein the image to be detected includes a first target object containing a second target object, and the model detection result includes an object frame, a moving direction and a category of the target object and a grounding point position and a grounding point category of the second target object, and the target detection model is obtained by the model training method described in any one of claims 1 to 5.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the model training method described in any one of claims 1 to 5, and / or can execute the target detection method described in claim 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a processor to implement the model training method described in any one of claims 1 to 5 when executed, and / or to be able to execute the target detection method described in claim 6.

Images