Model training method and device, object detection information generation method and device and electronic equipment

Abstract

The embodiment of the invention discloses a model training method, an object detection information generation method and device and electronic equipment. A specific embodiment of the method comprises the steps of obtaining an object storage atlas; for each object storage map in the object storage map set, the following generation steps are executed: edge marking is performed on a storage rack displayed by the object storage map to generate an edge label set, and an edge label represents the position of a storage position in the storage rack and represents whether a target object exists in the storage position; generating initial training data according to the object storage graph and the edge label set; performing data preprocessing on the initial training data to generate training data; and according to the obtained training data set, performing model training on the initial target object detection model to generate a target object detection model. The implementation mode is related to artificial intelligence, and the target object in the storage rack displayed by the object storage graph can be accurately detected by utilizing the generated target object detection model.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a model training method, an object detection information generating method, a device, and an electronic device. Background Art

[0002] At present, with the continuous development of artificial intelligence, target detection has become one of the current mainstream tasks. For the detection of target objects (e.g., poultry) stored in storage racks (e.g., poultry cages), the method usually adopted is: directly input the target object storage image into the YOLO model to generate the target object detection result.

[0003] However, the inventors have found that when the above method is used to generate detection information corresponding to the target object, the following technical problems often occur:

[0004] In the image corresponding to the captured target object, the storage racks are located on both sides of the image, resulting in the storage racks in the image gradually increasing from small to large, resulting in more interference information and invalid information corresponding to the rectangular frame, making the accuracy of target object detection based on the rectangular frame low.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a model training method, an object detection information generating method, an apparatus and an electronic device to solve the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a model training method, comprising: obtaining an object storage atlas, wherein the object storage map is an image of a storage rack storing a target object; for each object storage map in the object storage atlas, executing the following generation steps: edge labeling the storage rack displayed in the object storage map to generate an edge label set, wherein the edge label characterizes the position of a storage location in the storage rack and characterizes whether a target object exists in the storage location; generating initial training data based on the object storage map and the edge label set; performing data preprocessing on the initial training data to generate training data; and performing model training on an initial target object detection model based on the obtained training data set to generate a target object detection model.

[0009] Optionally, the storage racks displayed in the object storage diagram are edge marked to generate an edge label set, including: coordinate marking and color marking of the edges at the target positions corresponding to the storage racks displayed in the object storage diagram to generate an edge label set, wherein the edges corresponding to the target positions are connecting edges between the target storage locations in the storage racks.

[0010] Optionally, the data preprocessing of the initial training data to generate training data includes: performing image scaling on the initial training data according to a preset image scaling ratio to generate training data.

[0011] In a second aspect, some embodiments of the present disclosure provide a model training device, comprising: a first acquisition unit, configured to acquire an object storage atlas, wherein the object storage map is an image of a storage rack storing a target object; a determination unit, configured to determine at least one crop growth experimental model associated with the above-mentioned target crop information; an execution unit, configured to perform the following generation steps for each object storage map in the above-mentioned object storage atlas: edge labeling the storage rack displayed in the above-mentioned object storage map to generate an edge label set, wherein the edge label represents the position of a storage location in the storage rack and represents whether there is a target object in the storage location; generating initial training data based on the above-mentioned object storage map and the above-mentioned edge label set; performing data preprocessing on the above-mentioned initial training data to generate training data; a training unit, configured to perform model training on an initial target object detection model based on the obtained training data set to generate a target object detection model.

[0012] Optionally, the execution unit can be configured to: perform coordinate marking and color marking on the edges at the target position corresponding to the storage rack displayed in the object storage diagram to generate an edge label set, wherein the edges corresponding to the target position are connecting edges between the target storage locations in the storage rack.

[0013] Optionally, the execution unit may be configured to: perform image scaling on the initial training data according to a preset image scaling ratio to generate training data.

[0014] In a third aspect, some embodiments of the present disclosure provide a method for generating object detection information, comprising: obtaining a target object storage map; performing image preprocessing on the target object storage map to generate a preprocessed image; inputting the preprocessed image into a pre-trained target object detection model to generate target object detection information, wherein the target object detection model is generated based on a model training method.

[0015] Optionally, the target object detection model includes: an image feature extraction layer, a multi-scale feature output layer and a detection head layer, wherein the multi-scale feature output layer is a network layer that outputs feature information of the target at a reduced dimension scale.

[0016] Optionally, the above-mentioned preprocessed image is input into a pre-trained target object detection model to generate target object detection information, including: inputting the above-mentioned preprocessed image into the above-mentioned image feature extraction layer to generate image feature information; inputting the above-mentioned image feature information into the above-mentioned multi-scale feature output layer to generate feature information under the above-mentioned target dimensionality reduction scale as target scale feature information; inputting the above-mentioned target scale feature information into the above-mentioned detection head layer to generate the above-mentioned target object detection information.

[0017] Optionally, the target object detection information includes at least one of the following: position information corresponding to the target storage point in the storage location, side length information corresponding to the target storage point, confidence, and existence information characterizing whether the target object exists in the storage location; and the target scale feature information is input into the detection head layer to generate the target object detection information, including: inputting the target scale feature information into the detection head layer to generate at least one of the position information, the side length information, the confidence, and the existence information.

[0018] In a fourth aspect, some embodiments of the present disclosure provide an object detection information generating device, comprising: a second acquisition unit, configured to acquire a target object storage map; a preprocessing unit, configured to perform image preprocessing on the above-mentioned target object storage map to generate a preprocessed image; a generation unit, configured to input the above-mentioned preprocessed image into a pre-trained target object detection model to generate target object detection information, wherein the above-mentioned target object detection model is generated based on a model training method.

[0019] Optionally, the target object detection model includes: an image feature extraction layer, a multi-scale feature output layer and a detection head layer, wherein the multi-scale feature output layer is a network layer that outputs feature information of the target at a reduced dimension scale.

[0020] Optionally, the generation unit can be configured to: input the above-mentioned preprocessed image into the above-mentioned image feature extraction layer to generate image feature information; input the above-mentioned image feature information into the above-mentioned multi-scale feature output layer to generate feature information at the above-mentioned target dimensionality reduction scale as target scale feature information; input the above-mentioned target scale feature information into the above-mentioned detection head layer to generate the above-mentioned target object detection information.

[0021] Optionally, the target object detection information includes at least one of the following: position information corresponding to the target storage point in the storage location, side length information corresponding to the target storage point, confidence, and existence information characterizing whether the target object exists in the storage location; and the generation unit can be configured to: input the target scale feature information into the detection head layer to generate at least one of the position information, the side length information, the confidence, and the existence information.

[0022] In a fifth aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the implementation methods in the first and third aspects.

[0023] In a sixth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any one of the implementations in the first and third aspects is implemented.

[0024] In a seventh aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which, when executed by a processor, implements the method described in any one of the implementation modes of the first and third aspects above.

[0025] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the model training method of some embodiments of the present disclosure, the generated target object detection model can be used to accurately detect the target object in the storage rack displayed by the object storage diagram. Specifically, the reason why the detection of the target object in the storage rack displayed by the relevant object storage diagram is not accurate is that in the image corresponding to the captured target object, the storage rack is located on both sides of the image screen, resulting in the storage rack in the image gradually showing a trend of gradually reaching from small to large, resulting in more interference information and invalid information corresponding to the rectangular frame, so that the accuracy of target object detection based on the rectangular frame is low. Based on this, the model training method of some embodiments of the present disclosure first obtains an object storage atlas. Among them, the object storage map is an image of the storage rack storing the target object. Here, the obtained object storage atlas is used to generate the corresponding training data set data source in the subsequent training, so as to facilitate the subsequent training of the initial target object detection model. Then, for each object storage map in the above object storage atlas, the following generation steps are performed: the first step is to label the storage rack displayed by the above object storage map to generate an edge label set. Among them, the edge label represents the position of the storage location in the storage rack and represents whether there is a target object in the storage location. Here, by labeling the storage racks, the information about the location of the corresponding storage location and whether the target object exists can be effectively reflected, avoiding a lot of interference information and invalid information, so that the subsequent model can learn the label information more effectively. The second step is to generate initial training data based on the above-mentioned object storage diagram and the above-mentioned edge label set. The third step is to perform data preprocessing on the above-mentioned initial training data to generate training data, so as to dynamically adjust the data according to the model structure of the initial target object detection model in a targeted manner, so that the target object detection model can learn the data features more accurately. Finally, based on the obtained training data set, the initial target object detection model is trained to generate a target object detection model with more accurate target object detection. In summary, by labeling the storage racks, the information about the location of the corresponding storage location and whether the target object exists can be effectively reflected, avoiding a lot of interference information and invalid information, so that the subsequent model can learn the label information more effectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0027] Figure 1 is a schematic diagram of an application scenario of a model training method according to some embodiments of the present disclosure;

[0028] Figure 2is a flowchart of some embodiments of the model training method according to the present disclosure;

[0029] Figure 3 is a schematic diagram of a storage rack, a storage location, and a target object in some embodiments of the model training method according to the present disclosure;

[0030] Figure 4 is a flowchart of other embodiments of the model training method according to the present disclosure;

[0031] Figure 5 is a schematic diagram of training data in some embodiments of the model training method according to the present disclosure;

[0032] Figure 6 is a flow chart of some embodiments of the object detection information generation method according to the present disclosure;

[0033] Figure 7 is a schematic diagram of the structure of some embodiments of the model training device according to the present disclosure;

[0034] Figure 8 is a schematic diagram of the structure of some embodiments of the object detection information generating device according to the present disclosure;

[0035] Fig. 9 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0036] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0037] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0038] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0039] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0040] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0041] The collection, storage, and use of user personal information (such as user portraits and user historical behaviors) involved in this disclosure, and other operations, before performing the corresponding operations, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, fulfilling the obligation to inform the personal information subject, and obtaining the authorization and consent of the personal information subject in advance.

[0042] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0043] Figure 1 It is a schematic diagram of an application scenario of the model training method according to some embodiments of the present disclosure.

[0044] exist Figure 1 In the application scenario, first, the electronic device 101 can obtain an object storage atlas 102. The object storage map is an image of a storage rack storing a target object. In this application scenario, the object storage atlas 102 can be a chicken house image set. Then, for each object storage map in the above object storage atlas, the electronic device 101 can perform the following generation steps: the first step is to perform edge labeling on the storage rack displayed in the above object storage map 1021 to generate an edge label set 103. The edge label represents the position of the storage location in the storage rack and represents whether there is a target object in the storage location. In this application scenario, the object storage map 1021 is an object storage map in the object storage atlas 102. The second step is to generate initial training data 104 based on the above object storage map 1021 and the above edge label set 103. The third step is to perform data preprocessing on the above initial training data 104 to generate training data 1051. Finally, based on the obtained training data set 105, the initial target object detection model 106 is trained to generate a target object detection model 107.

[0045] It should be noted that the electronic device 101 can be hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the electronic device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0046] It should be understood that Figure 1The number of electronic devices in the embodiment is only for illustration. Any number of electronic devices may be provided according to implementation requirements.

[0047] Continue to refer Figure 2 , shows a process 200 of some embodiments of the model training method according to the present disclosure. The model training method comprises the following steps:

[0048] Step 201, obtain the object storage atlas.

[0049] In some embodiments, the execution entity of the above model training method (for example Figure 1 The electronic device 101 shown) can obtain an object storage atlas through a wired connection or a wireless connection. Among them, the object storage map is an image of a storage rack storing a target object. The storage rack can be a shelf for storing the target object. The storage rack can be a shelf in various forms. For example, the storage rack can be a shelf in the form of a cube and include various storage locations. The storage location can be a storage space in the form of a cube. For example, the storage rack can be a shelf including three layers of storage locations. The target object can be an object stored in a storage location. In practice, the target object is poultry. For example, the target object can be a chicken. In practice, the storage rack can be a shelf for storing poultry. For example, the storage rack can be a chicken coop.

[0050] In practice, see Figure 3 , showing a schematic diagram of a storage rack, storage locations and a target object.

[0051] Step 202: for each object storage graph in the object storage graph set, perform the following generation steps:

[0052] Step 2021, edge labeling is performed on the storage racks shown in the above object storage diagram to generate an edge label set.

[0053] In some embodiments, the execution entity may label the edges of the storage racks displayed in the object storage diagram to generate an edge label set. The edge labeling may be labeling certain edges in the storage rack. The edge label represents the position of the storage location in the storage rack and represents whether there is a target object in the storage location. The position of the storage location may be the pixel coordinates corresponding to the storage location displayed in the object storage diagram. The object identifier corresponding to the storage location may represent whether there is a target object in the storage location. For example, the edge label may be "(32,45), 01". (32,45) may represent the position of the storage location in the storage rack. "01" indicates that there is a target object in the storage location.

[0054] Step 2022: Generate initial training data based on the object storage graph and the edge label set.

[0055] In some embodiments, the execution entity may generate initial training data according to the object storage graph and the edge label set.

[0056] As an example, the execution entity may combine the object storage graph and the edge label set to generate initial training data.

[0057] Step 2023: preprocess the initial training data to generate training data.

[0058] In some embodiments, the execution entity may perform data preprocessing on the initial training data to generate training data.

[0059] As an example, the execution entity may perform data format conversion on the initial training data to convert it into a target format to generate training data.

[0060] In some optional implementations of some embodiments, the execution subject may perform image scaling on the initial training data according to a preset image scaling ratio to generate training data. For example, the image scaling ratio may be "height scaling 1 / 3, width unchanged".

[0061] Step 203: Perform model training on the initial target object detection model according to the obtained training data set to generate a target object detection model.

[0062] In some embodiments, the execution entity may perform model training on the initial target object detection model according to the obtained training data set to generate a target object detection model.

[0063] As an example, the above-mentioned execution entity can perform model training on the initial target object detection model based on the training data set and utilize the back-propagation model updating method to generate a target object detection model.

[0064] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the model training method of some embodiments of the present disclosure, the generated target object detection model can be used to accurately detect the target object in the storage rack displayed by the object storage diagram. Specifically, the reason why the detection of the target object in the storage rack displayed by the relevant object storage diagram is not accurate is that in the image corresponding to the captured target object, the storage rack is located on both sides of the image screen, resulting in the storage rack in the image gradually showing a trend of gradually reaching from small to large, resulting in more interference information and invalid information corresponding to the rectangular frame, so that the accuracy of target object detection based on the rectangular frame is low. Based on this, the model training method of some embodiments of the present disclosure first obtains an object storage atlas. Among them, the object storage map is an image of the storage rack storing the target object. Here, the obtained object storage atlas is used to generate the corresponding training data set data source in the subsequent training, so as to facilitate the subsequent training of the initial target object detection model. Then, for each object storage map in the above object storage atlas, the following generation steps are performed: the first step is to label the storage rack displayed by the above object storage map to generate an edge label set. Among them, the edge label represents the position of the storage location in the storage rack and represents whether there is a target object in the storage location. Here, by labeling the storage racks, the information about the location of the corresponding storage location and whether the target object exists can be effectively reflected, avoiding a lot of interference information and invalid information, so that the subsequent model can learn the label information more effectively. The second step is to generate initial training data based on the above-mentioned object storage diagram and the above-mentioned edge label set. The third step is to perform data preprocessing on the above-mentioned initial training data to generate training data, so as to dynamically adjust the data according to the model structure of the initial target object detection model in a targeted manner, so that the target object detection model can learn the data features more accurately. Finally, based on the obtained training data set, the initial target object detection model is trained to generate a target object detection model with more accurate target object detection. In summary, by labeling the storage racks, the information about the location of the corresponding storage location and whether the target object exists can be effectively reflected, avoiding a lot of interference information and invalid information, so that the subsequent model can learn the label information more effectively.

[0065] Further references Figure 4 , shows a process 400 of other embodiments of the model training method according to the present disclosure. The model training method comprises the following steps:

[0066] Step 401, obtain the object storage atlas.

[0067] Step 402: for each object storage graph in the object storage graph set, perform the following generation steps:

[0068] Step 4021, coordinate marking and color marking are performed on the edges at the target position corresponding to the storage rack displayed in the object storage diagram to generate an edge label set.

[0069] In some embodiments, an execution entity (e.g. Figure 1 The electronic device 101 shown in the figure can coordinate and color the edges corresponding to the storage rack and at the target position displayed in the object storage diagram to generate an edge label set. Among them, the edges corresponding to the target position are the connecting edges between the target storage locations in the storage rack. The target position can be the position corresponding to the vertex of the preset orientation in at least one same vertex corresponding to two target storage locations in the storage rack. Coordinate marking can be to mark the edges corresponding to the target position in the form of coordinates to characterize the position information of the edges corresponding to the target position. Color marking can be to characterize the edges of the target position in color to characterize whether there is a target object in the target storage location corresponding to the edge of the target position. For example, for the edges marked with green color, it is characterized that the corresponding target storage location has a target object. For the edges marked with red color, it is characterized that the corresponding target storage location does not have a target object. The target storage location corresponding to the edge of the target position can be the storage location corresponding to the target orientation of the edge of the target position. For example, the target orientation can be the right direction orientation.

[0070] In practice, coordinate annotation can include the following steps:

[0071] The first step is to determine the location coordinates corresponding to the target location.

[0072] The second step is to determine the first side length and the second side length in the first direction and the second direction corresponding to the target position. The first direction and the second direction can be set according to the position of the target position in the storage location. For example, if the target position is the lower right corner of the storage location, the first direction is the direction directly above the target position. The second direction is the direction to the left of the target position.

[0073] In the third step, the abscissa value and the ordinate value in the position coordinates are added according to the first side length and the second side length, respectively, to generate information after coordinate annotation.

[0074] In practice, see Figure 5 , which shows a schematic diagram of training data, wherein the training data is annotated with colors and coordinates.

[0075] Step 4022: Generate initial training data based on the object storage graph and the edge label set.

[0076] Step 4023: preprocess the initial training data to generate training data.

[0077] Step 403: Perform model training on the initial target object detection model according to the obtained training data set to generate a target object detection model.

[0078] In some embodiments, the specific implementation of steps 401, 4022, 4023-403 and the technical effects thereof can be referred to in Figure 2 Steps 201, 2022, and 2023-203 in the corresponding embodiments are not repeated here.

[0079] from Figure 4 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 4 The process 400 of the model training method in some corresponding embodiments can more effectively reflect the location information of the corresponding storage location and the information on whether the target object exists by marking the coordinates and representing the colors of the edges of the target location, thereby ensuring the accuracy of the subsequent training data set and making the training of the target object detection model more accurate and effective.

[0080] Continue to refer Figure 6 , shows a process 600 of some embodiments of the object detection information generation method according to the present disclosure. The object detection information generation method comprises the following steps:

[0081] Step 601, obtain the target object storage map.

[0082] In some embodiments, the execution subject (eg, electronic device) of the object detection information generation method may obtain the target object storage map by wired or wireless means, wherein the target object storage map may be an image of the target object to be detected.

[0083] Step 602: perform image preprocessing on the target object storage map to generate a preprocessed image.

[0084] In some embodiments, the execution entity may perform image preprocessing on the target object storage map to generate a preprocessed image.

[0085] As an example, the execution subject may reduce the target object storage map to a target scale to generate a preprocessed image.

[0086] Step 603: input the preprocessed image into a pre-trained target object detection model to generate target object detection information.

[0087] In some embodiments, the execution subject may input the pre-processed image into a pre-trained target object detection model to generate target object detection information, wherein the target object detection model is generated based on a model training method.

[0088] In some optional implementations of some embodiments, the target object detection model includes: an image feature extraction layer, a multi-scale feature output layer and a detection head layer. Among them, the multi-scale feature output layer is a network layer that outputs feature information at the target dimensionality reduction scale. Among them, the image feature extraction layer can be a network layer for image feature extraction. The multi-scale feature output layer can be a network layer that outputs multi-scale feature information. In practice, the image feature extraction layer can be a Backbone network layer. The multi-scale feature output layer can be an FPN feature pyramid model. The detection head layer can be a Head detection head regression model. The target dimensionality reduction scale can be the scale with the highest dimensionality reduction processing ratio. That is, the multi-scale feature output layer can be the output layer of the multi-scale dimensionality reduction processing.

[0089] In some optional implementations of some embodiments, the step of inputting the preprocessed image into a pre-trained target object detection model to generate target object detection information may include the following steps:

[0090] In the first step, the execution subject may input the pre-processed image into the image feature extraction layer to generate image feature information, wherein the image feature information may represent the image feature semantics of the pre-processed image.

[0091] In the second step, the execution subject may input the image feature information into the multi-scale feature output layer to generate feature information at the target dimension reduction scale as the target scale feature information. The target scale feature information may be feature information with the largest dimension reduction scale in the multi-scale dimension reduction process. For example, the single-scale feature map of FPN output is 10x14, which is equivalent to dividing the original image into a 10x14 grid.

[0092] It should be noted that the reason why the FPN output is designed for the target object of chicken is that the chicken coop has three layers: top, middle and bottom. The height of the grid is set to 10 grids, which is the best result after careful design and experimentation. The width of the grid is set to 14 grids. Considering that the aisle occupies a large width and area of ​​the imaging, and the chicken coop is only on the left and right sides of the image, 14 grids is also the best result after careful design and experimentation.

[0093] In a third step, the execution entity may input the target scale feature information into the detection head layer to generate the target object detection information.

[0094] In some optional implementations of some embodiments, the target object detection information includes at least one of the following: the corresponding position information of the target storage point in the storage location, the side length information corresponding to the target storage point, the confidence, and the existence information characterizing whether the target object exists in the storage location. The target storage point may be the storage point where two storage location edges (corresponding to two edge labels) intersect in the storage location. In practice, for a storage location in the shape of a rectangular parallelepiped, the corresponding target storage point may be the point where two labeled edges intersect facing the periphery of the storage rack. For example, the target storage point may be the lower left point facing the storage rack channel. The side length information corresponding to the target storage point may be the side information of the two edges corresponding to the target storage location. In practice, the intersection of the two corresponding edges is the target storage location. The edge information may include the position information of the edge, and may also include the edge size information. The confidence may characterize the accuracy of the target object detection information. The higher the confidence, the higher the accuracy of the target object detection information.

[0095] Optionally, the execution entity may input the target scale feature information into the detection head layer to generate at least one of the position information, the side length information, the confidence level, and the existence information.

[0096] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the object detection information generation method of some embodiments of the present disclosure, using the target object detection model, can accurately and efficiently generate object detection information corresponding to the target object storage map.

[0097] Further references Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a model training device, and these device embodiments are Figure 2 Corresponding to the method embodiments shown, the model training device can be specifically applied to various electronic devices.

[0098] like Figure 7As shown, a model training device 700 includes: a first acquisition unit 701, a determination unit 702, an execution unit 703 and a training unit 704. The first acquisition unit 701 is configured to acquire an object storage atlas, wherein the object storage map is an image of a storage rack storing a target object; the determination unit 702 is configured to determine at least one crop growth experimental model associated with the target crop information; the execution unit 703 is configured to perform the following generation steps for each object storage map in the object storage atlas: edge labeling the storage rack displayed in the object storage map to generate an edge label set, wherein the edge label represents the position of the storage location in the storage rack and represents whether the storage location has a target object; generating initial training data according to the object storage map and the edge label set; performing data preprocessing on the initial training data to generate training data; the training unit 704 is configured to perform model training on the initial target object detection model according to the obtained training data set to generate a target object detection model.

[0099] In some optional implementations of some embodiments, the execution unit 703 may be further configured to: perform coordinate marking and color marking on the edges at the target positions corresponding to the storage racks displayed in the object storage diagram to generate an edge label set, wherein the edges corresponding to the target positions are connecting edges between the target storage locations in the storage racks.

[0100] In some optional implementations of some embodiments, the execution unit 703 may be further configured to: perform image scaling on the initial training data according to a preset image scaling ratio to generate training data.

[0101] It is understandable that the units recorded in the model training device 700 are similar to those in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the model training device 700 and the units contained therein, and will not be repeated here.

[0102] Further references Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an object detection information generating device, and these device embodiments are Figure 5 Corresponding to the method embodiments shown, the object detection information generating device can be specifically applied to various electronic devices.

[0103] like Figure 8As shown, an object detection information generating device 800 includes: a second acquiring unit 801, a preprocessing unit 802 and a generating unit 803. The second acquiring unit 801 is configured to acquire a target object storage map; the preprocessing unit 802 is configured to perform image preprocessing on the target object storage map to generate a preprocessed image; the generating unit 803 is configured to input the preprocessed image into a pre-trained target object detection model to generate target object detection information, wherein the target object detection model is generated based on a model training method.

[0104] It can be understood that the units described in the object detection information generating device 800 are similar to those described in the reference Figure 5 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the object detection information generating device 800 and the units included therein, and will not be described in detail here.

[0105] Reference below Fig. 9 , which shows an electronic device (eg, Figure 1 Schematic diagram of the structure of the electronic device 101)900. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0106] like Fig. 9 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory 902 or a program loaded from a storage device 908 into a random access memory 903. Various programs and data required for the operation of the electronic device 900 are also stored in the random access memory 903. The processing device 901, the read-only memory 902, and the random access memory 903 are connected to each other via a bus 904. An input / output interface 905 is also connected to the bus 904.

[0107] Typically, the following devices may be connected to the input / output interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Fig. 9 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0108] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 909, or installed from the storage device 908, or installed from the read-only memory 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0109] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0110] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0111] The computer-readable medium may be included in the electronic device; or it may exist separately without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains an object storage atlas, wherein the object storage atlas is an image of a storage rack storing a target object; for each object storage atlas in the object storage atlas, performs the following generation steps to edge-label the storage rack displayed in the object storage atlas to generate an edge label set, wherein the edge label represents the position of the storage location in the storage rack and represents whether the storage location has a target object; generates initial training data based on the object storage atlas and the edge label set; performs data preprocessing on the initial training data to generate training data; performs model training on the initial target object detection model based on the obtained training data set to generate a target object detection model. Obtain a target object storage map; perform image preprocessing on the target object storage map to generate a preprocessed image; input the preprocessed image into a pre-trained target object detection model to generate target object detection information, wherein the target object detection model is generated based on a model training method.

[0112] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0114] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising a first acquisition unit, a determination unit, an execution unit, and a training unit. Among them, the names of these units do not constitute a limitation on the unit itself in certain cases, for example, the first acquisition unit may also be described as a "unit for acquiring an object storage atlas".

[0115] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0116] Some embodiments of the present disclosure also provide a computer program product, including a computer program, which implements any of the above-mentioned object detection information generation methods or model training methods when executed by a processor.

[0117] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A model training method, comprising: Obtaining an object storage atlas, wherein the object storage atlas is an image of a storage rack storing a target object; For each object storage graph in the object storage graph set, the following generation steps are performed: Performing edge labeling on the storage racks displayed in the object storage diagram to generate an edge label set, wherein the edge label represents the position of the storage location in the storage rack and represents whether the target object exists in the storage location; generating initial training data according to the object storage graph and the edge label set; Performing data preprocessing on the initial training data to generate training data; According to the obtained training data set, the initial target object detection model is trained to generate a target object detection model.

2. The method according to claim 1, wherein: The edge labeling of the storage racks displayed in the object storage diagram to generate an edge label set includes: The edges at the target positions corresponding to the storage racks displayed in the object storage diagram are labeled with coordinates and colors to generate an edge label set, wherein the edges corresponding to the target positions are connecting edges between the target storage locations in the storage racks.

3. The method according to claim 1, wherein: The performing data preprocessing on the initial training data to generate training data includes: The initial training data is image scaled according to a preset image scaling ratio to generate training data.

4. A method for generating object detection information, comprising: Get the target object storage map; Performing image preprocessing on the target object storage map to generate a preprocessed image; The preprocessed image is input into a pre-trained target object detection model to generate target object detection information, wherein the target object detection model is generated based on the method of any one of claims 1-3.

5. The method according to claim 4, wherein: The target object detection model includes: an image feature extraction layer, a multi-scale feature output layer and a detection head layer, wherein the multi-scale feature output layer is a network layer that outputs feature information of the target at a reduced dimension scale.

6. The method according to claim 5, wherein: The step of inputting the pre-processed image into a pre-trained target object detection model to generate target object detection information includes: Inputting the preprocessed image into the image feature extraction layer to generate image feature information; Inputting the image feature information into the multi-scale feature output layer to generate feature information at the target dimension reduction scale as target scale feature information; The target scale feature information is input into the detection head layer to generate the target object detection information.

7. The method according to claim 6, wherein: The target object detection information includes at least one of the following: position information corresponding to the target storage point in the storage location, side length information corresponding to the target storage point, confidence, and existence information indicating whether the target object exists in the storage location; as well as The step of inputting the target scale feature information into the detection head layer to generate the target object detection information includes: The target scale feature information is input into the detection head layer to generate at least one of the position information, the side length information, the confidence level, and the existence information.

8. A model training device, comprising: A first acquisition unit is configured to acquire an object storage atlas, wherein the object storage atlas is an image of a storage rack storing a target object; a determining unit configured to determine at least one crop growth experimental model associated with the target crop information; The execution unit is configured to perform the following generation steps for each object storage graph in the object storage graph set: edge labeling of the storage racks displayed in the object storage graph to generate an edge label set, wherein the edge label represents the position of the storage location in the storage rack and represents whether there is a target object in the storage location; generating initial training data according to the object storage graph and the edge label set; performing data preprocessing on the initial training data to generate training data; The training unit is configured to perform model training on the initial target object detection model according to the obtained training data set to generate a target object detection model.

9. An object detection information generating device, comprising: A second acquisition unit is configured to acquire a target object storage map; A preprocessing unit, configured to perform image preprocessing on the target object storage map to generate a preprocessed image; A generating unit is configured to input the preprocessed image into a pre-trained target object detection model to generate target object detection information, wherein the target object detection model is generated based on the method of one of claims 1-3.

10. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

11. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

12. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images