Object order determination method based on deep learning, computer readable storage medium

Abstract

This invention provides a method for determining the order of objects based on deep learning and a computer-readable storage medium. The method includes the following steps: S1, establishing an OrderNet network model, including using a convolutional network as the backbone for object feature detection and establishing a classification detection branch network in the detection head; S2, establishing an image dataset containing various object orderings; pre-encoding each object order; S3, using the dataset as input to the OrderNet network model; when the backbone network determines the object features, the classification detection branch network outputs its prediction vector; S4, decoding the prediction vector and obtaining the corresponding encoding to query the object's corresponding order. This achieves the ability to directly predict the order of multiple objects in an image and whether the order is correct, overcoming the limitations of existing technologies that can only process objects by segmentation and recombination.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method for determining the order of objects based on deep learning and a computer-readable storage medium. Background Technology

[0002] Deep learning (DL) is a new research direction in the field of machine learning (ML). Specifically, deep learning learns the inherent patterns and hierarchical representations of sample data. The information gained during this learning process greatly helps in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to have analytical and learning capabilities like humans, and to recognize data such as text, images, and sound.

[0003] The order of objects plays an extremely important role in many fields. For example, cable installation requires the correct sequence of each plug; the placement and order of electronic components such as chips during production and processing follow certain rules; and the sequential recognition of text or symbols will affect their true meaning.

[0004] Current deep learning-based methods for determining order typically involve segmenting and recombining the objects to be judged. This involves first segmenting and decomposing the whole or multiple objects from the image, then identifying and detecting them individually, and finally recombining them into a whole to determine the order of multiple objects in the image.

[0005] It is evident that current technologies are not yet able to directly predict and determine whether the sorting order of multiple objects is correct based on deep learning technology without segmentation processing and individual recognition. Summary of the Invention

[0006] Therefore, the main objective of this invention is to provide a method for determining the order of objects based on deep learning and a computer-readable storage medium to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, a first aspect of the present invention provides a method for determining the order of objects based on deep learning, comprising the following steps:

[0008] S1 establishes the OrderNet network model, including: using a convolutional network as the backbone network for object feature detection, and establishing a classification and detection branch network in the detection head;

[0009] S2 constructs a dataset containing images of various orderings of the target; each ordering of the target is pre-encoded;

[0010] S3 uses the dataset as input to the OrderNet network model. When the backbone network determines the target features, the classification and detection branch network outputs its predicted vector.

[0011] S4 decodes the predicted vector and obtains the corresponding encoding to query the sorting of the target.

[0012] In a possible preferred embodiment, the encoding step in step S2 includes: assuming there are C! sorting methods for C targets, C! image datasets need to be created, each dataset contains m images, and the total number of images is m multiplied by C!; and one-hot encoding is performed on the images for each sorting method.

[0013] In a possible preferred embodiment, the prediction vector is Where N is the total number of rankings C for the C targets, C! This is a one-hot encoded vector.

[0014] In a possible preferred embodiment, the loss function of the classification and detection branch network is cross-entropy: ,in Represents the true distribution. The model output distribution is represented by N, where N represents the total number of rankings C for the C targets.

[0015] In a possible preferred embodiment, the step of decoding the prediction vector in step S4 includes:

[0016] Define a set of encoded sequences of length C targets. ,in ; The quantity is Any element in S is a sequence of length C;

[0017] Define the set of permutations and combinations of classification detection branches as follows: ,in ;

[0018] When set and The injective condition is satisfied, that is any exist one of the Correspondingly, the mapping relationship can be obtained. , where the mapping function ;

[0019] definition ,calculate Obtain the sorted result of the decoding, where It is a continuous OR operator.

[0020] In a possible preferred embodiment, the step of decoding the prediction vector in step S4 further includes:

[0021] calculate To obtain the sequence number of the incorrect position in the sorting result.

[0022] To achieve the above objectives, a second aspect of the present invention also provides a method for determining the order of objects based on deep learning, comprising the following steps:

[0023] S1 establishes the OrderNet network model, including: using a convolutional network as the backbone network for object feature detection, and establishing a classification detection branch network and an object detection branch network in the detection head;

[0024] S2 constructs a dataset containing images of various orderings of the target; each ordering of the target is pre-encoded;

[0025] S3 takes the dataset as input to the OrderNet network model. When the backbone network determines the target features, the classification and detection branch network outputs its predicted vector, while the target detection branch network outputs the location information of the target in the image.

[0026] S4 decodes the prediction vector to obtain the corresponding encoding in order to query the sorting of the target;

[0027] S5 When it is determined that the target sorting obtained in step S4 is incorrect, output the position information of the target in the image where the incorrect sorting is located.

[0028] In a possible preferred embodiment, the object detection branch network includes two optimization objectives: pixel logistic regression loss of the image heatmap and size offset loss of the target box, wherein the pixel logistic regression loss function of the heatmap is:

[0029]

[0030] in The target output after the activation function Let be a Gaussian distribution of keypoints, where α and β are adjustable hyperparameters, N represents the total number of targets in the image, and for a target p, its coordinates in the feature map are denoted as . , It is that coordinate value, It is the standard deviation obtained based on the size of the target p;

[0031] When the actual size of target p is When, the corresponding predicted value is , Then the size loss of the target is defined as: , where W and H are the width and height of the target image, Ω represents the real number field, R is the output size scaling ratio, and C represents the number of target categories in the target detection task.

[0032] In a possible preferred embodiment, the encoding step in step S2 includes: assuming there are C! sorting methods for C targets, C! image datasets need to be created, each dataset contains m images, and the total number of images is m multiplied by C!; and one-hot encoding is performed on the images for each sorting method.

[0033] In a possible preferred embodiment, the step of decoding the prediction vector in step S4 includes:

[0034] Define a set of encoded sequences of length C targets. ,in ; The quantity is Any element in S is a sequence of length C;

[0035] Define the set of permutations and combinations of classification detection branches as follows: ,in ;

[0036] When set and The injective condition is satisfied, that is any exist one of the Correspondingly, the mapping relationship can be obtained. , where the mapping function ;

[0037] definition ,calculate Obtain the sorted result of the decoding, where It is a continuous OR operator.

[0038] In a possible preferred embodiment, step S5, which involves outputting the location information of the incorrectly sorted target within the image, includes:

[0039] calculate To obtain the sequence number of the incorrect position in the sorting result;

[0040] Output the location information of the incorrectly sorted target in the image based on the error position number.

[0041] To achieve the above objectives, a third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the deep learning-based object order determination method as described in any one of the above descriptions.

[0042] The object order determination method and computer-readable storage medium based on deep learning provided by this invention have the advantage of cleverly pre-encoding images containing various object sortings using one-hot encoding. This allows the use of the classification detection branch network in traditional convolutional neural networks, which is originally used for object classification, to achieve object sorting detection. This enables the direct prediction of the sorting order of multiple objects in an image and whether the sorting is correct. This overcomes the limitation of existing technologies that can only use the idea of ​​segmentation and recombination (that is, to segment and decompose the whole or multiple objects, identify and detect them individually, and then recombine them into a whole). Attached Figure Description

[0043] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0044] Figure 1 This is a schematic diagram illustrating the steps of a first embodiment of the object order determination method based on deep learning of the present invention;

[0045] Figure 2 This is a schematic diagram of the OrderNet network model structure of Embodiment 1 of the object order determination method based on deep learning of the present invention.

[0046] Figure 3 This is an example diagram of the one-hot encoding logic of the present invention;

[0047] Figure 4 This is a schematic diagram illustrating the steps of Embodiment 2 of the object order determination method based on deep learning of the present invention;

[0048] Figure 5 This is a schematic diagram of the OrderNet network model structure for Embodiment 2 of the object order determination method based on deep learning of the present invention.

[0049] Figure 6 This is a schematic diagram of the detection process of Embodiment 2 of the object order determination method based on deep learning of the present invention;

[0050] Figure 7 This is a schematic diagram of the OrderNet network model detection head structure in Embodiment 2 of the object order determination method based on deep learning of the present invention. Detailed Implementation

[0051] To enable those skilled in the art to better understand the technical solutions of the present invention, the specific technical solutions of the present invention will be clearly and completely described below in conjunction with embodiments, so as to help those skilled in the art further understand the present invention. Obviously, the embodiments described in this application are merely some embodiments of the present invention, and not all embodiments. It should be noted that, for those skilled in the art, the embodiments and features in the embodiments of this application can be combined with each other without departing from the concept of the present invention and without conflict. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the disclosure and protection scope of the present invention.

[0052] Furthermore, the terms "first," "second," "S1," "S2," etc., used in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those described herein. At the same time, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. Unless otherwise expressly specified and limited, the terms "set," "arranged," "installed," "connected," and "linked" should be interpreted broadly, for example, as a fixed connection, a detachable connection, or an integral connection; a mechanical connection or an electrical connection; a direct connection or an indirect connection through an intermediate medium; or a connection within two elements. Those skilled in the art can understand the specific meaning of the above terms in this case based on the specific circumstances and in conjunction with existing technology. (one)

[0054] Please see Figures 1 to 3 As shown, the object order determination method based on deep learning provided in the first aspect of the present invention includes the following steps:

[0055] S1 establishes the OrderNet network model, including: using a convolutional network as the backbone network for object feature detection, and establishing a classification and detection branch network in the detection head;

[0056] S2 constructs a dataset containing images of various orderings of the target; each ordering of the target is pre-encoded;

[0057] S3 uses the dataset as input to the OrderNet network model. When the backbone network determines the target features, the classification and detection branch network outputs its predicted vector.

[0058] S4 decodes the predicted vector and obtains the corresponding encoding to query the sorting of the target.

[0059] Specifically, such as Figure 2 As shown in the example, the OrderNet network model preferably uses a convolutional network as the backbone for object feature detection. The detection head includes a classification detection branch network that uses 1x1 convolutions instead of fully connected layers. The backbone network is also compatible with various basic models such as ResNet, MobileNet, and DLA.

[0060] In order to achieve the target ranking detection function through the classification detection branch network, this case first needs to encode the various permutations of the target, such as... Figure 3 As shown, the encoding step in step S2 includes: assuming there are C! sorting methods for C targets, C! image datasets need to be created, each dataset contains m images, and the total number of images is m multiplied by C!.

[0061] For example Figure 3 The example demonstrates sorting methods for three different shaped objects: ABC, A, B, and C. There are six possible permutations: ABC, ACB, BAC, BCA, CAB, and CBA. Assume 100 images for each permutation, totaling 600 images as the dataset. Each permutation is then one-hot encoded as follows: 000001, 000010, 000100, 001000, 010000, 100000. This one-hot encoding, also known as one-bit valid encoding, uses an N-bit state register to encode N states. Each state has its own independent register bit, and at any given time, only one bit is valid.

[0062] When training the OrderNet network model, the image samples in the dataset can be scaled to a fixed size (e.g., 640x320). The relevant data in the labels should also be scaled proportionally. One-hot encoding sorting information of the image should be added to the label data. The dataset can be divided into training and testing datasets in a ratio of 9:1 and input into the OrderNet network model. The classification and detection branch can use multiple convolutional layers to perform pooling operations for feature extraction. The last 1x1 convolution replaces the fully connected layer for dimension transformation.

[0063] For example, this classification detection branch network, for The input image is used to detect the target object in the output image. .in This is the prediction vector, where W and H are the width and height of the image, R is the output size scaling factor, C represents the number of target categories in the object detection task, and the length N represents the total number of permutations of C targets, N=C!. This is the One-Hot vector, also known as one-hot encoding. The loss function for this classification and detection branch network uses cross-entropy: ,in Represents the true distribution. This represents the output distribution of the model.

[0064] In fact, this classification detection branch is similar to image classification. It uses two 3x3 convolutional layers for compression extraction in the feature layers of the backbone network, and replaces the fully connected layer with a 1x1 convolutional layer for channel compression. Taking a 640x320 image as an example, the input is 1x3x320x640, and the final result is an image of length... The category to which the vector belongs can be determined by taking the index of the maximum value of a one-dimensional vector after performing a softmax operation on it.

[0065] Furthermore, after obtaining the prediction vector, it needs to be decoded to obtain the corresponding encoding and query the sorting of the target. Therefore, in the design of this invention, the correct sequence number of the target needs to be predetermined, along with the corresponding numbers for various incorrect sequences (of course, if it is not intended solely for judging right or wrong, the correct sequence number can be left unpreset, thus only used to determine the sorting method of the current image). For example, for object type C, there exists... Given various arrangement methods, the predicted lengths of the fully connected layers in the classification and detection branches are the same. Therefore, each position has two states: correct and incorrect. Thus, only one of these two states will exist. There are several encoding methods (there is no permutation with only one incorrect position). Because... There are multiple permutations that correspond to one encoding.

[0066] like Figure 3 As shown, taking three categories as an example, objects A, B, and C have 6 (3x2x1) possible arrangements (ABC, ACB, BAC, BCA, CAB, CBA). The designed order prediction FC layer has a length of 6, corresponding to codes from 000001 to 100000. There are 5 parsing codes: 000, 011, 110, 111, and 101. According to the established mapping relationship, the 6 one-hot codes can be mapped to the 5 parsing codes. This allows us to determine whether the currently predicted arrangement is correct and, in case of an error, the location of the error.

[0067] Therefore, in this example, the steps for decoding the prediction vector include:

[0068] Define a set of encoded sequences of length C targets. ,in ; The quantity is Any element in S is a sequence of length C;

[0069] Define the set of permutations and combinations of classification detection branches as follows: ,in ;

[0070] When set and The injective condition is satisfied, that is any exist one of the Correspondingly, the mapping relationship can be obtained. , where the mapping function Where G is the one-hot encoding to sequence number conversion function, and H is the sequence number to parsing code conversion mapping function;

[0071] definition ,calculate Obtain the sorted result of the decoding, where It is a continuous OR operator, and the final decoded result takes the value 0 or 1, such as Figure 3 As shown, if ABC is set to correct sorting 000, then 0 is correct and 1 is incorrect. If 1 appears in the decoding result, it can be determined that the target in the current image is sorted incorrectly, thereby completing the recognition.

[0072] It is understandable that the above decoding result itself refers to the sorting of targets in the image. Therefore, even if it is not used to judge right or wrong, the sorting of targets in the image can be known from the decoding result.

[0073] Furthermore, in order to obtain the sequence number of the error location of the erroneous target based on the identified error and correct sorting results, in a preferred example, the step of decoding the prediction vector further includes:

[0074] calculate To obtain the sequence number of the error position in the sorting result, for example, when the decoding result is 101, then E is {13}, and at this time it can be known that the first and third targets in the encoding sorting have sorting errors.

[0075] Therefore, the OrderNet network model is trained iteratively using the training sample dataset. The initial learning rate can be set to 0.0002, the gradient update method is Adam, and the batch size is 16. Let the overall training objective of the OrderNet network be Loss: ,in To adjust the coefficients, the default settings are preferably 0.1, 0.5, and 0.5. Set the specified number of training rounds; the training process ends when this number is reached. Then, validate the trained OrderNet model on the test dataset, calculate the model's accuracy, and repeat the above process until the expected accuracy is achieved. After this, it can be used for actual order prediction.

[0076] As can be seen, the object order judgment method based on deep learning provided in this example cleverly pre-encodes images containing various orderings of the target using one-hot encoding. This allows the classification detection branch network in traditional convolutional neural networks, which is originally used for target classification, to be used to detect the target order. This enables the direct prediction of the order of multiple objects in an image and whether the order is correct. This breaks through the limitation of existing technologies that can only use the idea of ​​segmentation and recombination (that is, to segment and decompose the whole or multiple objects, identify and detect them individually, and then recombine them into a whole). (two)

[0078] Please see Figures 3 to 7 As shown, the second aspect of the present invention provides a deep learning-based method for determining the order of objects, the steps of which include:

[0079] S1 establishes the OrderNet network model, including: using a convolutional network as the backbone network for object feature detection, and establishing a classification detection branch network and an object detection branch network in the detection head;

[0080] S2 constructs a dataset containing images of various orderings of the target; each ordering of the target is pre-encoded;

[0081] S3 takes the dataset as input to the OrderNet network model. When the backbone network determines the target features, the classification and detection branch network outputs its predicted vector, while the target detection branch network outputs the location information of the target in the image.

[0082] S4 decodes the prediction vector to obtain the corresponding encoding in order to query the sorting of the target;

[0083] S5 When it is determined that the target sorting obtained in step S4 is incorrect, output the position information of the target in the image where the incorrect sorting is located.

[0084] Specifically, based on Example 1, this example adds an object detection branch network to the detection head of the OrderNet network model to predict the position of each object in the image. In this example, the object detection branch can adopt a similar structure and processing logic as CenterNet, the difference being that there is no offset branch.

[0085] Meanwhile, the target's center detection uses a heatmap approach. The heatmap feature map predicts the center point of the target object, with values ​​increasing closer to the center. The WH feature layer has two layers, and each point predicts the length and width of the object at that point.

[0086] For example, object detection branch networks, for The input image is used to detect the target object in the output image. , where W and H are the width and height of the image, Ω represents the real number field, R is the output size scaling ratio, and C represents the number of target categories in the target detection task.

[0087] This object detection branch includes two optimization objectives: image heatmap loss and bounding box size offset loss. Specifically, the loss function is the pixel logistic regression loss function of the heatmap.

[0088]

[0089] in The target output after the activation function The keypoints are represented by a Gaussian distribution. Here, α and β are adjustable hyperparameters, which can be defaulted to 2 and 4 in this example, while N represents the total number of targets in the image. For a target p, its coordinates in the feature map are... , It is that coordinate value, It is the standard deviation obtained based on the size of the target p.

[0090] Assume the true size of target p is The corresponding predicted value is , The size loss of the target is defined as: .

[0091] The object detection branch aims to detect the object category and bounding box location in the image. The network outputs a heatmap for each category, and peak points need to be extracted to obtain the center location of the bounding box. All response points in the heatmap are compared with their eight neighbors; if a point is greater than or equal to a neighboring point, it is retained. Finally, the top K peak points that meet the requirements are retained. The resulting bounding box... ,in It predicts the coordinates of key points. The width and height of the bounding box predicted for that point are used to obtain the location information of the target in the image.

[0092] After the classification and detection branch obtains the prediction vector and completes the decoding as in the steps of Example 1, when it is determined that the image sorting is incorrect, the location information of the target in the image obtained by the target detection branch network can provide the control basis for the location information of each target in the image for subsequent troubleshooting or automation operations.

[0093] On the other hand, if the classification detection branch further calculates...

[0094] When obtaining the sequence number of the erroneous position in the sorting result, for example, when the decoding result is 101, then E is {13}. At this time, it can be known that the first and third targets in the encoding sorting have sorting errors. Based on the sequence number of the erroneous position, it can further provide a basis for filtering out the position information of the erroneous target in the image from the position information, so as to accurately locate the target that needs to be sorted and provide a basis for subsequent automated control.

[0095] As can be seen, the object order determination method based on deep learning provided in this example can not only cleverly pre-encode images containing various sortings of targets by performing one-hot encoding, but also utilize the classification detection branch network in traditional convolutional neural networks, which is originally used for target classification, to achieve target sorting detection, thereby achieving a prediction result that can directly predict the sorting order of multiple objects in an image and whether the sorting is correct.

[0096] It can also output the location information of the target in the image based on the target sorting and detection results, thereby providing a basis for subsequent automated control. This breaks through the limitation of existing technologies that can only use the idea of ​​segmentation and recombination (that is, to segment and disassemble the whole or multiple objects, identify and detect them separately, and then recombine them into a whole). (three)

[0098] To achieve the above objectives, a third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the deep learning-based object order determination method as described in any one of the above descriptions.

[0099] In summary, the object order determination method and computer-readable storage medium based on deep learning provided by this invention have the advantage of cleverly pre-encoding images containing various object sortings using one-hot encoding. This allows the use of the classification detection branch network in traditional convolutional neural networks, which is originally used for object classification, to achieve object sorting detection. This enables the direct prediction of the sorting order among multiple objects in an image and whether the sorting is correct. Furthermore, based on the object sorting detection results, the position information of the object in the image can be output, thus providing a basis for subsequent automated control. This overcomes the limitations of existing technologies that can only process objects by segmentation and recombination.

[0100] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The present invention is limited only by the claims and their full scope and equivalents. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

[0101] Those skilled in the art will understand that, besides implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0102] Furthermore, all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0103] Furthermore, various different implementations of the present invention can be combined arbitrarily, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

1. A method for determining the order of objects based on deep learning, characterized in that... The steps include: S1 establishes the OrderNet network model, including: using a convolutional network as the backbone network for object feature detection, and establishing a classification and detection branch network in the detection head that replaces the fully connected layers with 1x1 convolutions; S2 constructs a dataset containing images of various orderings of the target; each ordering of the target is pre-encoded; S3 uses the dataset as input to the OrderNet network model. When the backbone network determines the target features, the classification and detection branch network outputs its predicted vector. S4 decodes the prediction vector to obtain the corresponding encoding in order to query the sorting of the target. The steps include: Define a set of encoded sequences of length C targets. ,in ; The quantity is Any element in S is a sequence of length C; Define the set of permutations and combinations of classification detection branches as follows: ,in ; When set and The injective condition is satisfied, that is any exist one of the Correspondingly, the mapping relationship can be obtained. , where the mapping function ; definition ,calculate Obtain the sorted result of the decoding, where It is a continuous OR operator.

2. The object order determination method based on deep learning according to claim 1, characterized in that, Encoding step in step S2 This includes: Suppose there are C targets and C! sorting methods, and C! image datasets need to be created, each dataset containing m images, with a total number of images of m multiplied by C!; Perform one-hot encoding on the images for each sorting method.

3. The object order determination method based on deep learning according to claim 2, characterized in that, The prediction vector is Where N is the total number of sortings C! for the C targets. This is a one-hot encoded vector.

4. The object order determination method based on deep learning according to claim 1, characterized in that, The loss function of the classification and detection branch network is cross-entropy: ,in Represents the true distribution. The model output distribution is represented by N, where N represents the total number of rankings C for the C targets.

5. The object order determination method based on deep learning according to claim 1, characterized in that, The step of decoding the prediction vector in step S4 further includes: calculate To obtain the sequence number of the incorrect position in the sorting result.

6. A method for determining the order of objects based on deep learning, characterized in that... The steps include: S1 establishes the OrderNet network model, including: a convolutional network as the backbone network for object feature detection, a classification detection branch network with 1x1 convolutions replacing fully connected layers in the detection head, and an object detection branch network constructed using the CenterNet structure and processing logic without offset branches. S2 constructs a dataset containing images of various orderings of the target; each ordering of the target is pre-encoded; S3 takes the dataset as input to the OrderNet network model. When the backbone network determines the target features, the classification and detection branch network outputs its predicted vector, while the target detection branch network outputs the location information of the target in the image. S4 decodes the prediction vector to obtain the corresponding encoding in order to query the sorting of the target. The steps include: Define a set of encoded sequences of length C targets. ,in ; The quantity is Any element in S is a sequence of length C; Define the set of permutations and combinations of classification detection branches as follows: ,in ; When set and The injective condition is satisfied, that is any exist one of the Correspondingly, the mapping relationship can be obtained. , where the mapping function ; definition ,calculate Obtain the sorted result of the decoding, where It is a continuous OR operator; S5 When it is determined that the target sorting obtained in step S4 is incorrect, output the position information of the target in the image where the incorrect sorting is located.

7. The object order determination method based on deep learning according to claim 6, characterized in that, The object detection branch network includes two optimization objectives: pixel logistic regression loss based on the image heatmap and object bounding box size offset loss. The pixel logistic regression loss function for the heatmap is: ； in The target output after the activation function Let be a Gaussian distribution of keypoints, where α and β are adjustable hyperparameters, N represents the total number of targets in the image, and for a target p, its coordinates in the feature map are denoted as . , It is that coordinate value, It is the standard deviation obtained based on the size of the target p; When the actual size of target p is When, the corresponding predicted value is , Then the size loss of the target is defined as: , where W and H are the width and height of the target image, Ω represents the real number field, R is the output size scaling ratio, and C represents the number of target categories in the target detection task.

8. The object order determination method based on deep learning according to claim 6, characterized in that, Encoding step in step S2 This includes: Suppose there are C targets and C! sorting methods, and C! image datasets need to be created, each dataset containing m images, with a total number of images of m multiplied by C!; Perform one-hot encoding on the images for each sorting method.

9. The object order determination method based on deep learning according to claim 6, characterized in that, The step S5, which involves outputting the location information of the incorrectly sorted target in the image, includes: calculate To obtain the sequence number of the incorrect position in the sorting result; Output the location information of the incorrectly sorted target in the image based on the error position number.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the deep learning-based object order determination method as described in any one of claims 1-9.

Images