Loading rate prediction method, loading rate prediction device, electronic apparatus, storage medium, and program product

Abstract

The invention provides a loading rate prediction method, a loading rate prediction device, electronic equipment, a storage medium and a program product, which can be applied to the technical field of artificial intelligence and warehouse logistics. The method comprises the following steps: acquiring a target space image of an accommodating space, wherein the target space image comprises a side wall area of the accommodating space in execution of loading operation; inputting the target space image into the target image segmentation model, and outputting a predicted side wall region, the target image segmentation model is obtained by training the initial image segmentation model by using a plurality of training space images, at least one real target label corresponding to the training space images and at least one first pseudo label corresponding to the at least one real target label; the first pseudo label is obtained by predicting a to-be-determined area in the training space image by using a label-free prediction model; and determining the loading rate of the accommodating space according to the predicted side wall area and the real side wall area of the accommodating space.

Description

Technical Field

[0001] The present disclosure relates to the technical field of warehousing logistics, and more specifically, to a loading rate prediction method, a loading rate prediction device, an electronic device, a storage medium and a program product. Background Art

[0002] Identification of the loading rate of storage spaces (such as warehouses and trucks) plays an important role in logistics management. For example, during vehicle loading, identification of the loading rate can help the management system to grasp the progress of loading in real time, provide a basis for scheduling, and determine whether resources need to be reallocated based on the speed of the progress compared to the planned progress. After loading is completed, identification of the loading rate can help determine the rationality of resource allocation (for example, a low loading rate may indicate that the vehicle model used is too large) and provide suggestions for future resource allocation.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: the existing loading rate identification method has poor accuracy during the identification process. Summary of the invention

[0004] In view of this, the present disclosure provides a loading rate prediction method, a loading rate prediction device, an electronic device, a storage medium and a program product.

[0005] One aspect of the present disclosure provides a loading rate prediction method, comprising:

[0006] Acquire a target space image of the accommodation space, wherein the target space image includes a side wall area of ​​the accommodation space during the loading operation;

[0007] Input the target space image into a target image segmentation model, and output a predicted sidewall area, wherein the target image segmentation model is obtained by training an initial image segmentation model using a plurality of training space images and at least one true target label corresponding to the training space image and at least one first pseudo label corresponding to at least one of the true target labels, and the first pseudo label is obtained by predicting the to-be-determined area in the training space image using an unlabeled prediction model;

[0008] The loading rate of the accommodation space is determined according to the predicted side wall area and the actual side wall area of ​​the accommodation space, wherein the actual side wall area represents the side wall area of ​​the accommodation space when it is empty.

[0009] According to an embodiment of the present disclosure, when the accommodation space is a carriage, determining the loading rate of the accommodation space according to the predicted side wall area and the actual side wall area of ​​the accommodation space includes:

[0010] According to the predicted sidewall area and the actual sidewall area, respectively determine the predicted sidewall area and the actual sidewall area;

[0011] The loading rate is determined based on the predicted sidewall area and the actual sidewall area.

[0012] According to an embodiment of the present disclosure, determining the loading rate according to the predicted sidewall area and the actual sidewall area includes:

[0013] Determining area reduction information according to the predicted sidewall area and the actual sidewall area;

[0014] The above-mentioned loading ratio is generated based on the target fitting function and the above-mentioned area reduction information.

[0015] According to an embodiment of the present disclosure, the target image segmentation model is generated in the following manner:

[0016] Acquire an image training set, wherein the image training set includes a plurality of the training space images corresponding to at least one training space and at least one true target label corresponding to each of the training space images, and the training space images include a known space region of a known type corresponding to each of the true target labels and the first pseudo label obtained by predicting at least one unknown type of to-be-determined region through the label-free prediction model;

[0017] For each of the training space images, the initial image segmentation model is used to process the training space image to obtain a plurality of prediction labels, wherein one of the prediction labels corresponds to one of the known space regions or one of the to-be-determined regions;

[0018] Generate a target loss value according to the at least one true target label, the at least one first pseudo label, and the plurality of predicted labels;

[0019] The model parameters of the unlabeled prediction model and the initial image segmentation model are iteratively adjusted according to the target loss value to determine the adjusted initial image segmentation model as the target image segmentation model, and to determine the adjusted unlabeled prediction model as the new unlabeled prediction model.

[0020] According to an embodiment of the present disclosure, generating a target loss value according to the at least one true target label, the at least one first pseudo label and the plurality of predicted labels includes:

[0021] For each of the predicted labels, the first pseudo label and the predicted label corresponding to the predicted label are processed respectively by using the maximum likelihood estimation method to generate a first likelihood result and a second likelihood result;

[0022] Generate a target likelihood result according to the first likelihood result, the second likelihood result and the true target label corresponding to the predicted label;

[0023] The above target loss value is generated based on the above target likelihood result.

[0024] According to an embodiment of the present disclosure, the training space image is processed using the initial image segmentation model to obtain multiple prediction labels, including:

[0025] Processing the training space image using a first feature extraction network to obtain a first image feature corresponding to each space region in the training space image, wherein the space region includes the known space region or the to-be-determined region;

[0026] For each of the first image features, the first image features are processed using a first feature classification network to obtain the predicted label.

[0027] According to an embodiment of the present disclosure, the above-mentioned unlabeled prediction model is generated in the following manner:

[0028] For each of the training space images, use a second feature extraction network to process each known space region in the training space image to obtain a second image feature;

[0029] For each of the second image features, the second image features are processed using a second feature classification network to obtain an initial prediction label, wherein the initial prediction model includes the second feature extraction network and the second feature classification network;

[0030] The network parameters of the initial prediction model are iteratively adjusted according to the initial prediction label and the true target label corresponding to the initial prediction label to obtain the unlabeled prediction model.

[0031] According to an embodiment of the present disclosure, the above-mentioned target loss value is calculated based on the target loss function, and the above-mentioned target loss function is generated based on the above-mentioned unlabeled prediction model, the unlabeled model weight, the above-mentioned initial image segmentation model and the segmentation model weight.

[0032] According to an embodiment of the present disclosure, the unlabeled model weight and the segmentation model weight are determined in the following manner:

[0033] Acquire an image verification set, wherein the image verification set includes a plurality of verification space images and at least one true verification label corresponding to each of the verification space images, and the verification space image includes a spatial region corresponding to each of the true verification labels;

[0034] For each of the above verification space images, the above verification space images are processed using the above target image segmentation model with different initial weight combinations to obtain at least one predicted verification label corresponding to each of the above initial weight combinations, wherein different initial weight combinations include unlabeled model initial weights and segmentation model initial weights with different values;

[0035] Generating a prediction accuracy based on at least one of the prediction verification labels and at least one of the true verification labels;

[0036] An initial weight combination corresponding to a maximum value among the plurality of prediction accuracies is determined as a target weight combination, wherein the target weight combination includes the unlabeled model weight and the segmentation model weight.

[0037] Another aspect of the present disclosure provides a loading rate prediction device, comprising:

[0038] An acquisition module, used for acquiring a target space image of the accommodation space, wherein the target space image includes a side wall area of ​​the accommodation space during the loading operation;

[0039] A prediction module, used for inputting the target space image into a target image segmentation model, and outputting a predicted sidewall area, wherein the target image segmentation model is obtained by training an initial image segmentation model using a plurality of training space images and at least one true target label corresponding to the training space image and at least one first pseudo label corresponding to at least one of the true target labels, and the first pseudo label is obtained by predicting the to-be-determined area in the training space image using an unlabeled prediction model;

[0040] A determination module is used to determine the loading rate of the above-mentioned accommodation space according to the above-mentioned predicted side wall area and the actual side wall area of ​​the above-mentioned accommodation space, wherein the above-mentioned actual side wall area represents the side wall area of ​​the above-mentioned accommodation space when it is empty.

[0041] Another aspect of the present disclosure provides an electronic device, comprising:

[0042] one or more processors;

[0043] a memory for storing one or more programs,

[0044] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0045] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0046] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and when the instructions are executed, are used to implement the method as described above.

[0047] According to an embodiment of the present disclosure, by using a label-free prediction model to predict the area to be determined due to light or obstructions in the training space image to obtain a first pseudo label, combined with the training space image and the target image segmentation model obtained by the real target label, the target image segmentation model is used to collect the target space image of the accommodation space, and an accurate predicted side wall area can be obtained, and the predicted side wall area and the real side wall area of ​​the accommodation space are calculated to obtain the loading rate of the accommodation space. Since the label-free prediction model first predicts the area to be determined, and the target image segmentation model is trained using the predicted first pseudo label, the target image segmentation model can use more accurate labels during the training process, and the target image segmentation model obtained by training can obtain more accurate prediction results when predicting the loading rate of the accommodation space, thereby improving the recognition accuracy of the loading rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0049] Figure 1 A schematic diagram showing that the side of the carriage is difficult to identify due to light and obstructions;

[0050] Figure 2 An exemplary system architecture to which a method for predicting a loading rate and a device for predicting a loading rate according to an embodiment of the present disclosure can be applied is schematically shown;

[0051] Figure 3 A flow chart of a method for predicting a loading rate according to an embodiment of the present disclosure is schematically shown;

[0052] Figure 4 A schematic diagram schematically shows the effect of predicting the side wall area according to an embodiment of the present disclosure;

[0053] Figure 5 A schematic diagram schematically shows the display effect of a target output image according to an embodiment of the present disclosure;

[0054] Figure 6 A schematic diagram of the annotation effect of a training space image according to an embodiment of the present disclosure is schematically shown;

[0055] Figure 7 A block diagram of a loading rate prediction device according to an embodiment of the present disclosure is schematically shown;

[0056] Figure 8A block diagram schematically shows a training device for a target image segmentation model according to an embodiment of the present disclosure; and

[0057] Fig. 9 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0058] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0059] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0060] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0061] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0062] Figure 1 The schematic diagram shows that the side of the car is difficult to identify due to light and obstructions.

[0063] In the related art, the loading rate recognition method of the accommodation space based on computer vision can be divided into two types according to the sensor: based on a monocular camera and based on a depth camera.

[0064] The method based on monocular camera mainly uses deep network to directly predict the loading rate from the carriage image. This end-to-end deep network method does not establish a clear and explainable theoretical relationship between image features and loading rate. It only performs data fitting on the training data set, and the generalization performance is poor. When the loading scene changes slightly, the loading rate result may change dramatically. Some related technologies use deep models to estimate the depth of the carriage image, but the current accuracy of the depth estimation algorithm based on monocular images is still low, which seriously affects the loading rate prediction accuracy. Some related technologies segment the cargo and the carriage opening, and calculate the loading rate by the area ratio of the cargo and the carriage opening. This does not take into account the perspective characteristics of camera imaging, which is larger near and smaller far away. The area ratio of the cargo and the carriage opening is not linearly related to the actual loading rate. In addition, the segmentation of cargo is easily affected by the cargo on the conveyor belt, and the cargo on the conveyor belt is mistakenly counted as the stacked cargo area. In addition, the texture of cargo is ever-changing, which is not conducive to the generalization performance of the cargo segmentation model.

[0065] Related technologies identify the range of unloaded areas in the carriage by segmenting the areas on both sides of the carriage that are not occupied by cargo, and map the area of ​​this area to the actual loading rate through a machine learning model. This method effectively avoids the influence of cargo texture and conveyor belt. However, this solution uses image segmentation technology to determine which pixel areas belong to the side of the carriage, and it is necessary to manually label the pixel areas on the side of the carriage to help train the image segmentation model. However, the pictures collected in actual production scenarios are often unable to label whether some pixel areas belong to the side of the carriage due to light and occlusion. For example Figure 1 In the figure, the edge of the cargo and the carriage in the red rectangle on the left cannot be determined due to the light, and the area behind the fan shielding in the red rectangle on the right cannot be determined to be cargo or the edge of the carriage. Whether these areas can be reasonably labeled has a great impact on the data quality, and thus affects the effect of the model.

[0066] In view of this, the present disclosure provides a loading rate prediction method, a loading rate prediction device, an electronic device, a storage medium and a program product, the method comprising acquiring a target space image of a storage space, wherein the target space image includes a side wall area of ​​the storage space when performing a loading operation; inputting the target space image into a target image segmentation model, and outputting a predicted side wall area, wherein the target image segmentation model is obtained by training an initial image segmentation model using multiple training space images and at least one real target label corresponding to the training space image and at least one first pseudo label corresponding to the at least one real target label, and the first pseudo label is obtained by predicting a to-be-determined area in the training space image using an unlabeled prediction model; determining the loading rate of the storage space based on the predicted side wall area and the real side wall area of ​​the storage space, the real side wall area representing the side wall area of ​​the storage space when empty.

[0067] In the embodiments of the present disclosure, the collection, use, storage, sharing and transfer of the data involved (for example, including but not limited to user personal information) shall comply with the provisions of relevant laws and regulations, and it is necessary to inform the user and obtain the user's consent or authorization. When applicable, the user's personal information shall be de-identified and / or anonymized and / or encrypted.

[0068] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0069] Figure 2 The exemplary system architecture 200 to which the load rate prediction method and load rate prediction device according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 2 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0070] like Figure 2 As shown, the system architecture 200 according to this embodiment may include a first terminal device 201, a second terminal device 202, a third terminal device 203, a network 204, and a server 205. The network 204 is used to provide a medium for communication links between the first terminal device 201, the second terminal device 202, the third terminal device 203, and the server 205. The network 204 may include various connection types, such as wired and / or wireless communication links, etc.

[0071] The user may use the first terminal device 201, the second terminal device 202, and the third terminal device 203 to interact with the server 205 through the network 204 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 201, the second terminal device 202, and the third terminal device 203, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0072] The first terminal device 201, the second terminal device 202, and the third terminal device 203 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0073] The server 205 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 201, the second terminal device 202, and the third terminal device 203. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0074] It should be noted that the loading rate prediction method provided in the embodiment of the present disclosure can generally be executed by the server 205. Accordingly, the loading rate prediction device provided in the embodiment of the present disclosure can generally be set in the server 205. The loading rate prediction method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 205 and can communicate with the first terminal device 201, the second terminal device 202, the third terminal device 203 and / or the server 205. Correspondingly, the loading rate prediction device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 205 and can communicate with the first terminal device 201, the second terminal device 202, the third terminal device 203 and / or the server 205. Alternatively, the loading rate prediction method provided in the embodiment of the present disclosure can also be executed by the first terminal device 201, the second terminal device 202 or the third terminal device 203, or can also be executed by other terminal devices different from the first terminal device 201, the second terminal device 202 or the third terminal device 203. Accordingly, the loading rate prediction device provided in the embodiment of the present disclosure may also be arranged in the first terminal device 201, the second terminal device 202 or the third terminal device 203, or in other terminal devices different from the first terminal device 201, the second terminal device 202 or the third terminal device 203.

[0075] For example, the space image to be processed may be originally stored in any one of the first terminal device 201, the second terminal device 202 or the third terminal device 203 (for example, the first terminal device 201, but not limited thereto), or stored in an external storage device and may be imported into the first terminal device 201. Then, the first terminal device 201 may locally execute the loading rate prediction method provided by the embodiment of the present disclosure, or send the space image to be processed to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the space image to be processed may execute the loading rate prediction method provided by the embodiment of the present disclosure.

[0076] It should be understood that Figure 2 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0077] Figure 3 The flowchart of the loading rate prediction method according to the embodiment of the present disclosure is schematically shown. Figure 4 The schematic diagram schematically shows the effect of predicting the sidewall area according to an embodiment of the present disclosure.

[0078] like Figure 3 As shown, the loading rate prediction method includes operations S301 to S303.

[0079] In operation S301, a target space image of the accommodation space is acquired, wherein the target space image includes a side wall area of ​​the accommodation space during a loading operation;

[0080] In operation S302, the target space image is input into a target image segmentation model, and a predicted side wall region is output, wherein the target image segmentation model is obtained by training an initial image segmentation model using a plurality of training space images and at least one true target label corresponding to the training space image and at least one first pseudo label corresponding to the at least one true target label, and the first pseudo label is obtained by predicting a to-be-determined region in the training space image using an unlabeled prediction model;

[0081] In operation S303 , a loading rate of the accommodation space is determined according to the predicted sidewall area and the actual sidewall area of ​​the accommodation space, wherein the actual sidewall area represents the sidewall area of ​​the accommodation space when empty.

[0082] According to an embodiment of the present disclosure, the accommodating space may refer to a compartment of a box truck, a compartment of a train, a cargo hold of an airplane, a container, etc. The present disclosure uses a compartment of a truck for exemplary description.

[0083] According to an embodiment of the present disclosure, the real target label includes a mark for each area on the side wall of the accommodation space, such as a mark on the side wall that can be intuitively seen as a side wall, and a mark that cannot be determined from the image whether it is a side wall (i.e., the area to be determined) due to light or obstructions, but the staff can determine whether it is a side wall. The first pseudo-label is a mark of whether it is a side wall obtained by predicting the area to be determined by the unlabeled prediction model. Among them, the real target label and the first pseudo-label and other labels can be represented by probability values, for example, 1 represents a side wall, and 0 represents not a side wall.

[0084] According to an embodiment of the present disclosure, a feature classification network (non-initial image segmentation model) is trained using known spatial regions in training spatial images and their corresponding true target labels (excluding the region to be determined) to obtain an unlabeled prediction model. The first pseudo label of the region to be determined is obtained by predicting the region to be determined in the training spatial image using the trained network, and thus the target image segmentation model is obtained by training the initial image segmentation model using multiple training spatial images and at least one true target label corresponding to the training spatial image and at least one first pseudo label corresponding to at least one true target label.

[0085] According to the embodiments of the present disclosure, after the target image segmentation model is trained, the target space image of the current accommodation space can be collected periodically or in real time, for example, once every 3 minutes. For a certain type of van, the specific area size of the side wall of the van compartment is known before loading (i.e., empty). The collected target space image is input into the target image segmentation model to output the corresponding predicted side wall area, such as Figure 4 As shown, the predicted sidewall area can be marked with colors, for example, Figure 4 (a) and Figure 4 The green area in (b), where the output of the target image segmentation model is the probability value of each pixel belonging to the side of the car body, the present disclosure determines the pixel area greater than the probability threshold (e.g. 0.8) as the predicted side wall area. Thus, the loading rate of the truck car body at the current moment can be obtained by calculating the predicted side wall area and the specific area size of the side wall of the truck car body when it is empty (i.e. the actual side wall area).

[0086] It should be noted that the target space image can be collected using any type of image acquisition device such as a monocular camera or a depth camera. In order to reduce storage costs, a lower-cost monocular camera can be used for image acquisition.

[0087] According to an embodiment of the present disclosure, by using a label-free prediction model to predict the area to be determined due to light or obstructions in the training space image to obtain a first pseudo label, combined with the training space image and the target image segmentation model obtained by the real target label, the target image segmentation model is used to collect the target space image of the accommodation space, and an accurate predicted side wall area can be obtained, and the predicted side wall area and the real side wall area of ​​the accommodation space are calculated to obtain the loading rate of the accommodation space. Since the label-free prediction model first predicts the area to be determined, and the target image segmentation model is trained using the predicted first pseudo label, the target image segmentation model can use more accurate labels during the training process, and the target image segmentation model obtained by training can obtain more accurate prediction results when predicting the loading rate of the accommodation space, thereby improving the recognition accuracy of the loading rate.

[0088] Figure 5 A schematic diagram schematically shows the display effect of a target output image according to an embodiment of the present disclosure.

[0089] According to an embodiment of the present disclosure, when the accommodation space is a carriage, determining the loading rate of the accommodation space according to the predicted side wall area and the actual side wall area of ​​the accommodation space includes:

[0090] According to the predicted side wall area and the actual side wall area, the predicted side wall area and the actual side wall area are determined respectively;

[0091] Based on the predicted sidewall area and the actual sidewall area, the loading rate is determined.

[0092] According to an embodiment of the present disclosure, after the predicted sidewall area is determined, the predicted sidewall area can be calculated. Since the sidewalls of this type of compartment can be determined based on vehicle parameters before loading, the actual real sidewall area of ​​the compartment can also be calculated based on vehicle parameters. At this time, the loading rate can be calculated based on the predicted sidewall area and the real sidewall area.

[0093] In a specific embodiment, there is a certain relationship between the ratio of the predicted side wall area and the actual side wall area predicted during the loading process of the carriage and the actual loading rate, so the ratio can be determined as the predicted loading rate.

[0094] According to an embodiment of the present disclosure, the loading rate prediction method further includes: generating a target output image according to the target space image and the loading rate, and displaying the target output image.

[0095] According to the embodiments of the present disclosure, Figure 5As shown, the target output image can be displayed on a display or a mobile terminal such as a mobile phone, so that the staff can grasp the loading status of the carriage in a timely manner.

[0096] In another embodiment, determining the loading rate according to the predicted sidewall area and the actual sidewall area includes:

[0097] Determining area reduction information based on the predicted sidewall area and the actual sidewall area;

[0098] Based on the target fitting function and the area reduction information, a loading ratio is generated.

[0099] According to the embodiment of the present disclosure, the ratio of the area reduction on both sides of the carriage is related to the loading rate. This relationship can be fitted by the data consisting of the area reduction of the carriage side walls and the actual loading rate in the previous loading process, thereby obtaining the target fitting function g(). For example, if the ratio of area reduction is z, the corresponding loading rate can be processed by the target fitting function g(), so that the loading rate can be represented by g(z).

[0100] Figure 6 A schematic diagram of the annotation effect of a training space image according to an embodiment of the present disclosure is schematically shown.

[0101] According to an embodiment of the present disclosure, the target image segmentation model is generated in the following manner:

[0102] Acquire an image training set, wherein the image training set includes a plurality of training space images corresponding to at least one training space and at least one true target label corresponding to each training space image, and the training space image includes a known space region of a known type corresponding to each true target label and a first pseudo label obtained by predicting at least one unknown type of to-be-determined region through a label-free prediction model;

[0103] For each training space image, the initial image segmentation model is used to process the training space image to obtain a plurality of prediction labels, wherein one prediction label corresponds to a known space region or a region to be determined;

[0104] generating a target loss value according to at least one true target label, at least one first pseudo label, and a plurality of predicted labels;

[0105] The model parameters of the unlabeled prediction model and the initial image segmentation model are iteratively adjusted according to the target loss value to determine the adjusted initial image segmentation model as the target image segmentation model, and to determine the adjusted unlabeled prediction model as the new unlabeled prediction model.

[0106] According to an embodiment of the present disclosure, the initial image segmentation model may be a Deeplabv3 model, but is not limited to a specific semantic segmentation type. Existing semantic segmentation models (other algorithms in the Deeplab series, such as Unet, etc.) may be used as the initial image segmentation model.

[0107] According to the embodiments of the present disclosure, when labeling the training space images, it can be done in software tools such as open source Labelme. There are three types of labels, namely, the side of the carriage, the non-side of the carriage, and uncertain. The area or pixel marked as uncertain corresponds to the unlabeled data of the area to be determined. There are corresponding real target labels for the known space area and the area to be determined. For example, for a certain area to be determined, the staff will give a real target label of whether it is / is not the side of the carriage according to the actual situation of the carriage.

[0108] In a specific embodiment, the training space image is annotated, and the effect of the annotation is as follows: Figure 6 As shown, the dark red area is the area corresponding to the side of the car, and the green area is the uncertain area. Figure 6 In the image, the area behind the fan (i.e., the obstruction) is not visible and cannot be distinguished as the side of the carriage, so it is marked as an undetermined area. The undetermined area on the left is affected by light and the boundary between the cargo and the carriage cannot be clearly identified. Since all other areas except these two areas are not the side of the carriage, only the side of the carriage and the uncertain area need to be marked during the image annotation process, and the unmarked area is automatically identified as the non-side of the carriage area.

[0109] According to an embodiment of the present disclosure, in the training space image in the training set, the area to be determined in the training space image can be predicted by the label-free prediction model to obtain a predicted first pseudo-label, and the first pseudo-label indicates whether the area to be determined is predicted by the label-free prediction model to be the side of the car.

[0110] According to an embodiment of the present disclosure, the training space image and the first pseudo label are input into the initial image segmentation model for prediction, so as to obtain multiple predicted labels of the training space image, wherein each predicted label represents whether an area or pixel in the training space image belongs to the side of the car.

[0111] According to an embodiment of the present disclosure, the target loss value can be generated by inputting the true target label, the first pseudo label and the predicted label into the target loss function, and the model parameters of the unlabeled prediction model and the initial image segmentation model are adjusted based on the target loss value to determine the adjusted initial image segmentation model as the target image segmentation model, and determine the adjusted unlabeled prediction model as the new unlabeled prediction model.

[0112] In a specific embodiment, the initial image segmentation model can be expressed as ,in For training space images, For the image segmentation model Line The output of the column, Indicates the probability that the pixel belongs to the side area of ​​the car, that is, the predicted label, is the true category of this pixel, 1 indicates the side of the car, and 0 indicates it is not the side of the car. In specific implementation, a deep neural network can be used to build an initial image segmentation model, and the probability corresponding to each pixel is activated using a sigmoid function, limiting it to between 0 and 1. Once an output data set is collected (a set of pictures and the predicted labels of each pixel in each picture in the set), the Adam optimizer can be used to minimize the cross entropy loss and train the target image segmentation model.

[0113] According to an embodiment of the present disclosure, in the process of adjusting the model parameters of the initial image segmentation model using the target loss value, the trained unlabeled prediction model is also updated, so that the subsequent pseudo-label predictions can be closer to the true target label of the area to be determined. As a result, the prediction accuracy of the target image segmentation model is improved due to the more accurate image training set.

[0114] According to an embodiment of the present disclosure, generating a target loss value according to at least one true target label, at least one first pseudo label and a plurality of predicted labels includes:

[0115] For each predicted label, the first pseudo label and the predicted label corresponding to the predicted label are processed respectively by using the maximum likelihood estimation method to generate a first likelihood result and a second likelihood result;

[0116] Generate a target likelihood result according to the first likelihood result, the second likelihood result, and the true target label corresponding to the predicted label;

[0117] Generates a target loss value based on the target likelihood result.

[0118] According to an embodiment of the present disclosure, for a training space image And the true target label set , the overall cross entropy loss corresponding to the training space image can be expressed as ,in is the cross entropy loss of a single pixel. After considering the unlabeled data, the loss corresponding to the unlabeled data (i.e. the area to be determined) also needs to be added to the overall loss function.

[0119] According to an embodiment of the present disclosure, in order to achieve the above purpose, a label-free prediction model (expressed as ), only the labeled pixels (i.e. the known spatial regions determined to be the side walls of the car and the non-side walls of the car) are used for training. Represents the model structure of the unlabeled prediction model, Represents the first pseudo label output by the model.

[0120] Assume that the training space image The true target label set is , the training loss function can be inferred by the maximum likelihood estimation method. In the training space image, the probability of each pixel being the side of the car follows the Bernoulli distribution. On this basis, the first likelihood result of the unlabeled prediction model is shown in formula (1):

[0121] (1)

[0122] The corresponding loss function is The second likelihood result of the initial image segmentation model is shown in formula (2):

[0123] (2)

[0124] The corresponding loss function includes and Two parts. Multiplying these two likelihood results gives the overall target likelihood result, thus obtaining the target loss function as shown in formula (3), from which the target loss value can be calculated:

[0125] (3)

[0126] The loss function can be optimized by the Adam optimizer. Express The output and true target label Optimizing consistency between Express The output and true target label Optimizing consistency between Express and Optimize the consistency between the outputs. and is a hyperparameter, and Represent the target image segmentation model and the unlabeled prediction model respectively.

[0127] According to an embodiment of the present disclosure, during the optimization process of the Adam optimizer, a certain learning rate and iteration number threshold can be set, for example, the iteration number threshold is 1000 times, the learning rate is 0.003, and the iteration can be terminated when any difference between multiple adjacent target loss values ​​is less than a preset threshold, and the initial image segmentation model of the last iteration is determined as the target image segmentation model, wherein the preset threshold can be set according to actual needs, for example, it can be 0.5.

[0128] According to an embodiment of the present disclosure, the training space image is processed using the initial image segmentation model to obtain multiple prediction labels, including:

[0129] Processing the training space image using a first feature extraction network to obtain a first image feature corresponding to each spatial region in the training space image, wherein the spatial region includes a known spatial region or a region to be determined;

[0130] For each first image feature, the first image feature is processed using the first feature classification network to obtain a predicted label.

[0131] According to the embodiments of the present disclosure, the first feature extraction network can be a resnet50 network or other types of neural networks for feature extraction, the first feature classification network can be a feature classification network in a Deeplabv3 network or other types of neural networks capable of performing feature classification, and the initial image segmentation model can be constructed using the first feature extraction network and the first feature classification network. .

[0132] According to an embodiment of the present disclosure, using the initial image segmentation model The first feature extraction network in the training space image extracts features to obtain the first image features corresponding to each space region, and then uses the first feature classification network to process the first image features to obtain a prediction label, which represents whether the space region belongs to the side of the car. The prediction label can be represented by a probability value.

[0133] According to an embodiment of the present disclosure, the unlabeled prediction model is generated in the following manner:

[0134] For each training space image, using a second feature extraction network to process each known space region in the training space image to obtain a second image feature;

[0135] For each second image feature, the second image feature is processed using a second feature classification network to obtain an initial prediction label, wherein the initial prediction model includes a second feature extraction network and a second feature classification network;

[0136] The network parameters of the initial prediction model are iteratively adjusted according to the initial prediction labels and the true target labels corresponding to the initial prediction labels to obtain an unlabeled prediction model.

[0137] According to the embodiments of the present disclosure, the second feature extraction network can be a resnet18 network or other types of neural networks for feature extraction, and the second feature classification network can be a feature classification network in a Deeplabv3 network or other types of neural networks capable of performing feature classification. The second feature extraction network and the second feature classification network can be used to form an unlabeled prediction model. .

[0138] According to the embodiment of the present disclosure, the second feature extraction network is used to process the known spatial region that has been determined to be the side of the car, and the corresponding second image feature can be obtained. The second feature classification network is then used to process the second image feature to obtain an initial prediction label that characterizes whether it is the side of the car. The network parameters of the initial prediction model are adjusted according to the initial prediction label and the corresponding true target label that is unambiguously determined by manual annotation, thereby obtaining an unlabeled prediction model.

[0139] According to an embodiment of the present disclosure, by using a known spatial region of whether it is a car side and its corresponding real target label to train an unlabeled prediction model, the trained unlabeled prediction model can be used to predict the type of the area to be determined, thereby providing a target image segmentation model with more accurate training samples, thereby improving the prediction accuracy of the target image segmentation model.

[0140] According to an embodiment of the present disclosure, the target loss value is calculated based on the target loss function, and the target loss function is generated based on the unlabeled prediction model, the unlabeled model weight, the initial image segmentation model and the segmentation model weight.

[0141] According to an embodiment of the present disclosure, the unlabeled model weight and the segmentation model weight may be the above-mentioned hyperparameters and .

[0142] In a specific embodiment, and It can be determined based on prior knowledge, for example, it can be set to 1 and 2 respectively.

[0143] In another specific embodiment, the unlabeled model weight and the segmentation model weight may also be determined in the following manner:

[0144] Acquire an image verification set, wherein the image verification set includes a plurality of verification space images and at least one true verification label corresponding to each verification space image, and the verification space image includes a spatial region corresponding to each true verification label;

[0145] For each verification space image, the verification space image is processed using the target image segmentation model with different initial weight combinations to obtain at least one predicted verification label corresponding to each initial weight combination, wherein the different initial weight combinations include unlabeled model initial weights and segmentation model initial weights with different values;

[0146] generating a prediction accuracy based on at least one predicted verification label and at least one true verification label;

[0147] An initial weight combination corresponding to a maximum value among a plurality of prediction accuracies is determined as a target weight combination, wherein the target weight combination includes an unlabeled model weight and a segmentation model weight.

[0148] According to an embodiment of the present disclosure, an image verification set is set, each verification space image in the image verification set has at least one corresponding real verification label, and the verification space image may include a known space area and a to-be-determined area, and each area has a corresponding real verification label.

[0149] According to an embodiment of the present disclosure, firstly, multiple initial weight combinations are listed for the initial weights of the unlabeled model and the initial weights of the segmentation model in formula (3), for example, the initial weight combination is [ ], a and b can be any value from 1 to 10.

[0150] According to an embodiment of the present disclosure, for a target image segmentation model of each initial weight combination, each verification space image in an image verification set is processed using the target image segmentation model, and for each predicted verification label and corresponding true verification label outputted, the prediction accuracy of the target image segmentation model of the initial weight combination is generated.

[0151] According to the embodiment of the present disclosure, the prediction accuracy of the target image segmentation model of different initial weight combinations is compared, and the initial weight combination with the highest prediction accuracy is determined as the target weight combination, thereby determining and The specific value of .

[0152] According to an embodiment of the present disclosure, by processing the image verification set using the target image segmentation model under different initial weight combinations, the most appropriate unlabeled model weights and segmentation model weights are determined based on the prediction accuracy, thereby further improving the prediction accuracy of the target image segmentation model.

[0153] Figure 7 A block diagram of a loading rate prediction device according to an embodiment of the present disclosure is schematically shown.

[0154] like Figure 7 As shown, the loading rate prediction device 700 includes an acquisition module 710 , a prediction module 720 , and a determination module 730 .

[0155] An acquisition module 710 is used to acquire a target space image of the accommodation space, wherein the target space image includes a side wall area of ​​the accommodation space during the loading operation;

[0156] A prediction module 720 is used to input the target space image into a target image segmentation model, and output a predicted sidewall area, wherein the target image segmentation model is obtained by training an initial image segmentation model using a plurality of training space images and at least one true target label corresponding to the training space image and at least one first pseudo label corresponding to the at least one true target label, and the first pseudo label is obtained by predicting the to-be-determined area in the training space image using an unlabeled prediction model;

[0157] The determination module 730 is used to determine the loading rate of the accommodation space according to the predicted side wall area and the actual side wall area of ​​the accommodation space, wherein the actual side wall area represents the side wall area of ​​the accommodation space when it is empty.

[0158] According to an embodiment of the present disclosure, by using a label-free prediction model to predict the area to be determined due to light or obstructions in the training space image to obtain a first pseudo label, combined with the training space image and the target image segmentation model obtained by the real target label, the target image segmentation model is used to collect the target space image of the accommodation space, and an accurate predicted side wall area can be obtained, and the predicted side wall area and the real side wall area of ​​the accommodation space are calculated to obtain the loading rate of the accommodation space. Since the label-free prediction model first predicts the area to be determined, and the target image segmentation model is trained using the predicted first pseudo label, the target image segmentation model can use more accurate labels during the training process, and the target image segmentation model obtained by training can obtain more accurate prediction results when predicting the loading rate of the accommodation space, thereby improving the recognition accuracy of the loading rate.

[0159] According to an embodiment of the present disclosure, when the accommodating space is a carriage, the determining module 730 includes a first determining unit and a second determining unit.

[0160] A first determining unit, configured to determine a predicted sidewall area and a real sidewall area respectively according to the predicted sidewall area and the real sidewall area;

[0161] The second determining unit is used to determine the loading rate according to the predicted side wall area and the actual side wall area.

[0162] According to an embodiment of the present disclosure, the second determining unit includes a determining subunit and a generating subunit.

[0163] A determination subunit, for determining area reduction information according to the predicted sidewall area and the actual sidewall area;

[0164] The generating subunit is used to generate a loading ratio according to the target fitting function and the area reduction information.

[0165] Figure 8 A block diagram of a training device for a target image segmentation model according to an embodiment of the present disclosure is schematically shown.

[0166] like Figure 8 As shown, the training device 800 of the target image segmentation model includes a second acquisition module 810, a first obtaining module 820, a generation module 830, and a training module 840.

[0167] A second acquisition module 810 is used to acquire an image training set, wherein the image training set includes a plurality of training space images corresponding to at least one training space and at least one real target label corresponding to each training space image, and the training space image includes a known space region of a known type corresponding to each real target label and a first pseudo label obtained by predicting at least one unknown type of to-be-determined region through a label-free prediction model;

[0168] A first obtaining module 820 is used to process each training space image using the initial image segmentation model to obtain a plurality of prediction labels, wherein one prediction label corresponds to a known space region or a region to be determined;

[0169] A generating module 830, configured to generate a target loss value according to at least one true target label, at least one first pseudo label, and a plurality of predicted labels;

[0170] The training module 840 is used to iteratively adjust the model parameters of the unlabeled prediction model and the initial image segmentation model according to the target loss value to determine the adjusted initial image segmentation model as the target image segmentation model and determine the adjusted unlabeled prediction model as the new unlabeled prediction model.

[0171] According to an embodiment of the present disclosure, the generation module 830 includes a first generation unit, a second generation unit, and a third generation unit.

[0172] A first generating unit is used to process the first pseudo label and the predicted label corresponding to each predicted label respectively by using the maximum likelihood estimation method to generate a first likelihood result and a second likelihood result;

[0173] A second generating unit, configured to generate a target likelihood result according to the first likelihood result, the second likelihood result, and a true target label corresponding to the predicted label;

[0174] The third generating unit is used to generate a target loss value according to the target likelihood result.

[0175] According to an embodiment of the present disclosure, the first obtaining module 820 includes a first obtaining unit and a second obtaining unit.

[0176] A first obtaining unit is used to process the training space image using a first feature extraction network to obtain a first image feature corresponding to each spatial region in the training space image, wherein the spatial region includes a known spatial region or a region to be determined;

[0177] The second obtaining unit is used to process the first image feature using the first feature classification network for each first image feature to obtain a predicted label.

[0178] According to an embodiment of the present disclosure, the unlabeled prediction model is generated by a second obtaining module, a third obtaining module, and an adjustment module.

[0179] A second obtaining module is used for processing each known spatial region in the training spatial image using a second feature extraction network for each training spatial image to obtain a second image feature;

[0180] A third obtaining module is used for processing the second image features using a second feature classification network for each second image feature to obtain an initial prediction label, wherein the initial prediction model includes a second feature extraction network and a second feature classification network;

[0181] The adjustment module is used to iteratively adjust the network parameters of the initial prediction model according to the initial prediction label and the true target label corresponding to the initial prediction label to obtain the unlabeled prediction model.

[0182] According to an embodiment of the present disclosure, the target loss value is calculated based on the target loss function, and the target loss function is generated based on the unlabeled prediction model, the unlabeled model weight, the initial image segmentation model and the segmentation model weight.

[0183] According to an embodiment of the present disclosure, the unlabeled model weight and the segmentation model weight are determined by an acquisition unit, a processing unit, a fourth generation unit, and a third determination unit.

[0184] An acquisition unit, configured to acquire an image verification set, wherein the image verification set includes a plurality of verification space images and at least one true verification label corresponding to each verification space image, and the verification space image includes a spatial region corresponding to each true verification label;

[0185] A processing unit is used to process the verification space image using the target image segmentation model with different initial weight combinations for each verification space image, and obtain at least one predicted verification label corresponding to each initial weight combination, wherein the different initial weight combinations include unlabeled model initial weights and segmentation model initial weights with different values;

[0186] a fourth generating unit, configured to generate a prediction accuracy based on at least one prediction verification label and at least one true verification label;

[0187] The third determining unit is used to determine an initial weight combination corresponding to a maximum value among multiple prediction accuracies as a target weight combination, wherein the target weight combination includes an unlabeled model weight and a segmentation model weight.

[0188] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0189] For example, any of the acquisition module 710, the prediction module 720, the determination module 730, or the second acquisition module 810, the first acquisition module 820, the generation module 830, and the training module 840 can be combined in one module / unit / sub-unit for implementation, or any of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the acquisition module 710, the prediction module 720, the determination module 730, or the second acquisition module 810, the first acquisition module 820, the generation module 830, and the training module 840 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 710, the prediction module 720, the determination module 730, or the second acquisition module 810, the first acquisition module 820, the generation module 830, and the training module 840 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0190] It should be noted that the loading rate prediction device part in the embodiment of the present disclosure corresponds to the loading rate prediction method part in the embodiment of the present disclosure, and the description of the loading rate prediction device part specifically refers to the loading rate prediction method part, which will not be repeated here. Similarly, the training device part of the target image segmentation model in the embodiment of the present disclosure corresponds to the training method part of the target image segmentation model in the embodiment of the present disclosure, and the description of the training device part of the target image segmentation model specifically refers to the training method part of the target image segmentation model, which will not be repeated here.

[0191] Fig. 9 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0192] like Fig. 9As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0193] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0194] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0195] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0196] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0197] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0198] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 902 and / or the RAM 903 described above and / or one or more memories other than the ROM 902 and the RAM 903 .

[0199] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the loading rate prediction method or the target image segmentation model training method provided by the embodiment of the present disclosure.

[0200] When the computer program is executed by the processor 901, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0201] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0202] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0203] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0204] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for predicting a loading rate, comprising: Acquire a target space image of the accommodation space, wherein the target space image includes a side wall area of ​​the accommodation space during the loading operation; Inputting the target space image into a target image segmentation model, and outputting a predicted sidewall area, wherein the target image segmentation model is obtained by training an initial image segmentation model using a plurality of training space images and at least one true target label corresponding to the training space image and at least one first pseudo label corresponding to at least one of the true target labels, and the first pseudo label is obtained by predicting a to-be-determined area in the training space image using an unlabeled prediction model; The loading rate of the accommodation space is determined according to the predicted side wall area and the actual side wall area of ​​the accommodation space, wherein the actual side wall area represents the side wall area of ​​the accommodation space when it is empty.

2. The method according to claim 1, wherein: In the case where the accommodation space is a carriage, determining the loading rate of the accommodation space according to the predicted side wall area and the actual side wall area of ​​the accommodation space includes: Determine a predicted sidewall area and a real sidewall area respectively according to the predicted sidewall area and the real sidewall area; The loading rate is determined based on the predicted sidewall area and the actual sidewall area.

3. The method according to claim 2, wherein: Determining the loading rate according to the predicted sidewall area and the actual sidewall area includes: determining area reduction information according to the predicted sidewall area and the actual sidewall area; The loading ratio is generated according to the target fitting function and the area reduction information.

4. The method according to claim 1, wherein: The target image segmentation model is generated in the following way: Acquire an image training set, wherein the image training set includes a plurality of training space images corresponding to at least one training space and at least one true target label corresponding to each of the training space images, and the training space images include a known space region of a known type corresponding to each of the true target labels and the first pseudo label obtained by predicting at least one unknown type of to-be-determined region through the label-free prediction model; For each of the training space images, the initial image segmentation model is used to process the training space image to obtain a plurality of prediction labels, wherein one of the prediction labels corresponds to one of the known space regions or one of the regions to be determined; Generate a target loss value according to the at least one true target label, the at least one first pseudo label, and the plurality of predicted labels; The model parameters of the unlabeled prediction model and the initial image segmentation model are iteratively adjusted according to the target loss value to determine the adjusted initial image segmentation model as the target image segmentation model, and to determine the adjusted unlabeled prediction model as a new unlabeled prediction model.

5. The method according to claim 4, wherein: Generating a target loss value according to the at least one true target label, the at least one first pseudo label, and the plurality of predicted labels, comprises: For each of the predicted labels, the first pseudo label and the predicted label corresponding to the predicted label are processed respectively by using a maximum likelihood estimation method to generate a first likelihood result and a second likelihood result; Generate a target likelihood result according to the first likelihood result, the second likelihood result, and a true target label corresponding to the predicted label; The target loss value is generated according to the target likelihood result.

6. The method according to claim 4, wherein: The training space image is processed using the initial image segmentation model to obtain a plurality of predicted labels, including: Processing the training space image using a first feature extraction network to obtain a first image feature corresponding to each space region in the training space image, wherein the space region includes the known space region or the to-be-determined region; For each of the first image features, the first image features are processed using a first feature classification network to obtain a predicted label.

7. The method according to any one of claims 1 to 4, wherein The unlabeled prediction model is generated in the following way: For each of the training space images, use a second feature extraction network to process each known space region in the training space image to obtain a second image feature; For each of the second image features, use a second feature classification network to process the second image features to obtain an initial prediction label, the initial prediction model including the second feature extraction network and the second feature classification network; The network parameters of the initial prediction model are iteratively adjusted according to the initial prediction label and the true target label corresponding to the initial prediction label to obtain the unlabeled prediction model.

8. The method according to claim 4 or 5, wherein: The target loss value is calculated based on a target loss function, and the target loss function is generated according to the unlabeled prediction model, the unlabeled model weight, the initial image segmentation model, and the segmentation model weight.

9. The method according to claim 8, wherein: The unlabeled model weight and the segmentation model weight are determined as follows: Acquire an image verification set, wherein the image verification set includes a plurality of verification space images and at least one true verification label corresponding to each of the verification space images, and the verification space image includes a spatial region corresponding to each of the true verification labels; For each of the verification space images, the verification space image is processed using the target image segmentation model with different initial weight combinations to obtain at least one predicted verification label corresponding to each of the initial weight combinations, wherein different initial weight combinations include unlabeled model initial weights and segmentation model initial weights with different values; generating a prediction accuracy based on at least one of the prediction verification labels and at least one of the true verification labels; An initial weight combination corresponding to a maximum value among the plurality of prediction accuracies is determined as a target weight combination, wherein the target weight combination includes the unlabeled model weight and the segmentation model weight.

10. A loading rate prediction device, comprising: An acquisition module, configured to acquire a target space image of the accommodation space, wherein the target space image includes a side wall area of ​​the accommodation space during the loading operation; A prediction module, used for inputting the target space image into a target image segmentation model, and outputting a predicted sidewall area, wherein the target image segmentation model is obtained by training an initial image segmentation model using a plurality of training space images and at least one true target label corresponding to the training space image and at least one first pseudo label corresponding to at least one of the true target labels, and the first pseudo label is obtained by predicting a to-be-determined area in the training space image using an unlabeled prediction model; A determination module is used to determine the loading rate of the accommodation space according to the predicted side wall area and the actual side wall area of ​​the accommodation space, wherein the actual side wall area represents the side wall area of ​​the accommodation space when it is empty.

11. An electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images