Freight compartment loading rate identification method and device
Through the combination of deep detection network and hidden Markov model, the problem that monocular cameras cannot accurately estimate the load rate of the freight compartment is solved, and more accurate load rate evaluation is achieved, reducing costs and avoiding the impact of perspective effect.
Patent Information
- Application Number
- CN202411203839.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
AI Technical Summary
The freight foy load rate identification method based on a monocular camera cannot accurately obtain depth information, resulting in excessive load rate estimation, especially when the cargo is not fully placed, the perspective phenomenon affects the nonlinear relationship between area changes and depth changes.
The depth estimation of the cargo in the image sequence is performed through the depth detection network, the cargo volume is calculated, and the loading rate is calculated based on the size of the freight box. The loading rate with the highest confidence is determined based on the loading rate changes using the Hidden Markov model, and the Viterbi algorithm is used to determine the loading rate and The observation matrix transfer matrix solves the state sequence, corrects the loading rate estimation, and avoids the influence of perspective effect.
A more accurate cargo loading rate assessment is achieved, reducing costs, ensuring the accuracy of identification, and avoiding miscalculation of loading rate caused by incomplete cargo placement.
Smart Images

Figure CN119091352A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly to a method and device for identifying a loading rate of a freight compartment. Background Art
[0002] Freight compartment refers to the space for loading goods in various means of transportation such as trucks, trains, airplanes and ships, and can also be called cargo hold. The identification of freight compartment loading rate plays an important role in logistics management. During the loading process of the freight compartment, the identification of loading rate can help the management system to grasp the progress of loading in real time, provide a basis for scheduling, and judge whether resources need to be reallocated based on the speed of the progress compared to the planned progress. After loading is completed, the identification of loading rate can help judge the rationality of resource allocation (for example, a loading rate that is too low may indicate that the vehicle model used is too large) and provide suggestions for future resource allocation. The current computer vision-based freight compartment loading rate identification methods can be divided into two types according to the sensor: monocular camera-based and depth camera-based.
[0003] The overall cost of the method based on a monocular camera is low, but a common problem is that it is impossible to obtain accurate depth information using a monocular camera, and it can only be estimated from the information on the two-dimensional plane. Due to the influence of the perspective phenomenon, the information on the two-dimensional plane and the depth information are not a simple linear relationship. In other words, the change in the area of the cargo or freight compartment area and the change in the depth of the cargo present a nonlinear relationship. When loading, the cargo is sometimes piled in certain locations in the middle of the freight compartment, and then placed at the end of the freight compartment and stacked after more loaders return. If the picture at this time is selected for estimation, the loading rate will be overestimated. Summary of the invention
[0004] The embodiments of the present disclosure provide a method and device for identifying a loading rate of a freight compartment.
[0005] In a first aspect, an embodiment of the present disclosure provides a method for identifying a loading rate of a freight compartment, comprising: in response to receiving a request to identify the loading rate of a freight compartment, determining a reporting time period and a sampling frequency according to the size of the freight compartment; acquiring images inside the freight compartment at the sampling frequency within the time period to obtain an image sequence; performing loading rate identification on images in the image sequence to obtain an observed loading rate sequence; and determining a loading rate with the highest confidence based on changes in the loading rates in the loading rate sequence.
[0006] In some embodiments, loading rate recognition is performed on images in an image sequence to obtain an observed loading rate sequence, including: estimating the depth of the cargo in each image in the image sequence through a depth detection network to obtain cargo depth information; for each image, calculating the cargo volume according to the cargo depth information of the image; and calculating the observed loading rate sequence according to the cargo volume and cargo compartment size corresponding to each image.
[0007] In some embodiments, the loading rate with the highest confidence is determined based on the change of the loading rate in the loading rate sequence, including: discretizing the loading rate sequence to obtain an estimated loading rate sequence as an observation sequence of a preset hidden Markov model; solving the state sequence of the hidden Markov model based on a predefined observation matrix and transfer matrix as an actual loading rate sequence; and determining, based on the observation matrix, from the actual loading rate sequence the actual loading rate that is closest to the estimated loading rate sequence as the loading rate with the highest confidence.
[0008] In some embodiments, the method further includes: reporting the acquisition time of the image corresponding to the loading rate with the highest confidence.
[0009] In some embodiments, the method further includes: in response to the loading rate with the highest confidence corresponding to two or more acquisition times, determining the loading rate corresponding to the image with the latest acquisition time as the loading rate with the highest confidence.
[0010] In some embodiments, solving the state sequence of the hidden Markov model as the actual loading rate sequence according to the predefined observation matrix and transfer matrix includes: solving the state sequence of the hidden Markov model as the actual loading rate sequence according to the predefined observation matrix and transfer matrix based on the Viterbi algorithm.
[0011] In some embodiments, the length of the observation sequence and the length of the state sequence of the hidden Markov model are determined according to the size of the freight compartment.
[0012] In a second aspect, an embodiment of the present disclosure provides a device for identifying a loading rate of a freight compartment, comprising: a request unit, configured to determine a reporting time period and a sampling frequency according to the size of the freight compartment in response to receiving a request for identifying the loading rate of the freight compartment; a sampling unit, configured to acquire images inside the freight compartment at the sampling frequency within a time period to obtain an image sequence; an identification unit, configured to perform loading rate identification on images in the image sequence to obtain an observed loading rate sequence; and a determination unit, configured to determine a loading rate with the highest confidence level according to changes in the loading rates in the loading rate sequence.
[0013] In some embodiments, the recognition unit is further configured to: estimate the depth of the cargo in each image in the image sequence through a depth detection network to obtain cargo depth information; for each image, calculate the cargo volume based on the cargo depth information of the image; and calculate the observed loading rate sequence based on the cargo volume and cargo compartment size corresponding to each image.
[0014] In some embodiments, the determination unit is further configured to: discretize the loading rate sequence to obtain an estimated loading rate sequence as an observation sequence of a preset hidden Markov model; solve the state sequence of the hidden Markov model according to a predefined observation matrix and transfer matrix as an actual loading rate sequence; and determine, according to the observation matrix, from the actual loading rate sequence the actual loading rate that is closest to the estimated loading rate sequence as the loading rate with the highest confidence.
[0015] In some embodiments, the device further includes a statistical unit configured to: report the acquisition time of the image corresponding to the loading rate with the highest confidence.
[0016] In some embodiments, the statistical unit is further configured to: in response to the loading rate with the highest confidence corresponding to two or more acquisition times, determine the loading rate corresponding to the image with the latest acquisition time as the loading rate with the highest confidence.
[0017] In some embodiments, the determination unit is further configured to: solve the state sequence of the hidden Markov model as the actual loading rate sequence based on the Viterbi algorithm according to the predefined observation matrix and the transfer matrix.
[0018] In some embodiments, the length of the observation sequence and the length of the state sequence of the hidden Markov model are determined according to the size of the freight compartment.
[0019] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device on which one or more computer programs are stored, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement a method as described in any one of the first aspects.
[0020] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of the first aspects.
[0021] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which implements the method as described in any one of the first aspect or the second aspect when executed by a processor.
[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0024] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;
[0025] Figure 2 is a flow chart of an embodiment of a method for identifying a loading rate of a freight compartment according to the present disclosure;
[0026] Figure 3 is a schematic diagram of an application scenario of the freight compartment loading rate identification method according to the present disclosure;
[0027] Figure 4 is a flow chart of another embodiment of a method for identifying a loading rate of a freight compartment according to the present disclosure;
[0028] Figure 5 is a structural schematic diagram of an embodiment of a freight compartment loading rate identification device according to the present disclosure;
[0029] Figure 6 It is a structural diagram of a computer system of an electronic device suitable for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION
[0030] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0031] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0032] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the freight compartment loading rate identification method or freight compartment loading rate identification apparatus of the present disclosure may be applied.
[0033] like Figure 1As shown, system architecture 100 may include cargo box 101, network 102 and server 103. Network 102 is used to provide a medium for a communication link between cargo box 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links or fiber optic cables, etc.
[0034] Cameras 1011 and 1012 may be installed inside the cargo compartment 101 to collect images inside the cargo compartment, and then send the collected images to the server 103. The server 103 identifies the loading rate based on the images inside the cargo compartment and feeds it back to the logistics personnel to provide suggestions for future resource allocation.
[0035] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, multiple software or software modules used to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.
[0036] It should be noted that the freight compartment loading rate identification method provided in the embodiments of the present disclosure is generally executed by the server 103 , and accordingly, the freight compartment loading rate identification device is generally disposed in the server 103 .
[0037] It should be understood that Figure 1 The number of freight compartments, networks and servers in the embodiment is only illustrative. Any number of freight compartments, networks and servers may be provided as required.
[0038] Continue to refer Figure 2 , shows a process 200 of an embodiment of a method for identifying a loading rate of a freight compartment according to the present disclosure. The method for identifying a loading rate of a freight compartment comprises the following steps:
[0039] Step 201, in response to receiving a request to identify a freight compartment loading rate, determining a reporting time period and a sampling frequency according to the freight compartment size.
[0040] In this embodiment, the execution body of the freight compartment loading rate identification method (for example Figure 1The server shown in the figure) can receive a request for identifying the loading rate of the freight compartment through a wired connection or a wireless connection. The request may include the model of the freight compartment, or the number of the freight compartment (for example, the license plate number, the flight number, etc.). The size of the freight compartment can be queried by the license plate number or the flight number. The user can also directly enter the size of the freight compartment when sending the request. Since the loading progress of freight compartments of different sizes is different, the reporting time period can be determined according to the size of the freight compartment. For example, a 9-meter-long compartment reports the loading rate every 30 minutes. The present application does not only collect images during the reporting period, but collects images multiple times within a time period, identifies a loading rate based on each image, and selects the most reliable loading rate from them. Therefore, it is also necessary to determine the sampling frequency, for example, sampling 6 times within the time period. The larger the size of the freight compartment, the longer the reporting time period and the higher the required sampling frequency.
[0041] Step 202, acquiring images in the freight compartment at a sampling frequency within a time period to obtain an image sequence.
[0042] In this embodiment, it is only necessary to acquire a predetermined number of images in the freight compartment at a sampling frequency, and it is not necessary to acquire the entire video. For example, a 9-meter-long compartment needs to acquire 6 images within half an hour. These images are arranged in the order of acquisition time to form an image sequence. These images can be images without depth information acquired by a monocular camera, or images with depth information acquired by a binocular camera.
[0043] Step 203, performing loading rate recognition on the images in the image sequence to obtain an observed loading rate sequence.
[0044] In this embodiment, for images without depth information captured by a monocular camera, the image sequence of the cargo compartment and the corresponding image sequence inside the compartment can be respectively input into the trained neural network, and the temporal features of each image sequence can be extracted, and finally the loading rate classification result can be obtained through the neural network. For images with depth information captured by a binocular camera, the cargo volume is calculated according to the depth of the cargo in the target cargo compartment, and the cargo volume is divided by the pre-measured volume of the target cargo compartment to obtain the loading rate of the target cargo compartment.
[0045] Step 204: Determine the loading rate with the highest confidence based on the change of the loading rate in the loading rate sequence.
[0046] In this embodiment, during the loading process, the loading rate should be gradually increased in chronological order. If there is a decrease in the middle, the loading rate is not credible. Unreliable loading rates can be filtered out. The average loading rate change rate can be calculated based on the loading rate information in the historical loading process. The real-time loading rate change rate is calculated based on any two adjacent loading rates to obtain a group of real-time loading rate change rates. From the group of real-time loading rate change rates, two adjacent loading rates corresponding to the real-time loading rate change rate closest to the average loading rate change rate are determined, and the loading rate with the latest time in these two adjacent loading rates is used as the loading rate with the highest confidence. Similarly, during the unloading process, the loading rate should be gradually reduced in chronological order. If there is an increase in the middle, the loading rate is not credible. Unreliable loading rates can be filtered out. Then, the loading rate with the highest confidence is determined by the average loading rate change rate calculated by the loading rate information in the historical unloading process. This method is the same as the method in the loading process, so it will not be repeated.
[0047] The method provided by the above-mentioned embodiments of the present disclosure can be used in conjunction with the existing loading rate identification method. The theoretical basis of this scheme is that it is difficult for the loading rate to change too much in a short period of time, and in actual operation, the loading rate does not need to be updated at a particularly high frequency (for example, for a large box truck of about 9 meters in length, uploading once every half an hour can meet the requirements, and for a small box truck, once every 10 minutes can also meet the requirements). Based on the above conclusions, the strategy of this scheme is to sample and identify the loading rate at a higher frequency (such as 2 minutes), and judge which samples have the highest confidence in the past period (such as 10 minutes) based on the changes in the loading rate.
[0048] In some optional implementations of the present embodiment, loading rate recognition is performed on images in an image sequence to obtain an observed loading rate sequence, including: estimating the depth of the cargo in each image in the image sequence through a depth detection network to obtain cargo depth information; for each image, calculating the cargo volume according to the cargo depth information of the image; and calculating the observed loading rate sequence according to the cargo volume and the size of the freight compartment corresponding to each image. The outline of the cargo can be determined based on the cargo depth information, thereby calculating the cargo volume. The total volume can be calculated based on the size of the freight compartment, and the loading rate is obtained by dividing the cargo volume by the total volume. A loading rate can be calculated for each image to form a loading rate sequence. This method utilizes the existing depth detection network, which can reduce costs and ensure recognition accuracy.
[0049] In some optional implementations of this embodiment, the method further includes: reporting the acquisition time of the image corresponding to the loading rate with the highest confidence. Although the reporting is still periodic, the acquisition time interval of the image corresponding to the loading rate with the highest confidence does not have a regular periodicity, so the acquisition time can be reported to provide data support for subsequent resource allocation.
[0050] In some optional implementations of this embodiment, reporting the acquisition time of the image corresponding to the loading rate with the highest confidence includes: in response to the loading rate with the highest confidence corresponding to two or more acquisition times, determining the loading rate corresponding to the image with the latest acquisition time as the loading rate with the highest confidence. If there are two loading rates with the highest confidence, the latest acquisition shall prevail. The time to be reported is also the acquisition time corresponding to the image.
[0051] Continue to see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an application scenario of the method for identifying the loading rate of a freight compartment according to this embodiment. Figure 3 In the application scenario, the goods are first piled in the middle of the carriage. At this time, the cargo volume calculated by image recognition of the depth of the goods is higher than the actual cargo volume, which will cause the calculated loading rate to be falsely high. The solution of this application will obtain images in the cargo compartment according to the sampling frequency within a time period and calculate the loading rate respectively. In this way, the images when the goods are pushed into the end of the carriage can also be obtained, and a set of loading rates can be calculated. According to the loading rate change rules summarized in the historical data, the loading rate can be filtered out. Figure 3 The loading rate corresponding to the image in the figure is then determined based on the average loading rate change rate to determine the loading rate with the highest confidence, and then the loading rate with the highest confidence and the corresponding image acquisition time are reported.
[0052] Further references Figure 4 , which shows a process 400 of another embodiment of a method for identifying a freight compartment loading rate. The process 400 of the method for identifying a freight compartment loading rate comprises the following steps:
[0053] Step 401, in response to receiving a request to identify a freight compartment loading rate, determining a reporting time period and a sampling frequency according to the freight compartment size.
[0054] Step 402, acquiring images in the freight compartment at a sampling frequency within a time period to obtain an image sequence.
[0055] Step 403, performing loading rate recognition on the images in the image sequence to obtain an observed loading rate sequence.
[0056] Steps 401-403 are substantially the same as steps 201-203, and thus are not described in detail herein.
[0057] Step 404 , discretize the loading rate sequence to obtain an estimated loading rate sequence as an observation sequence of a preset hidden Markov model.
[0058] In this embodiment, the accuracy of the loading rate identified by step 403 is inconsistent and needs to be discretized to obtain a quantized value with uniform accuracy. For example, the loading rate sequence of 6%, 12%, and 17% becomes 5%, 10%, and 20% after discretization, that is, the accuracy is uniformly 0.05.
[0059] Hidden Markov Model (HMM) is used to describe a Markov process with hidden unknown parameters. In the Markov model, each state represents an observable event, so the Markov model is sometimes called a visible Markov model (VMM), which limits the adaptability of the model to some extent.
[0060] In a normal Markov model, the state is directly visible to the observer. Thus, the transition probability of the state is the only parameter. In a hidden Markov model, the state is not directly visible, but some variables affected by the state are visible. Each state has a probability distribution over the possible output symbols. Therefore, the sequence of output symbols can reveal some information about the sequence of states.
[0061] This scheme uses a hidden Markov model to describe the loading progress. The state in the hidden Markov model represents the actual loading rate, and the observation in the hidden Markov model represents the loading rate estimated by the algorithm. The loading rate is discretized into N sub-intervals, from 1 / N to 1. In addition, 0 is also considered as a case. Therefore, there are N+1 states and observations respectively, and the set of states is [s 0 ,…,s N ], the set of observations is [o 0 ,…,o N ]. At the same time, define the state transition probability P(s j |s i ) is the state of the previous moment s i , and the next moment state is s j The probability of observation. Define the observation probability P(o j |s i ) is the state s i , and the observation is o j For actual application scenarios, the number of states and their corresponding probabilities can be set based on experience, but the basic principle is that the transition probability between adjacent states is greater than the transition probability between non-adjacent states, and the observation probability between states and observations corresponding to the same loading rate is greater than the observation probability corresponding to different loading rates.
[0062] Step 405 , solving the state sequence of the hidden Markov model as the actual loading rate sequence according to the predefined observation matrix and transfer matrix.
[0063] In this embodiment, after completing the above definition, the results of periodic sampling and recognition are analyzed, and the most likely loading rate is estimated. Taking a 9-meter-long box truck as an example, assuming that the business demand is to report the loading rate every 30 minutes, the system can extract a picture from the camera monitoring the inside of the freight compartment every 5 minutes, and use a visual algorithm based on a monocular camera to estimate the loading rate. After collecting 6 pictures of 30 minutes and their corresponding loading rate estimation results, they are mapped to the sequence of observations. Discretize it and represent it with [o(1), o(2),…, o(M)]. Among them, M is the sequence length, which is 6 in this example. In other application scenarios, it needs to be determined according to the reporting and sampling cycle. According to the sequence of loading rate estimation and the pre-defined state transition probability and observation probability, it is necessary to obtain the state sequence with the highest probability. This step can be solved by the following methods: forward algorithm, Viterbi algorithm and Baum-Welch algorithm.
[0064] The observation matrix and the transfer matrix can be obtained through sample learning. For example, the maximum likelihood estimation or Bayesian estimation method can be used to estimate the values of these two parameters based on the given state sequence and observation sequence.
[0065] Step 406, according to the observation matrix, determine the actual loading rate closest to the estimated loading rate sequence from the actual loading rate sequence as the loading rate with the highest confidence.
[0066] In this embodiment, after the state estimation is completed, a state sequence [s(1), s(2), ..., s(M)] is obtained. For each estimated load rate, it is compared with the corresponding element in the state sequence [s(1), s(2), ..., s(M)], and the one closest to the corresponding element in the state sequence and its corresponding sampling time are returned. For example, the load rate closest to the corresponding state is obtained by sampling at 15:05, and the estimated load rate is 56%, so the timestamp and the corresponding load rate are reported together. If there are two or more time points where the load rate is equal to the corresponding state, the latest one is reported.
[0067] The closest loading rate of each element in the state sequence can be determined by the observation matrix. Assume that the observation matrix is:
[0068] <![CDATA[P(o j |s i )]]> <![CDATA[s 1 ]]> <![CDATA[s 2 ]]> <![CDATA[s 3 ]]> <![CDATA[o 1 ]]> 0.8 0.4 0.3 <![CDATA[o 2 ]]> 0.1 0.2 0.5 <![CDATA[o 3 ]]> 0.1 0.4 0.2
[0069] For the estimated loading rate o 1 , o2 , o 3 , find the closest state from the observation matrix, and you can see that the maximum value in the observation matrix is 0.8, which means o 1 With s 1 The closest, can be s 1 As the loading rate with the highest confidence. 1 The corresponding image acquisition time is reported.
[0070] If the maximum value in the observation matrix appears more than twice, the predicted loading rate with the most recent image acquisition time is taken as the loading rate with the highest confidence.
[0071] The method provided by the above-mentioned embodiments of the present disclosure utilizes the time-varying pattern of the loading rate and the correlation between the previous and next time points, and uses a hidden Markov model to correct the estimated loading rate, thereby overcoming the influence of the perspective effect without incurring additional costs, avoiding the wrong estimation of the loading rate caused by the goods not being fully placed in place, and being able to more accurately evaluate the loading rate of the freight compartment.
[0072] In some optional implementations of this embodiment, the state sequence of the hidden Markov model is solved as the actual loading rate sequence according to the predefined observation matrix and transfer matrix, including: based on the Viterbi algorithm, the state sequence of the hidden Markov model is solved as the actual loading rate sequence according to the predefined observation matrix and transfer matrix. The Viterbi algorithm solves the shortest path problem of a fence-type graph, in which the nodes of the graph are organized in columns, the number of nodes in each column can be different, and the nodes in each column can only be connected to the nodes in the adjacent columns, and cannot be connected across columns. The Viterbi algorithm can improve the decoding speed of the hidden Markov model.
[0073] In some optional implementations of this embodiment, the length of the observation sequence and the length of the state sequence of the hidden Markov model are determined according to the size of the freight compartment. The larger the size of the freight compartment, the higher the sampling frequency, and the longer the length of the corresponding observation sequence and the length of the state sequence. This can improve the accuracy of the hidden Markov model prediction.
[0074] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a freight compartment loading rate identification device. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0075] like Figure 5As shown, the cargo compartment loading rate identification device 500 of this embodiment includes: a request unit 501, a sampling unit 502, an identification unit 503 and a determination unit 504. The request unit 501 is configured to determine the reporting time period and sampling frequency according to the cargo compartment size in response to receiving a request for identifying the cargo compartment loading rate; the sampling unit 502 is configured to obtain images in the cargo compartment according to the sampling frequency within the time period to obtain an image sequence; the identification unit 503 is configured to perform loading rate identification on the images in the image sequence to obtain an observed loading rate sequence; the determination unit 504 is configured to determine the loading rate with the highest confidence according to the change of the loading rate in the loading rate sequence.
[0076] In this embodiment, the specific processing of the request unit 501, the sampling unit 502, the identification unit 503 and the determination unit 504 of the freight compartment loading rate identification device 500 can refer to Figure 2 This corresponds to step 201, step 202, step 203 and step 204 in the embodiment.
[0077] In some optional implementations of this embodiment, the recognition unit 503 is further configured to: estimate the depth of the cargo in each image in the image sequence through a depth detection network to obtain cargo depth information; for each image, calculate the cargo volume according to the cargo depth information of the image; and calculate the observed loading rate sequence according to the cargo volume and cargo compartment size corresponding to each image.
[0078] In some optional implementations of this embodiment, the determination unit 504 is further configured to: discretize the loading rate sequence to obtain an estimated loading rate sequence as an observation sequence of a preset hidden Markov model; solve the state sequence of the hidden Markov model according to a predefined observation matrix and transfer matrix as an actual loading rate sequence; and determine, according to the observation matrix, from the actual loading rate sequence the actual loading rate that is closest to the estimated loading rate sequence as the loading rate with the highest confidence.
[0079] In some optional implementations of this embodiment, the device 500 further includes a statistical unit (not shown in the drawings) configured to report the acquisition time of the image corresponding to the loading rate with the highest confidence.
[0080] In some optional implementations of this embodiment, the statistical unit is further configured to: in response to the loading rate with the highest confidence corresponding to two or more acquisition times, determine the loading rate corresponding to the image with the latest acquisition time as the loading rate with the highest confidence.
[0081] In some optional implementations of this embodiment, the determination unit 504 is further configured to: solve the state sequence of the hidden Markov model as the actual loading rate sequence based on the Viterbi algorithm according to the predefined observation matrix and the transfer matrix.
[0082] In some optional implementations of this embodiment, the length of the observation sequence and the length of the state sequence of the hidden Markov model are determined according to the size of the freight compartment.
[0083] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.
[0084] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.
[0085] An electronic device comprises: one or more processors; a storage device on which one or more computer programs are stored, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement the method described in process 200 or 400.
[0086] A computer-readable medium stores a computer program, wherein the computer program implements the method described in process 200 or 400 when executed by a processor.
[0087] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0088] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0089] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0090] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the road area planning method. For example, in some embodiments, the road area planning method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the road area planning method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the road area planning method in any other appropriate manner (e.g., by means of firmware).
[0091] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0092] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0093] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0095] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0096] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a server of a distributed system, or a server combined with a blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server may be a server of a distributed system, or a server combined with a blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.
[0097] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0098] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A method for identifying a loading rate of a freight compartment, comprising: In response to receiving a request to identify a freight compartment loading rate, determining a reporting time period and a sampling frequency according to the freight compartment size; Acquire images in the freight compartment at the sampling frequency within the time period to obtain an image sequence; Performing loading rate recognition on images in the image sequence to obtain an observed loading rate sequence; The loading rate with the highest confidence is determined according to the change of the loading rate in the loading rate sequence.
2. The method according to claim 1, wherein: The step of performing loading rate recognition on the images in the image sequence to obtain an observed loading rate sequence includes: Estimate the depth of the goods in each image in the image sequence through a depth detection network to obtain depth information of the goods; For each image, the cargo volume is calculated based on the cargo depth information of the image; The observed loading rate sequence is calculated based on the cargo volume corresponding to each image and the cargo compartment size.
3. The method according to claim 1, wherein: The step of determining the loading rate with the highest confidence according to the change of the loading rate in the loading rate sequence includes: Discretizing the loading rate sequence to obtain an estimated loading rate sequence as an observation sequence of a preset hidden Markov model; Solving the state sequence of the hidden Markov model as the actual loading rate sequence according to the predefined observation matrix and transfer matrix; According to the observation matrix, the actual loading rate closest to the estimated loading rate sequence is determined from the actual loading rate sequence as the loading rate with the highest confidence.
4. The method according to claim 1, wherein: The method further comprises: Report the acquisition time of the image corresponding to the loading rate with the highest confidence.
5. The method according to claim 1, wherein: The method further comprises: In response to the loading rate with the highest confidence corresponding to two or more acquisition times, the loading rate corresponding to the image with the latest acquisition time is determined as the loading rate with the highest confidence.
6. The method according to claim 3, wherein: The step of solving the state sequence of the hidden Markov model as the actual loading rate sequence according to the predefined observation matrix and transfer matrix includes: Based on the Viterbi algorithm, the state sequence of the hidden Markov model is solved according to the predefined observation matrix and transfer matrix as the actual loading rate sequence.
7. The method according to claim 3, wherein: The length of the observation sequence and the length of the state sequence of the hidden Markov model are determined according to the size of the freight compartment.
8. A freight compartment loading rate identification device, comprising: a request unit configured to determine a reporting time period and a sampling frequency according to the size of the freight compartment in response to receiving a request to identify a freight compartment loading rate; a sampling unit configured to acquire images in the freight compartment at the sampling frequency within the time period to obtain an image sequence; an identification unit configured to perform loading rate identification on the images in the image sequence to obtain an observed loading rate sequence; The determination unit is configured to determine the loading rate with the highest confidence according to the change of the loading rate in the loading rate sequence.
9. An electronic device, comprising: one or more processors; a storage device having one or more computer programs stored thereon, When the one or more computer programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
10. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
11. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Monocular image depth estimation loading rate identification method and device
CN111583324A
Loading and unloading rate determination method, system and device and storage medium
CN111597863A
Cited By
Loading state detection method, device and equipment and computer program product
CN119359193A
A method, device, equipment and computer program product for detecting a loading state
CN119359193B
Cargo packaging method and device
CN121788001A
