Cargo alarm method, device, equipment, computer readable medium and program product

Abstract

The embodiment of the invention discloses a cargo alarm method, device and equipment, a computer readable medium and a program product. A specific embodiment of the method comprises the steps of obtaining a real-time cargo image and a target cargo image; respectively inputting the real-time cargo image and the target cargo image into a cargo information generation model to generate first cargo segmentation information, a first appearance type, second cargo segmentation information and a second appearance type; determining first cargo category information and second cargo category information; generating real-time cargo change information according to the first cargo segmentation information, the second cargo segmentation information, the first cargo category information and the second cargo category information; and in response to determining that the real-time cargo change information does not conform to the predetermined cargo change information, executing cargo alarm processing for the target cargo processing area. The implementation mode is related to artificial intelligence, real-time cargo changes in the cargo processing area can be accurately and efficiently generated, and accurate monitoring in the cargo processing process is achieved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to cargo alarm methods, devices, equipment, computer-readable media, and program products. Background Art

[0002] At present, there are a wide variety of bulk commodities in digital warehouses, such as steel bars, steel coils, rubber, polyethylene, polypropylene, etc. Different categories of goods have different values. In the actual warehousing and outbound operations, it is easy to have abnormal situations such as wrong entry, errors and mismatches with the goods operation order. For the monitoring of abnormal conditions of goods, the usual method is to monitor the abnormal conditions of goods in or out of the warehouse through manual verification.

[0003] However, the inventors have found that when the above method is used to monitor cargo abnormalities, the following technical problems often occur:

[0004] Manual inspection is an inefficient way to monitor abnormal cargo and wastes a lot of human resources.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose cargo alarm methods, devices, equipment, computer-readable media and program products to solve the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a cargo alarm method, comprising: acquiring a real-time cargo image and a target cargo image for a target cargo processing area, wherein a corresponding shooting time of the target cargo image is a predetermined time length before a corresponding shooting time of the real-time cargo image; inputting the real-time cargo image and the target cargo image into a pre-trained cargo information generation model, respectively, to generate first cargo segmentation information, a first appearance type, a second cargo segmentation information, and a second appearance type; using a category association table for the target cargo processing area, determining first cargo category information corresponding to the first appearance type and second cargo category information corresponding to the second appearance type, wherein the category association table represents a correspondence between appearance types and cargo categories; generating real-time cargo change information for the target cargo processing area according to the first cargo segmentation information, the second cargo segmentation information, the first cargo category information, and the second cargo category information; and in response to determining that the real-time cargo change information does not conform to the predetermined cargo change information, performing cargo alarm processing for the target cargo processing area.

[0009] Optionally, the real-time cargo change information for the target cargo processing area is generated according to the first cargo segmentation information, the second cargo segmentation information, the first cargo category information and the second cargo category information, including: generating a first category mask matrix according to the first cargo segmentation information and the first cargo category information; generating a second category mask matrix according to the second cargo segmentation information and the second cargo category information; generating mask change information between the first category mask matrix and the second category mask matrix; generating the real-time cargo change information under each cargo category according to the mask change information.

[0010] Optionally, the first cargo segmentation information includes: at least one first cargo segmentation outline, the first cargo category information includes: at least one first cargo category corresponding to the at least one first cargo segmentation outline; and the generating of the first category mask matrix according to the first cargo segmentation information and the first cargo category information includes: for each first cargo segmentation outline in the at least one first cargo segmentation outline, according to the first cargo category corresponding to the first cargo segmentation outline, filling the outline content corresponding to the first cargo segmentation outline with the numerical value corresponding to the first cargo category; generating the first category mask matrix according to the at least one filled cargo segmentation outline obtained, wherein the numerical value in the first category mask matrix represents the area corresponding to the cargo category.

[0011] Optionally, the second cargo segmentation information includes: at least one second cargo segmentation outline, the second cargo category information includes: at least one second cargo category corresponding to the at least one second cargo segmentation outline; and the second cargo category information is used to generate a second category mask matrix, including: for each second cargo segmentation outline in the at least one second cargo segmentation outline, according to the second cargo category corresponding to the second cargo segmentation outline, filling the outline content corresponding to the second cargo segmentation outline with the numerical value corresponding to the second cargo category; generating a second category mask matrix based on the at least one filled cargo segmentation outline obtained, wherein the numerical value in the second category mask matrix represents the area corresponding to the cargo category.

[0012] Optionally, the above-mentioned generation of mask change information between the above-mentioned first category mask matrix and the above-mentioned second category mask matrix includes: performing corresponding XOR processing on the elements in the above-mentioned first category mask matrix and the elements in the above-mentioned second category mask matrix to generate an XOR matrix; and determining at least one numerical region in the above-mentioned XOR matrix as the above-mentioned mask change information.

[0013] Optionally, the above-mentioned real-time cargo change information under each cargo category is generated according to the above-mentioned mask change information, including: for each numerical area in the above-mentioned at least one numerical area, a first generation step is executed: in response to determining that the above-mentioned numerical area belongs to the above-mentioned first category mask matrix, the cargo category corresponding to the target numerical value is determined as the first target cargo category, wherein the above-mentioned target numerical value is a numerical value within the above-mentioned numerical area; cargo change information representing an increase in cargo under the above-mentioned first target cargo category is generated; in response to determining that the above-mentioned numerical area belongs to the above-mentioned second category mask matrix, the cargo category corresponding to the target numerical value is determined as the second target cargo category; cargo change information representing a decrease in cargo under the above-mentioned second target cargo category is generated; and real-time cargo change information is generated according to at least one cargo change obtained.

[0014] Optionally, the step of inputting the real-time cargo image and the target cargo image into a pre-trained cargo information generation model to generate first cargo segmentation information, a first appearance type, second cargo segmentation information, and a second appearance type comprises: inputting the real-time cargo image into a feature extraction layer included in the cargo information generation model to generate image feature information; inputting the image feature information into a feature dimension change layer included in the cargo information generation model to generate changed feature information; inputting the changed feature information into a multi-scale head layer included in the cargo information generation model to generate output feature information at each scale; performing a second generation step for each scale at each scale: inputting the output feature information corresponding to the scale into an attention layer included in the cargo information generation model to generate attention feature information; inputting the attention information into a segmentation layer and an appearance type generation layer included in the cargo information generation model to generate first initial cargo segmentation information and a first initial appearance type; and generating first cargo segmentation information and a first appearance type according to each first initial cargo segmentation information and each first initial appearance type obtained.

[0015] In a second aspect, some embodiments of the present disclosure provide a cargo alarm device, including: an acquisition unit, configured to acquire a real-time cargo image and a target cargo image for a target cargo processing area, wherein the corresponding shooting time of the target cargo image is a predetermined time length before the corresponding shooting time of the real-time cargo image; a first generation unit, configured to input the real-time cargo image and the target cargo image into a pre-trained cargo information generation model, respectively, to generate first cargo segmentation information, a first appearance type, a second cargo segmentation information, and a second appearance type; a determination unit, configured to determine the first cargo category information corresponding to the first appearance type and the second cargo category information corresponding to the second appearance type using a category association table for the target cargo processing area, wherein the category association table represents the correspondence between the appearance type and the cargo category; a second generation unit, configured to generate real-time cargo change information for the target cargo processing area according to the first cargo segmentation information, the second cargo segmentation information, the first cargo category information, and the second cargo category information; and an execution unit, configured to execute cargo alarm processing for the target cargo processing area in response to determining that the real-time cargo change information does not conform to the predetermined cargo change information.

[0016] Optionally, the second generating unit may be configured to: generate a first category mask matrix according to the first cargo segmentation information and the first cargo category information; generate a second category mask matrix according to the second cargo segmentation information and the second cargo category information; generate mask change information between the first category mask matrix and the second category mask matrix; and generate the real-time cargo change information under each cargo category according to the mask change information.

[0017] Optionally, the first cargo segmentation information includes: at least one first cargo segmentation outline, and the first cargo category information includes: at least one first cargo category corresponding to the at least one first cargo segmentation outline; and the second generation unit can be configured to: for each first cargo segmentation outline in the at least one first cargo segmentation outline, according to the first cargo category corresponding to the first cargo segmentation outline, fill the outline content corresponding to the first cargo segmentation outline with the numerical value corresponding to the first cargo category; based on the obtained at least one filled cargo segmentation outline, generate a first category mask matrix, wherein the numerical value in the first category mask matrix represents the area corresponding to the cargo category.

[0018] Optionally, the second cargo segmentation information includes: at least one second cargo segmentation outline, and the second cargo category information includes: at least one second cargo category corresponding to the at least one second cargo segmentation outline; and the second generation unit can be configured to: for each second cargo segmentation outline in the at least one second cargo segmentation outline, according to the second cargo category corresponding to the second cargo segmentation outline, fill the outline content corresponding to the second cargo segmentation outline with the numerical value corresponding to the second cargo category; based on the obtained at least one filled cargo segmentation outline, generate a second category mask matrix, wherein the numerical value in the second category mask matrix represents the corresponding area of ​​the cargo category.

[0019] Optionally, the second generation unit can be configured to: perform corresponding XOR processing on the elements in the above-mentioned first category mask matrix and the elements in the above-mentioned second category mask matrix to generate an XOR matrix; and determine at least one numerical region in the above-mentioned XOR matrix as the above-mentioned mask change information.

[0020] Optionally, the second generation unit can be configured to: for each numerical area of ​​the at least one numerical area, perform the first generation step: in response to determining that the numerical area belongs to the first category mask matrix, determine the cargo category corresponding to the target value as the first target cargo category, wherein the target value is a value within the numerical area; generate cargo change information representing an increase in cargo under the first target cargo category; in response to determining that the numerical area belongs to the second category mask matrix, determine the cargo category corresponding to the target value as the second target cargo category; generate cargo change information representing a decrease in cargo under the second target cargo category; and generate real-time cargo change information based on at least one cargo change obtained.

[0021] Optionally, the first generating unit may be configured to: input the real-time cargo image into a feature extraction layer included in the cargo information generating model to generate image feature information; input the image feature information into a feature dimension change layer included in the cargo information generating model to generate changed feature information; input the changed feature information into a multi-scale head layer included in the cargo information generating model to generate output feature information at each scale; for each scale at each scale, perform a second generating step: input the output feature information corresponding to the scale into an attention layer included in the cargo information generating model to generate attention feature information; input the attention information into a segmentation layer and an appearance type generating layer included in the cargo information generating model to generate first initial cargo segmentation information and a first initial appearance type; and generate first cargo segmentation information and a first appearance type according to each first initial cargo segmentation information and each first initial appearance type obtained.

[0022] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0023] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0024] In a fifth aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which implements the method described in any implementation manner in the above-mentioned first aspect when executed by a processor.

[0025] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the cargo alarm method of some embodiments of the present disclosure, real-time cargo changes in the cargo processing area can be accurately and efficiently generated to achieve accurate monitoring during the cargo processing process. Specifically, the reason for the inaccuracy of the related cargo processing process monitoring is that the manual verification method is inefficient in monitoring cargo abnormalities and wastes a lot of human resources. Based on this, the cargo alarm method of some embodiments of the present disclosure first obtains a real-time cargo image and a target cargo image for the target cargo processing area. Among them, the corresponding shooting time of the above-mentioned target cargo image is a predetermined time before the corresponding shooting time of the above-mentioned real-time cargo image. Here, the real-time cargo image and the target cargo image separated by a predetermined time are used to facilitate the subsequent determination of the changes between the cargoes. Then, the above-mentioned real-time cargo image and the above-mentioned target cargo image are respectively input into the pre-trained cargo information generation model to accurately generate the first cargo segmentation information, the first appearance type, the second cargo segmentation information and the second appearance type. Here, for different types of cargo, by determining the corresponding appearance type and cargo segmentation information, it is easier to determine the cargo under the corresponding type, and it is more convenient to determine the changes of the cargo later. Next, the first cargo category information corresponding to the first appearance type and the second cargo category information corresponding to the second appearance type are determined by using the category association table for the target cargo processing area, wherein the category association table represents the corresponding relationship between the appearance type and the cargo category. Here, through the pre-maintained category association table, the effective conversion between the appearance type and the cargo category can be realized to determine the cargo information under each category. Furthermore, in the case of cargo comparison between the first cargo category information and the second cargo category information, real-time cargo change information for the target cargo processing area can be accurately generated. Finally, in response to determining that the real-time cargo change information does not conform to the predetermined cargo change information, the cargo alarm processing for the target cargo processing area is performed to realize the timely alarm of cargo abnormality. In summary, through the cargo information generation model, the appearance type and cargo classification information before and after the cargo change can be accurately generated. Through the category conversion between the category association tables, the before and after changes between the cargo category information corresponding to the appearance type before and after the cargo change can be accurately determined. Based on this, real-time cargo change information can be accurately generated to realize timely alarm processing during cargo processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0027] Figure 1is a schematic diagram of an application scenario of a cargo alarm method according to some embodiments of the present disclosure;

[0028] Figure 2 is a flow chart of some embodiments of the cargo alarm method according to the present disclosure;

[0029] Figure 3 is a schematic diagram showing a first appearance type and a second appearance type in some embodiments of the cargo alarm method according to the present disclosure;

[0030] Figure 4 is a structural diagram showing a cargo information generation model in some embodiments of the cargo alarm method disclosed herein;

[0031] Figure 5 is a schematic diagram showing an associated information setting page in some embodiments of the cargo alarm method according to the present disclosure;

[0032] Figure 6 is a flow chart of other embodiments of the cargo alarm method according to the present disclosure;

[0033] Figure 7 is a schematic diagram of the structure of some embodiments of the cargo alarm device according to the present disclosure;

[0034] Figure 8 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0035] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0036] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0037] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0038] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0039] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0040] With regard to the collection, storage, and use of user personal information (such as cargo change information) involved in this disclosure, before performing the corresponding operations, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, fulfilling the obligation to inform the personal information subject, and obtaining the authorization and consent of the personal information subject in advance.

[0041] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0042] Figure 1 It is a schematic diagram of an application scenario of the cargo alarm method according to some embodiments of the present disclosure.

[0043] exist Figure 1In the application scenario, first, the electronic device 101 can obtain a real-time cargo image 103 and a target cargo image 102 for the target cargo handling area. The corresponding shooting time of the above-mentioned target cargo image 102 is a predetermined time before the corresponding shooting time of the above-mentioned real-time cargo image 103. In this application scenario, the target cargo handling area can be the first area in warehouse A. Then, the electronic device 101 can input the above-mentioned real-time cargo image 103 and the above-mentioned target cargo image 102 into the pre-trained cargo information generation model 104 respectively to generate the first cargo segmentation information 107, the first appearance type 105, the second cargo segmentation information 108 and the second appearance type 106. In this application scenario, the first cargo segmentation information 107 can be segmentation information A. The first appearance type 105 can be a white bag. The second cargo segmentation information 108 can be segmentation information B. The second appearance type 106 can be a white bag. Next, the electronic device 101 may use the category association table 109 for the target cargo handling area to determine the first cargo category information 110 corresponding to the first appearance type 105 and the second cargo category information 111 corresponding to the second appearance type 106, wherein the category association table 109 represents the correspondence between the appearance type and the cargo category. In this application scenario, the first cargo category information 110 may be cargo category 1. The second cargo category information 111 may be cargo category 1. Furthermore, the electronic device 101 may generate the real-time cargo change information 112 for the target cargo handling area according to the first cargo segmentation information 107, the second cargo segmentation information 108, the first cargo category information 110, and the second cargo category information 111. In this application scenario, the real-time cargo change information 112 may be "the cargo type that has undergone cargo change is different". Finally, in response to determining that the real-time cargo change information 112 does not conform to the predetermined cargo change information, the electronic device 101 may perform cargo alarm processing for the target cargo handling area.

[0044] It should be noted that the electronic device 101 can be hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the electronic device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0045] It should be understood that Figure 1 The number of electronic devices in the embodiment is only for illustration. Any number of electronic devices may be provided according to implementation requirements.

[0046] Continue to refer Figure 2, shows a process 200 of some embodiments of the cargo alarm method according to the present disclosure. The cargo alarm method comprises the following steps:

[0047] Step 201, obtaining a real-time cargo image and a target cargo image for a target cargo processing area.

[0048] In some embodiments, the execution subject of the cargo alarm method (for example Figure 1 The electronic device 101 shown can obtain the real-time cargo image and the target cargo image for the target cargo handling area through a wired connection or a wireless connection. The target cargo handling area can be an area to be monitored for the cargo handling process. In practice, the target cargo handling area can be an area in a cargo warehouse to be handled, or an area on a platform for cargo handling. Here, cargo handling can be one of the following: cargo inbound handling, cargo outbound handling. The target cargo handling area can store at least one type of cargo. In addition, the target cargo handling area supports the storage of at least one cargo of different appearance types that are the same cargo. The appearance type can be the appearance packaging type of the cargo. For example, the appearance type can be but is not limited to at least one of the following: white plastic appearance, black plastic appearance, blue iron frame appearance, yellow bag packaging. The real-time cargo image can be an image of the cargo taken in the target cargo handling area at the current time. The target cargo image can be an image taken in the target cargo handling area at a historical time. The corresponding shooting time of the above-mentioned target cargo image is a predetermined time length before the corresponding shooting time of the above-mentioned real-time cargo image. For example, the predetermined time length can be 5 minutes. In an actual scenario, the real-time cargo image and the target cargo image may be cargo images captured by the same camera device in the target cargo processing area with a time difference between the previous and the next captures.

[0049] Step 202 : Input the real-time cargo image and the target cargo image into a pre-trained cargo information generation model respectively to generate first cargo segmentation information, a first appearance type, second cargo segmentation information and a second appearance type.

[0050] In some embodiments, the execution subject may input the real-time cargo image and the target cargo image into a pre-trained cargo information generation model, respectively, to generate first cargo segmentation information, a first appearance type, a second cargo segmentation information, and a second appearance type. The first cargo segmentation information and the first appearance type are model output results corresponding to the real-time cargo image. The second cargo segmentation information and the second appearance type are model output results corresponding to the real-time cargo image. The first cargo segmentation information may be a segmentation result after cargo segmentation of the real-time cargo image. The second cargo segmentation information may be a segmentation result after cargo segmentation of the target cargo image. The first appearance type may be the actual appearance type corresponding to each cargo in the real-time cargo image. The cargo information generation model may be a neural network model that generates cargo segmentation information and the appearance type of the cargo in the image. In practice, the appearance type generation model may be a YOLO v8 model. The appearance type generation model may be trained by a back-propagation model training method. The first appearance type may characterize the appearance type corresponding to each cargo area in the real-time cargo image. The second appearance type may characterize the appearance type corresponding to each cargo area in the target cargo image.

[0051] It should be noted that during the training process, the loss function corresponding to the cargo information generation model is the focal loss function.

[0052] See also Figure 3 , showing a schematic diagram of a first appearance type and a second appearance type.

[0053] Type 301 in the figure shows the appearance type corresponding to each cargo area in the real-time cargo image.

[0054] Type 302 in the figure shows the appearance type corresponding to each cargo area in the target cargo image.

[0055] In some optional implementations of some embodiments, the execution entity may input the real-time cargo image and the target cargo image into a pre-trained cargo information generation model, respectively, to generate first cargo segmentation information, a first appearance type, second cargo segmentation information, and a second appearance type, including the following steps:

[0056] In the first step, the real-time cargo image is input into the feature extraction layer included in the cargo information generation model to generate image feature information. The feature extraction layer may be a network layer of feature semantic content in the image. In practice, the feature extraction layer may be the backbone network layer in the yolov8-seg model. The image feature information may represent the image feature information in the real-time cargo image.

[0057] The second step is to input the above image feature information into the feature dimension change layer included in the above cargo information generation model to generate the changed feature information. The feature dimension change layer can be a network layer that changes the feature dimension of the input feature information and performs feature fusion on the feature map output by the backbone network. In practice, the feature extraction layer can be the middle (Neck) network layer in the yolov8-seg model.

[0058] In the third step, the changed feature information is input into the multi-scale head layer included in the cargo information generation model to generate output feature information at each scale. The multi-scale head layer may be a head layer with multi-scale changes. Each scale may be preset.

[0059] Here, by setting a multi-scale changing head layer, features of different scales can be fused to achieve full feature learning and produce better segmentation effects.

[0060] Step 4: For each of the above scales, perform the second generation step:

[0061] Sub-step 1: Input the output feature information corresponding to the above scale into the attention layer included in the above cargo information generation model to generate attention feature information. The attention layer can be a network layer based on the attention mechanism to ensure that each decoupling head corresponding to the head layer has different feature emphases. In practice, the feature emphases can be but are not limited to one of the following: classification branch emphases, box regression branch emphases, and segmentation branch emphases.

[0062] Sub-step 2, inputting the attention information into the segmentation layer and the appearance type generation layer included in the cargo information generation model to generate the first initial cargo segmentation information and the first initial appearance type. The segmentation layer may be a network layer for cargo segmentation. The appearance type generation layer may be a network layer for generating appearance types. In practice, the segmentation layer and the appearance type generation layer may be multiple layers of convolutional layers connected in series.

[0063] In the fifth step, first goods segmentation information and first appearance types are generated according to the obtained first initial goods segmentation information and first initial appearance types.

[0064] As an example, the execution subject may perform a union of each first initial product segmentation information to generate the first product segmentation information. The execution subject may select the initial appearance type with the highest frequency of occurrence from each first initial appearance type as the first appearance type.

[0065] like Figure 4As shown, the cargo information generation model includes: feature extraction layer, feature dimension change layer, multi-scale head layer, attention layer, segmentation layer and appearance type generation layer.

[0066] It should be noted that during the training of the cargo information generation model, a box regression layer will be added after the attention layer to decouple the multi-scale head layer and obtain outputs of more types to improve the accuracy of the cargo information generation model.

[0067] In addition, the training sample set during training can be annotated using the labelme tool.

[0068] Step 203: using the category association table for the target cargo processing area, determine the first cargo category information corresponding to the first appearance type and the second cargo category information corresponding to the second appearance type.

[0069] In some embodiments, the execution entity may determine the first cargo category information corresponding to the first appearance type and the second cargo category information corresponding to the second appearance type using the category association table for the target cargo handling area. The category association table represents the correspondence between the appearance type and the cargo category. The category association represents an association table preset for the target cargo handling area. That is, the category association table is applicable to the target cargo handling area. The first cargo category information may represent the cargo category corresponding to each cargo area in the real-time cargo image. The second cargo category information may represent the cargo category corresponding to each cargo area in the target cargo image.

[0070] For example, part of the category association table can be as follows:

[0071] [“white square bag” <-> “rubber”, “yellow square bag” <-> “rubber”, “white round bag” <-> “steel coil”, “black round bag” <-> “steel coil”].

[0072] In practice, each category association information in the category association table (ie, each piece of information in the category association table) is set through the association information setting page in the target goods platform.

[0073] For details, please refer to Figure 5 , showing a schematic diagram of the association information setting page.

[0074] Step 204: Generate real-time cargo change information for the target cargo processing area according to the first cargo segmentation information, the second cargo segmentation information, the first cargo category information and the second cargo category information.

[0075] In some embodiments, the execution entity may generate real-time cargo change information for the target cargo processing area based on the first cargo segmentation information, the second cargo segmentation information, the first cargo category information, and the second cargo category information. The real-time cargo change information may be cargo changes that occur to each cargo in the target cargo processing area within a predetermined period of time. For example, the real-time change information may be cargo handling position change information for each cargo in the area. For cargo handling scenarios, the cargo handling direction may be the position of cargo handling in the cargo area (i.e., the position of cargo handling in the cargo pile). For cargo moving-in scenarios, the cargo handling direction may be the position of cargo moving-in in the cargo area (i.e., the position of cargo moving-in in the cargo pile).

[0076] As an example, the execution entity may generate a comparison result as real-time cargo change information by comparing cargo segmentation contents between the first cargo segmentation information and the second cargo segmentation information and by comparing categories between the first cargo category information and the second cargo category information.

[0077] In some optional implementations of some embodiments, the scheduled goods change information includes at least one of the following: scheduled goods category change information, scheduled goods category change information. The scheduled goods category change information may be pre-set information representing the scheduled change corresponding to each goods category. The scheduled goods category change information may be pre-set information representing the scheduled change corresponding to each goods quantity.

[0078] Optionally, in response to determining that the real-time cargo category change information does not conform to the predetermined cargo category change information and / or the real-time cargo quantity change information does not conform to the predetermined cargo quantity change information, the execution entity may execute cargo alarm processing for the target cargo processing area.

[0079] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the cargo alarm method of some embodiments of the present disclosure, real-time cargo changes in the cargo processing area can be accurately and efficiently generated to achieve accurate monitoring during the cargo processing process. Specifically, the reason for the inaccuracy of the related cargo processing process monitoring is that the manual verification method is inefficient in monitoring cargo abnormalities and wastes a lot of human resources. Based on this, the cargo alarm method of some embodiments of the present disclosure first obtains a real-time cargo image and a target cargo image for the target cargo processing area. Among them, the corresponding shooting time of the above-mentioned target cargo image is a predetermined time before the corresponding shooting time of the above-mentioned real-time cargo image. Here, the real-time cargo image and the target cargo image separated by a predetermined time are used to facilitate the subsequent determination of the changes between the cargoes. Then, the above-mentioned real-time cargo image and the above-mentioned target cargo image are respectively input into the pre-trained cargo information generation model to accurately generate the first cargo segmentation information, the first appearance type, the second cargo segmentation information and the second appearance type. Here, for different types of cargo, by determining the corresponding appearance type and cargo segmentation information, it is easier to determine the cargo under the corresponding type, and it is more convenient to determine the changes of the cargo later. Next, the first cargo category information corresponding to the first appearance type and the second cargo category information corresponding to the second appearance type are determined by using the category association table for the target cargo processing area, wherein the category association table represents the corresponding relationship between the appearance type and the cargo category. Here, through the pre-maintained category association table, the effective conversion between the appearance type and the cargo category can be realized to determine the cargo information under each category. Furthermore, in the case of cargo comparison between the first cargo category information and the second cargo category information, real-time cargo change information for the target cargo processing area can be accurately generated. Finally, in response to determining that the real-time cargo change information does not conform to the predetermined cargo change information, the cargo alarm processing for the target cargo processing area is performed to realize the timely alarm of cargo abnormality. In summary, through the cargo information generation model, the appearance type and cargo classification information before and after the cargo change can be accurately generated. Through the category conversion between the category association tables, the before and after changes between the cargo category information corresponding to the appearance type before and after the cargo change can be accurately determined. Based on this, real-time cargo change information can be accurately generated to realize timely alarm processing during cargo processing.

[0080] Further references Figure 6 , shows a process 600 of another embodiment of the cargo alarm method according to the present disclosure. The cargo alarm method comprises the following steps:

[0081] Step 601, obtaining a real-time cargo image and a target cargo image for a target cargo processing area.

[0082] Step 602: Input the real-time cargo image and the target cargo image into a pre-trained cargo information generation model respectively to generate first cargo segmentation information, a first appearance type, second cargo segmentation information and a second appearance type.

[0083] Step 603: using the category association table for the target cargo processing area, determine the first cargo category information corresponding to the first appearance type and the second cargo category information corresponding to the second appearance type.

[0084] Step 604: Generate a first category mask matrix according to the first cargo segmentation information and the first cargo category information.

[0085] In some embodiments, an execution entity (e.g. Figure 1 The electronic device 101 shown can generate a first category mask matrix according to the first cargo segmentation information and the first cargo category information. The first category mask matrix can be a mask matrix in matrix form that represents the cargo location area and cargo category in the real-time cargo image.

[0086] As an example, first, determine the category identifier corresponding to the first cargo category information. Then, determine the segmentation bounding box corresponding to the first cargo segmentation information. Next, determine the bounding box area corresponding to the segmentation bounding box in the real-time cargo image, wherein the bounding box area may be a ring area. Then, set the pixel value of the bounding box area in the real-time cargo image as the category identifier to obtain a set image. Finally, convert the set image into a matrix form to generate a first category mask matrix.

[0087] In some optional implementations of some embodiments, the first cargo segmentation information includes: at least one first cargo segmentation outline, and the first cargo category information includes: at least one first cargo category corresponding to the at least one first cargo segmentation outline. The first cargo segmentation outline may be the outline information of the corresponding cargo segmentation in the real-time cargo image. Each first cargo segmentation outline in the at least one first cargo segmentation outline has an independently placed cargo pile. Similarly, there is a one-to-one correspondence between the first cargo segmentation outline in the at least one first cargo segmentation outline and the first cargo category in the at least one first cargo category. That is, each cargo pile has a corresponding cargo category.

[0088] Optionally, the execution entity may generate a first category mask matrix according to the first goods segmentation information and the first goods category information, comprising the following steps:

[0089] In the first step, for each of the at least one first cargo segmentation profiles, according to the first cargo category corresponding to the first cargo segmentation profile, fill the profile content corresponding to the first cargo segmentation profile with the value corresponding to the first cargo category. Each cargo category has a corresponding value, and the value here can represent the category identity information. Similarly, the value can be a category identifier.

[0090] The second step is to generate a first category mask matrix based on the obtained at least one filled cargo segmentation contour. The values ​​in the first category mask matrix represent the corresponding areas of cargo categories. The values ​​of the non-cargo areas in the first category mask matrix are 0.

[0091] As an example, the execution subject may set a corresponding blank matrix according to the image matrix corresponding to the real-time cargo image. Then, according to the contour content value and contour position corresponding to at least one filled cargo segmentation contour, fill the blank matrix with corresponding values ​​to generate the first category mask matrix. It should be noted that the value of the non-cargo area in the first category mask matrix is ​​0.

[0092] Step 605: Generate a second category mask matrix according to the second cargo segmentation information and the second cargo category information.

[0093] In some embodiments, the execution subject may generate a second category mask matrix according to the second cargo segmentation information and the second cargo category information. The second category mask matrix may be a mask matrix in matrix form that characterizes cargo location areas and cargo categories in the target cargo image. For specific implementation methods, refer to the generation of the first category mask matrix.

[0094] In some optional implementations of some embodiments, the second cargo segmentation information includes: at least one second cargo segmentation profile, and the second cargo category information includes: at least one second cargo category corresponding to the at least one second cargo segmentation profile. Specific explanations are omitted.

[0095] Optionally, the execution entity may generate a second category mask matrix according to the second goods segmentation information and the second goods category information, comprising the following steps:

[0096] In the first step, for each second cargo segmentation profile in the at least one second cargo segmentation profile, according to the second cargo category corresponding to the second cargo segmentation profile, the profile content corresponding to the second cargo segmentation profile is filled with the value corresponding to the second cargo category. The specific implementation method will not be repeated here.

[0097] The second step is to generate a second category mask matrix based on the obtained at least one filled cargo segmentation contour. The values ​​in the second category mask matrix represent the corresponding areas of cargo categories. The specific implementation method will not be described in detail.

[0098] Step 606: Generate mask change information between the first category mask matrix and the second category mask matrix.

[0099] In some embodiments, the execution subject may generate mask change information between the first category mask matrix and the second category mask matrix, wherein the mask change information may represent element changes between the first category mask matrix and the second category mask matrix.

[0100] As an example, the execution entity may determine the numerical difference region information between the first category mask matrix and the second category mask matrix, and obtain at least one numerical difference region information as the mask change information.

[0101] In some optional implementations of some embodiments, the execution subject may generate mask change information between the first category mask matrix and the second category mask matrix, including the following steps:

[0102] In the first step, the elements in the first category mask matrix and the elements in the second category mask matrix are subjected to corresponding XOR processing to generate an XOR matrix. Here, the same elements in the first category mask matrix and the second category mask matrix are set to 0, and the different elements are set to the corresponding values ​​of the non-0 area in the mask matrix.

[0103] The second step is to determine at least one value region in the XOR matrix as the mask change information. In practice, the value region may be a non-zero region.

[0104] Step 607: Generate the above-mentioned real-time cargo change information under each cargo category according to the above-mentioned mask change information.

[0105] In some embodiments, the execution entity may generate the real-time cargo change information for each cargo category based on the mask change information.

[0106] As an example, first, the mask area corresponding to the mask change information is intercepted. Then, the pixel sum of the real-time cargo image in the mask area is determined to obtain the first pixel sum. Next, the pixel integration of the target cargo image in the mask area is determined to obtain the second pixel sum. According to the numerical difference between the first pixel sum and the second pixel sum, the corresponding cargo change amount is matched as the real-time cargo change information.

[0107] In some optional implementations of some embodiments, the execution entity may generate the real-time cargo change information under each cargo category according to the mask change information, including the following steps:

[0108] In the first step, for each value region in the at least one value region, a first generation step is performed:

[0109] Sub-step 1, in response to determining that the numerical region belongs to the first category mask matrix, determining the category of goods corresponding to the target value as the first target category of goods, wherein the target value is a value within the numerical region. The numerical region belonging to the first category mask matrix may be that the sub-region in the first category mask matrix at the corresponding position of the numerical region is consistent with the numerical region.

[0110] Sub-step 2: generating cargo change information indicating an increase in cargo under the first target cargo category.

[0111] Sub-step 3, in response to determining that the numerical region belongs to the second category mask matrix, determining the category of goods corresponding to the target numerical value as the second target category of goods. The numerical region belonging to the second category mask matrix may be that the sub-region in the second category mask matrix at the corresponding position of the numerical region is consistent with the numerical region.

[0112] Sub-step 4: generating cargo change information indicating a decrease in cargo under the second target cargo category.

[0113] The second step is to generate real-time cargo change information based on the at least one cargo change information obtained.

[0114] As an example, the execution entity may directly determine at least one cargo change information as real-time cargo change information.

[0115] Step 608, in response to determining that the real-time cargo change information does not conform to the predetermined cargo change information, executing cargo alarm processing for the target cargo processing area.

[0116] In some embodiments, the specific implementation of steps 601-603 and 608 and the technical effects thereof can be referred to in Figure 2 The corresponding steps 201-203 and 205 in the embodiment are not described in detail here.

[0117] from Figure 6 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 6In the corresponding process 600 of the cargo alarm method in some embodiments, by generating the first category mask matrix and the second category mask matrix, the cargo situation in the real-time cargo image and the target cargo image can be more explicitly displayed. In addition, it is also convenient to compare the cargo information in the two images, so as to quickly and accurately determine the real-time cargo change information.

[0118] Further references Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a cargo alarm device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the cargo alarm device can be specifically applied to various electronic devices.

[0119] like Figure 7 As shown, a cargo alarm device 700 includes: an acquisition unit 701, a first generation unit 702, a determination unit 703, a second generation unit 704 and an execution unit 705. The acquisition unit 701 is configured to acquire a real-time cargo image and a target cargo image for a target cargo processing area, wherein the corresponding shooting time of the target cargo image is a predetermined time length before the corresponding shooting time of the real-time cargo image; the first generation unit 702 is configured to input the real-time cargo image and the target cargo image into a pre-trained cargo information generation model, respectively, to generate first cargo segmentation information, a first appearance type, a second cargo segmentation information, and a second appearance type; the determination unit 703 is configured to determine the first cargo category information corresponding to the first appearance type and the second cargo category information corresponding to the second appearance type using a category association table for the target cargo processing area, wherein the category association table represents the corresponding relationship between the appearance type and the cargo category; the second generation unit 704 is configured to generate real-time cargo change information for the target cargo processing area according to the first cargo segmentation information, the second cargo segmentation information, the first cargo category information, and the second cargo category information; the execution unit 705 is configured to execute cargo alarm processing for the target cargo processing area in response to determining that the real-time cargo change information does not conform to the predetermined cargo change information.

[0120] In some optional implementations of some embodiments, the second generation unit 704 can be further configured to: generate a first category mask matrix based on the above-mentioned first cargo segmentation information and the above-mentioned first cargo category information; generate a second category mask matrix based on the above-mentioned second cargo segmentation information and the above-mentioned second cargo category information; generate mask change information between the above-mentioned first category mask matrix and the above-mentioned second category mask matrix; generate the above-mentioned real-time cargo change information under each cargo category based on the above-mentioned mask change information.

[0121] In some optional implementations of some embodiments, the first cargo segmentation information includes: at least one first cargo segmentation outline, and the first cargo category information includes: at least one first cargo category corresponding to the at least one first cargo segmentation outline; and the second generation unit 704 can be further configured to: for each first cargo segmentation outline in the at least one first cargo segmentation outline, according to the first cargo category corresponding to the first cargo segmentation outline, fill the outline content corresponding to the first cargo segmentation outline with the numerical value corresponding to the first cargo category; based on the obtained at least one filled cargo segmentation outline, generate a first category mask matrix, wherein the numerical value in the first category mask matrix represents the area corresponding to the cargo category.

[0122] In some optional implementations of some embodiments, the second cargo segmentation information includes: at least one second cargo segmentation outline, and the second cargo category information includes: at least one second cargo category corresponding to the at least one second cargo segmentation outline; and the second generation unit 704 can be further configured to: for each second cargo segmentation outline in the at least one second cargo segmentation outline, according to the second cargo category corresponding to the second cargo segmentation outline, fill the outline content corresponding to the second cargo segmentation outline with the numerical value corresponding to the second cargo category; based on the obtained at least one filled cargo segmentation outline, generate a second category mask matrix, wherein the numerical value in the second category mask matrix represents the area corresponding to the cargo category.

[0123] In some optional implementations of some embodiments, the second generation unit 704 can be further configured to: perform corresponding XOR processing on the elements in the above-mentioned first category mask matrix and the elements in the above-mentioned second category mask matrix to generate an XOR matrix; and determine at least one numerical region in the above-mentioned XOR matrix as the above-mentioned mask change information.

[0124] In some optional implementations of some embodiments, the second generation unit 704 can be further configured to: for each numerical area in the at least one numerical area mentioned above, perform the first generation step: in response to determining that the numerical area belongs to the first category mask matrix mentioned above, determine the cargo category corresponding to the target value as the first target cargo category, wherein the target value is a value within the numerical area; generate cargo change information representing an increase in cargo under the first target cargo category; in response to determining that the numerical area belongs to the second category mask matrix mentioned above, determine the cargo category corresponding to the target value as the second target cargo category; generate cargo change information representing a decrease in cargo under the second target cargo category; and generate real-time cargo change information based on at least one cargo change obtained.

[0125] In some optional implementations of some embodiments, the first generation unit 702 may be further configured to: input the real-time cargo image into the feature extraction layer included in the cargo information generation model to generate image feature information; input the image feature information into the feature dimension change layer included in the cargo information generation model to generate changed feature information; input the changed feature information into the multi-scale head layer included in the cargo information generation model to generate output feature information at each scale; for each scale at each scale, perform a second generation step: input the output feature information corresponding to the scale into the attention layer included in the cargo information generation model to generate attention feature information; input the attention information into the segmentation layer and the appearance type generation layer included in the cargo information generation model to generate first initial cargo segmentation information and first initial appearance type; generate first cargo segmentation information and first appearance type according to each first initial cargo segmentation information and each first initial appearance type obtained.

[0126] It is understood that the units described in the cargo alarm device 700 are similar to those described in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the cargo alarm device 700 and the units included therein, and will not be described in detail here.

[0127] Reference below Figure 8 , which shows an electronic device (eg, Figure 1 Schematic diagram of the structure of the electronic device 101)800. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0128] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory 802 or a program loaded from a storage device 808 into a random access memory 803. Various programs and data required for the operation of the electronic device 800 are also stored in the random access memory 803. The processing device 801, the read-only memory 802, and the random access memory 803 are connected to each other via a bus 804. An input / output interface 805 is also connected to the bus 804.

[0129] Typically, the following devices may be connected to the input / output interface 805: an input device 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 8 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0130] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a read-only memory 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0131] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0132] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0133] The computer-readable medium may be included in the electronic device; or may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a real-time cargo image and a target cargo image for a target cargo processing area, wherein the corresponding shooting time of the target cargo image is a predetermined time before the corresponding shooting time of the real-time cargo image; inputs the real-time cargo image and the target cargo image into a pre-trained cargo information generation model to generate first cargo segmentation information, a first appearance type, a second cargo segmentation information, and a second appearance type; determines the first cargo category information corresponding to the first appearance type and the second cargo category information corresponding to the second appearance type using a category association table for the target cargo processing area, wherein the category association table represents the corresponding relationship between the appearance type and the cargo category; generates real-time cargo change information for the target cargo processing area according to the first cargo segmentation information, the second cargo segmentation information, the first cargo category information, and the second cargo category information; and performs cargo alarm processing for the target cargo processing area in response to determining that the real-time cargo change information does not conform to the predetermined cargo change information.

[0134] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes an acquisition unit, a first generation unit, a determination unit, a second generation unit, and an execution unit. The names of these units do not constitute limitations on the units themselves in some cases, for example, the acquisition unit may also be described as "a unit for acquiring real-time cargo images and target cargo images for a target cargo handling area".

[0137] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0138] Some embodiments of the present disclosure also provide a computer program product, including a computer program, which implements any of the above-mentioned cargo alarm methods when executed by a processor.

[0139] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A cargo alarm method, comprising: Acquire a real-time cargo image and a target cargo image for a target cargo processing area, wherein a corresponding shooting time of the target cargo image is a predetermined time before a corresponding shooting time of the real-time cargo image; Inputting the real-time cargo image and the target cargo image into a pre-trained cargo information generation model respectively to generate first cargo segmentation information, a first appearance type, second cargo segmentation information, and a second appearance type; Determine first cargo category information corresponding to the first appearance type and second cargo category information corresponding to the second appearance type by using a category association table for the target cargo handling area, wherein the category association table represents a correspondence between appearance types and cargo categories; generating real-time cargo change information for the target cargo processing area according to the first cargo segmentation information, the second cargo segmentation information, the first cargo category information, and the second cargo category information; In response to determining that the real-time cargo change information does not conform to predetermined cargo change information, a cargo alarm process for the target cargo processing area is performed.

2. The method according to claim 1, wherein: Generating real-time cargo change information for the target cargo processing area according to the first cargo segmentation information, the second cargo segmentation information, the first cargo category information, and the second cargo category information includes: Generate a first category mask matrix according to the first goods segmentation information and the first goods category information; Generate a second category mask matrix according to the second goods segmentation information and the second goods category information; Generate mask change information between the first category mask matrix and the second category mask matrix; The real-time cargo change information under each cargo category is generated according to the mask change information.

3. The method according to claim 2, wherein: The first cargo segmentation information includes: at least one first cargo segmentation profile, and the first cargo category information includes: at least one first cargo category corresponding to the at least one first cargo segmentation profile; and The step of generating a first category mask matrix according to the first goods segmentation information and the first goods category information includes: For each first cargo segmentation profile of the at least one first cargo segmentation profile, according to the first cargo category corresponding to the first cargo segmentation profile, filling the profile content corresponding to the first cargo segmentation profile with a value corresponding to the first cargo category; A first category mask matrix is ​​generated based on at least one filled cargo segmentation contour obtained, wherein the values ​​in the first category mask matrix represent areas corresponding to cargo categories.

4. The method according to claim 2, wherein: The second cargo segmentation information includes: at least one second cargo segmentation profile, and the second cargo category information includes: at least one second cargo category corresponding to the at least one second cargo segmentation profile; and The step of generating a second category mask matrix according to the second goods segmentation information and the second goods category information includes: For each second cargo segmentation profile of the at least one second cargo segmentation profile, according to the second cargo category corresponding to the second cargo segmentation profile, filling the profile content corresponding to the second cargo segmentation profile with a value corresponding to the second cargo category; A second category mask matrix is ​​generated based on at least one filled cargo segmentation contour obtained, wherein the values ​​in the second category mask matrix represent areas corresponding to cargo categories.

5. The method according to claim 2, wherein: The generating mask change information between the first category mask matrix and the second category mask matrix includes: Performing corresponding XOR processing on the elements in the first category mask matrix and the elements in the second category mask matrix to generate an XOR matrix; At least one value region in the XOR matrix is ​​determined as the mask change information.

6. The method according to claim 5, wherein: Generating the real-time cargo change information under each cargo category according to the mask change information includes: For each of the at least one value region, a first generating step is performed: In response to determining that the value region belongs to the first category mask matrix, determining a category of goods corresponding to a target value as a first target category of goods, wherein the target value is a value within the value region; Generating cargo change information indicating an increase in cargo under the first target cargo category; In response to determining that the value region belongs to the second category mask matrix, determining the category of goods corresponding to the target value as the second target goods category; generating cargo change information indicating that cargo under the second target cargo category has decreased; Real-time cargo change information is generated according to the obtained at least one cargo change information.

7. The method according to claim 1, wherein: The step of inputting the real-time cargo image and the target cargo image into a pre-trained cargo information generation model to generate first cargo segmentation information, a first appearance type, second cargo segmentation information, and a second appearance type comprises: Inputting the real-time cargo image into the feature extraction layer included in the cargo information generation model to generate image feature information; Inputting the image feature information into the feature dimension change layer included in the cargo information generation model to generate changed feature information; Inputting the changed feature information into the multi-scale head layer included in the cargo information generation model to generate output feature information at each scale; For each of the scales, perform the second generation step: Inputting the output feature information corresponding to the scale into the attention layer included in the cargo information generation model to generate attention feature information; Inputting the attention information into the segmentation layer and the appearance type generation layer included in the cargo information generation model to generate first initial cargo segmentation information and a first initial appearance type; First cargo segmentation information and first appearance type are generated according to each first initial cargo segmentation information and each first initial appearance type obtained.

8. A cargo alarm device, comprising: an acquisition unit configured to acquire a real-time cargo image and a target cargo image for a target cargo processing area, wherein a corresponding shooting time of the target cargo image is a predetermined time length before a corresponding shooting time of the real-time cargo image; a first generating unit configured to input the real-time cargo image and the target cargo image into a pre-trained cargo information generating model, respectively, to generate first cargo segmentation information, a first appearance type, second cargo segmentation information, and a second appearance type; a determining unit configured to determine first cargo category information corresponding to the first appearance type and second cargo category information corresponding to the second appearance type by using a category association table for the target cargo handling area, wherein the category association table represents a correspondence between appearance types and cargo categories; A second generating unit is configured to generate real-time cargo change information for the target cargo processing area according to the first cargo segmentation information, the second cargo segmentation information, the first cargo category information, and the second cargo category information; The execution unit is configured to execute cargo alarm processing for the target cargo processing area in response to determining that the real-time cargo change information does not conform to the predetermined cargo change information.

9. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images