Method, device, electronic device and storage medium for monitoring abnormal cargo

Abstract

The present application proposes a method, device, computer equipment and storage medium for monitoring cargo anomalies, wherein the method includes obtaining video or image data of cargo placement in a first warehouse; generating detection results based on the video or image data and a trained target detection model, wherein the trained target detection model is jointly trained by federated learning based on sample video or image data of cargo placement in the first warehouse and the second warehouse. The method, device, computer equipment and storage medium for monitoring cargo anomalies of the present application can obtain sufficient effective sample video or image data to train the target detection model, thereby improving the performance of the target detection model and truly achieving the effect of monitoring cargo anomalies.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, electronic device and storage medium for monitoring abnormal cargo. Background Art

[0002] Warehousing operations involve the unloading, loading, handling, and storage of goods, which are directly related to the safety of goods, personnel, and warehouse facilities. However, warehousing operations are prone to accidents such as cargo collapse and falling, which can cause personal injury and damage to goods. The main reason is the unreasonable placement of goods in the warehouse, such as uneven placement of goods on shelves or random placement of goods. Therefore, how to detect unreasonable placement of goods in a timely manner and realize abnormal monitoring of goods is the most urgent problem to be solved.

[0003] In related technologies, when warehouses of various enterprises implement cargo abnormality monitoring, they often train cargo detection models based on their own accumulated data. However, in actual warehousing scenarios, there are few abnormal cargo placement states, and it is difficult to obtain sufficient effective data to train cargo detection models, resulting in poor performance of cargo detection models and failure to truly achieve the effect of cargo abnormality monitoring. Summary of the invention

[0004] The present application proposes a method, device, electronic device and storage medium for monitoring abnormal cargo.

[0005] The first aspect of the present application proposes a method for monitoring abnormal cargo, including: obtaining video or image data of cargo placement in a first warehouse; generating a detection result based on the video or image data and a trained target detection model, wherein the trained target detection model is jointly trained by federated learning based on sample video or image data of cargo placement in the first warehouse and the second warehouse.

[0006] The method for monitoring abnormal cargo in the embodiment of the present application is to obtain a trained target detection model by jointly training sample video or image data of cargo placed in the first warehouse and the second warehouse through federated learning, obtain video or image data of cargo placed in the first warehouse, and generate detection results based on the video or image data and the trained target detection model. Since the trained target detection model is obtained by jointly training sample video or image data of cargo placed in the first warehouse and the second warehouse, it is possible to obtain sufficient effective sample video or image data to train the target detection model, thereby improving the performance of the target detection model and truly achieving the effect of monitoring abnormal cargo.

[0007] The second aspect of the present application proposes a device for monitoring abnormal goods, including: a first acquisition module, configured to acquire video or image data of the placement of goods in a first warehouse; a first generation module, configured to generate a detection result based on the video or image data and a trained target detection model, wherein the trained target detection model is jointly trained by federated learning based on sample video or image data of the placement of goods in the first warehouse and the second warehouse.

[0008] The device for monitoring abnormal cargo in the embodiment of the present application obtains a trained target detection model by joint training through federated learning based on sample video or image data of cargo placed in the first warehouse and the second warehouse, obtains video or image data of cargo placed in the first warehouse, and generates detection results based on the video or image data and the trained target detection model. Since the trained target detection model is obtained by joint training based on sample video or image data of cargo placed in the first warehouse and the second warehouse, sufficient effective sample video or image data can be obtained to train the target detection model, thereby improving the performance of the target detection model and truly achieving the effect of monitoring abnormal cargo.

[0009] The third aspect embodiment of the present application proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for abnormal cargo monitoring as described in the first aspect embodiment above.

[0010] The fourth aspect of the present application provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method for monitoring abnormal cargo as described in the first aspect of the present application.

[0011] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0013] Figure 1 A flowchart of a method for monitoring abnormal cargo provided in one embodiment of the present application;

[0014] Figure 2 A flowchart of a method for monitoring abnormal cargo provided by another embodiment of the present application;

[0015] Figure 3 A flowchart of a method for monitoring abnormal cargo provided by another embodiment of the present application;

[0016] Figure 4 A logical diagram of a method for monitoring abnormal cargo provided in an embodiment of the present application;

[0017] Figure 5 A logical diagram of offline training of a target detection model provided in one embodiment of the present application;

[0018] Figure 6 A schematic diagram of the structure of a device for monitoring abnormal cargo provided in one embodiment of the present application;

[0019] Figure 7 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0020] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0021] In recent years, with the continuous acceleration of my country's industrial process, the transportation demand for industrial raw materials, semi-finished products and even commodities has steadily increased, and my country's logistics industry has developed rapidly. The total amount of social logistics in the country has continued to increase. According to the survey, the total amount of logistics in the country reached 21.59 trillion yuan in the first three quarters of 2019. In the entire logistics industry, warehousing is of vital importance and is an indispensable link in the circulation of goods and goods in the logistics system.

[0022] Warehousing operations involve the unloading, loading, handling, and storage of goods, which are directly related to the safety of goods, personnel, and warehouse facilities. However, warehousing operations are prone to accidents such as cargo collapse and falling, which can cause personal injury and damage to goods. The main reason is the unreasonable placement of goods in the warehouse, such as uneven placement of goods on shelves or random placement of goods. Therefore, how to detect unreasonable placement of goods in a timely manner and realize abnormal monitoring of goods is the most urgent problem to be solved.

[0023] In the related art, when each enterprise warehouse implements cargo abnormality monitoring, it is often based on the training of cargo detection models based on their own accumulated data. However, in actual warehousing scenarios, there are fewer abnormal cargo placement states, and it is difficult to obtain sufficient effective data to train the cargo detection model, resulting in poor performance of the cargo detection model, which cannot truly achieve the effect of cargo abnormality monitoring. To this end, the present application proposes a method, device, electronic device and storage medium for cargo abnormality monitoring, which can obtain sufficient effective sample video or image data to train the target detection model, thereby improving the performance of the target detection model and truly achieving the effect of cargo abnormality monitoring.

[0024] The following describes the method, device, electronic device and storage medium for monitoring abnormal cargo in embodiments of the present application with reference to the accompanying drawings.

[0025] Figure 1 The flowchart of the method for monitoring abnormal cargo provided in one embodiment of the present application is shown in FIG. The method for monitoring abnormal cargo provided in the embodiment of the present application can be executed by the device for monitoring abnormal cargo provided in the embodiment of the present application. Figure 1 As shown, the method for monitoring abnormal cargo in the embodiment of the present application may specifically include the following steps:

[0026] S101, obtaining video or image data of goods placed in a first warehouse.

[0027] Specifically, the first warehouse is a warehouse to be monitored for abnormal cargo. When monitoring the abnormal placement of cargo in the first warehouse, video or image data of the cargo placement in the first warehouse is obtained. The video or image data of the cargo placement in the first warehouse can be obtained by an image acquisition device, such as a camera, disposed in the first warehouse.

[0028] S102, generating a detection result based on the video or image data and the trained target detection model, wherein the trained target detection model is jointly trained by federated learning based on sample video or image data of the goods placed in the first warehouse and the second warehouse.

[0029] Specifically, the video or image data acquired in step S101 is input into a pre-trained target detection model, and the pre-trained target detection model outputs a corresponding detection result. The detection result may specifically be the location of the goods and the probability of abnormal placement.

[0030] The trained target detection model is obtained by pre-training, specifically, by joint training through federated learning based on sample video or image data of goods placed in the first warehouse and sample video or image data of goods placed in other warehouses other than the first warehouse, namely the second warehouse. Since the training data used for model training includes not only sample video or image data of goods placed in the first warehouse, but also sample video or image data of goods placed in other warehouses other than the first warehouse, namely the second warehouse, sufficient effective data can be obtained to train the target detection model, thereby improving the performance of the target detection model and truly achieving the effect of abnormal cargo monitoring.

[0031] Federated learning is also known as federated machine learning, joint learning, and alliance learning. Federated learning is a machine learning framework that can effectively help multiple institutions use data and conduct machine learning modeling while meeting the requirements of user privacy protection, data security, and government regulations. In the embodiment of the present application, the trained target detection model is obtained by joint training in the form of federated learning. Therefore, the model can be trained using multi-party data without the original data of each warehouse leaving the warehouse, thereby improving the performance of the target detection model and truly achieving the effect of abnormal cargo monitoring.

[0032] It should be noted here that the specific process of joint training through federated learning can be found in the relevant description in the following embodiment, which will not be repeated here.

[0033] The method for monitoring abnormal cargo in the embodiment of the present application obtains a trained target detection model by joint training based on sample video or image data of cargo placed in the first warehouse and the second warehouse, obtains video or image data of cargo placed in the first warehouse, and generates detection results based on the video or image data and the trained target detection model. Since the trained target detection model is obtained by joint training based on sample video or image data of cargo placed in the first warehouse and the second warehouse, it is possible to obtain sufficient effective sample video or image data to train the target detection model, thereby improving the performance of the target detection model and truly achieving the effect of monitoring abnormal cargo. By using a federated learning method to generate a trained target detection model, it is possible to use multi-party data for model training without the original data of each warehouse leaving the warehouse, thereby improving the performance of the target detection model and truly achieving the effect of monitoring abnormal cargo.

[0034] In practical applications, in order to further improve the performance of the trained target detection model, in one embodiment of the present application, the acquired video or image data may also be processed to obtain training data. Figure 2 To explain, Figure 2 A flowchart of a method for abnormal cargo monitoring provided by another embodiment of the present application. Figure 2As shown, the method for monitoring abnormal cargo in the embodiment of the present application may specifically include the following steps:

[0035] S201, obtaining video or image data of goods placed in a first warehouse.

[0036] Specifically, step S201 is similar to step S101 in the above embodiment, and the specific process will not be repeated here.

[0037] Step S102 in the above embodiment may specifically include the following steps S202 and S203.

[0038] S202, pre-processing the video or image data.

[0039] Specifically, in order to further improve the performance of the trained object detection model, the video or image data obtained in step S201 is preprocessed to obtain preprocessed video or image data. The preprocessing may specifically include but is not limited to data cleaning, feature engineering and other processing.

[0040] Data cleaning is the process of re-examining and verifying data, with the aim of deleting duplicate information, correcting existing errors, and providing data consistency. The specific method can be various existing data cleaning methods, and this application does not limit this.

[0041] Feature engineering is the process of transforming raw data into training data for the model. Its purpose is to obtain better training data features so that the machine learning model approaches this upper limit. Feature engineering generally includes three parts: feature construction, feature extraction, and feature selection. Feature construction refers to manually finding some features with physical meaning from the raw data. Both feature extraction and feature selection are aimed at finding the most effective features from the raw features. However, feature extraction emphasizes obtaining a set of features with obvious physical or statistical meanings through feature conversion, while feature selection is to select a set of feature subsets with obvious physical or statistical meanings from the feature set. Both feature extraction and feature selection can help reduce feature dimensions and data redundancy. Feature extraction can sometimes discover more meaningful feature attributes, and the feature selection process can often indicate the importance of each feature to model construction.

[0042] S203, input the preprocessed video or image data into the trained target detection model to generate a detection result.

[0043] Specifically, the preprocessed video or image data obtained in step S202 is input into the trained target detection model, and the trained target detection model outputs the corresponding detection result. The specific process can be found in the relevant description of step S102 in the above embodiment, which will not be repeated here.

[0044] S204, sending the detection result to the storage center of the first warehouse, so that the storage center can send the abnormal monitoring information to the first warehouse according to the detection result.

[0045] Specifically, the detection result obtained in step S203 is sent to the storage center of the first warehouse, and the storage center of the first warehouse determines whether the goods in the first warehouse are placed abnormally according to the detection result. If abnormal, the abnormal monitoring information is sent to the first warehouse for timely remediation. Among them, judging whether the goods in the first warehouse are placed abnormally according to the detection result can specifically be judging whether the probability of abnormal placement in the detection result exceeds a preset probability threshold, such as 70%. If it exceeds the probability threshold, it is judged that the goods in the first warehouse are placed abnormally.

[0046] The method for monitoring abnormal cargo in the embodiment of the present application obtains a trained target detection model by joint training based on sample video or image data of cargo placed in the first warehouse and the second warehouse, obtains video or image data of cargo placed in the first warehouse, and generates detection results based on the video or image data and the trained target detection model. Since the trained target detection model is obtained by joint training based on sample video or image data of cargo placed in the first warehouse and the second warehouse, it is possible to obtain sufficient effective sample video or image data to train the target detection model, thereby improving the performance of the target detection model and truly achieving the effect of monitoring abnormal cargo. By using a federated learning method to generate a trained target detection model, it is possible to use multi-party data for model training without the original data of each warehouse leaving the warehouse, thereby improving the performance of the target detection model and truly achieving the effect of monitoring abnormal cargo.

[0047] In one embodiment of the present application, Figure 3 The method shown is used to train a trained target detection model. Figure 3 A flowchart of a method for abnormal cargo monitoring provided by another embodiment of the present application. Figure 3 As shown, the training process of the trained target detection model in the above embodiment may specifically include the following steps:

[0048] S301, obtaining sample video or image data of goods placed in a first warehouse and manually marked goods positions and corresponding placement states.

[0049] Specifically, historical video or image data of the placement of goods in the first warehouse is obtained as sample video or image data. The sample video or image data is labeled by manual labeling, wherein the label may specifically include the location of the goods and the corresponding placement status. The placement status includes abnormal placement or normal placement.

[0050] S302, training the target detection model to be trained according to the sample video or image data and the cargo position and the corresponding placement status to obtain a candidate target detection model.

[0051] Specifically, the sample video or image data obtained in step S301 is used as the input data of the model, and the cargo position and corresponding placement state obtained in step S301 are used as the input data of the model, and the target detection model to be trained is trained to obtain a candidate target detection model. The target detection model to be trained can be a target detection model with preset parameters, and the target detection model can be various existing target detection models, which is not limited in this application.

[0052] As a feasible implementation, the above step S302 may specifically include the following steps: preprocessing the sample video or image data; training the target detection model to be trained according to the preprocessed sample video or image data and the cargo position and corresponding placement state. The specific process of the preprocessing can be found in the relevant description in the above embodiment, which will not be repeated here.

[0053] S303: Generate a trained target detection model based on the candidate target detection model.

[0054] Specifically, according to the candidate target detection model obtained in step S302, a trained target detection model can be generated by using a federated learning method.

[0055] As a feasible implementation, the above step S303 may specifically include the following steps: sending the model parameters in the candidate target detection model to the server; receiving the aggregated model parameters sent by the server, where the aggregated model parameters are obtained by the server based on the aggregation of model parameters received from multiple warehouses; updating the candidate target detection model based on the aggregated model parameters to obtain a trained target detection model.

[0056] The method for monitoring abnormal cargo in the embodiment of the present application obtains a trained target detection model by joint training based on sample video or image data of cargo placed in the first warehouse and the second warehouse, obtains video or image data of cargo placed in the first warehouse, and generates detection results based on the video or image data and the trained target detection model. Since the trained target detection model is obtained by joint training based on sample video or image data of cargo placed in the first warehouse and the second warehouse, it is possible to obtain sufficient effective sample video or image data to train the target detection model, thereby improving the performance of the target detection model and truly achieving the effect of monitoring abnormal cargo. By using a federated learning method to generate a trained target detection model, it is possible to use multi-party data for model training without the original data of each warehouse leaving the warehouse, thereby improving the performance of the target detection model and truly achieving the effect of monitoring abnormal cargo.

[0057] To clearly illustrate the method for monitoring abnormal cargo in the embodiment of the present application, Figure 4 Describe it in detail. Figure 4 As shown in the figure, taking the warehouse of enterprise A and the warehouse of enterprise B as examples, the offline platform of enterprise A collects sample video or image data of the warehouse site of enterprise A as the data source, and performs data preprocessing on the data source to obtain effective training data. The offline platform of enterprise B collects sample video or image data of the warehouse site of enterprise B as the data source, and performs data preprocessing on the data source to obtain effective training data. According to the effective training data of enterprises A and B, cross-domain joint modeling, i.e., joint model training, is performed. After the training is completed, enterprises A and B respectively obtain the trained offline models (i.e., the trained target detection models), and push the trained offline models to their corresponding online platforms. The online platforms of enterprises A and B collect real-time video or image data of their respective warehouse sites through video surveillance, and input the real-time video or image data into the online models pushed by each for anomaly detection to obtain the detection results. The detection results are sent to the storage centers of their respective warehouses. The storage centers send the danger warning information (i.e., abnormal monitoring information) to their respective warehouses according to the detection results for timely remediation. Through continuous iterations, real-time video or image data collected by the online platform can be collected into the offline platform data source for data expansion to improve the performance of the trained target detection model.

[0058] To clearly illustrate the method for monitoring abnormal cargo in the embodiment of the present application, Figure 5 The training process of the target detection model is described in detail. Figure 5 As shown, still taking the warehouse of enterprise A and the warehouse of enterprise B as an example, sample video or image data of the warehouses of enterprises A and B are collected, and data preprocessing is performed on the sample video or image data to obtain effective training data. Model training is performed on each of the effective training data of enterprises A and B to obtain a candidate target detection model, and the model parameters in the candidate target detection model are sent to the main service center (i.e., the server). The main service center aggregates the model parameters of the warehouses of enterprises A and B received to obtain aggregated model parameters, and sends the aggregated model parameters to enterprises A and B. Enterprises A and B update the previously trained candidate target detection models according to the aggregated model parameters to obtain trained target detection models, and perform abnormal monitoring of the goods in their respective warehouses according to the trained target detection models.

[0059] In order to implement the above embodiment, the embodiment of the present application also proposes a device for monitoring abnormal cargo. Figure 6 FIG. 1 is a schematic diagram of the structure of a device for monitoring abnormal cargo according to an embodiment of the present application. Figure 6As shown, the device 600 for monitoring abnormal cargo in the embodiment of the present application may specifically include: a first acquisition module 601 and a first generation module 602 .

[0060] The first acquisition module 601 is configured to acquire video or image data of goods placed in the first warehouse.

[0061] The first generation module 602 is configured to generate a detection result based on the video or image data and a trained target detection model, where the trained target detection model is jointly trained by federated learning based on sample video or image data of the goods placed in the first warehouse and the second warehouse.

[0062] In one embodiment of the present application, the first generation module 602 may specifically include: a first preprocessing unit, configured to preprocess the video or image data; a generation unit, configured to input the preprocessed video or image data into a trained target detection model to generate a detection result.

[0063] In one embodiment of the present application, the device 600 for monitoring cargo abnormalities may also include: a second acquisition module, configured to acquire sample video or image data of cargo placement in the first warehouse and manually annotated cargo positions and corresponding placement states; a training module, configured to train a target detection model to be trained based on the sample video or image data and cargo positions and corresponding placement states to obtain a candidate target detection model; and a second generation module, configured to generate a trained target detection model based on the candidate target detection model.

[0064] In one embodiment of the present application, the second generation module may specifically include: a sending unit, configured to send the model parameters in the candidate target detection model to the server; a receiving unit, configured to receive the aggregated model parameters sent by the server, the aggregated model parameters are obtained by the server based on the aggregation of model parameters received from multiple warehouses; an updating unit, configured to update the candidate target detection model based on the aggregated model parameters to obtain a trained target detection model.

[0065] In one embodiment of the present application, the training module may specifically include: a second preprocessing unit, configured to preprocess the sample video or image data; a training unit, configured to train the target detection model to be trained based on the preprocessed sample video or image data and the cargo position and corresponding placement status.

[0066] In one embodiment of the present application, the device 600 for monitoring abnormalities of goods may further include: a sending module, configured to send the detection results to the storage center of the first warehouse, so that the storage center can send abnormality monitoring information to the first warehouse according to the detection results.

[0067] In one embodiment of the present application, preprocessing includes data cleaning and / or feature engineering.

[0068] It should be noted that the above explanation of the embodiment of the method for monitoring abnormal cargo is also applicable to the device for monitoring abnormal cargo in this embodiment, and the specific process will not be repeated here.

[0069] The device for monitoring abnormal cargo in the embodiment of the present application obtains a trained target detection model by joint training based on sample video or image data of cargo placed in the first warehouse and the second warehouse, obtains video or image data of cargo placed in the first warehouse, and generates detection results based on the video or image data and the trained target detection model. Since the trained target detection model is obtained by joint training based on sample video or image data of cargo placed in the first warehouse and the second warehouse, it is possible to obtain sufficient effective sample video or image data to train the target detection model, thereby improving the performance of the target detection model and truly achieving the effect of monitoring abnormal cargo. By using a federated learning method to generate a trained target detection model, it is possible to use multi-party data to train the model without the original data of each warehouse leaving the warehouse, thereby improving the performance of the target detection model and truly achieving the effect of monitoring abnormal cargo.

[0070] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0071] like Figure 7 , is a block diagram of an electronic device according to a method for abnormal monitoring of goods according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as intelligent voice interaction devices, personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0072] like Figure 7As shown, the electronic device includes: one or more processors 701, a memory 702, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor 701 can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 701 is taken as an example.

[0073] The memory 702 is a non-transitory computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to perform the method for monitoring abnormal cargo provided in the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the method for monitoring abnormal cargo provided in the present application.

[0074] The memory 702 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the method for abnormal cargo monitoring in the embodiment of the present application (for example, the attached Figure 6 The processor 701 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 702, that is, the method for abnormal cargo monitoring in the above method embodiment is implemented.

[0075] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device of the method for monitoring abnormal cargo, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely arranged relative to the processor 701, and these remote memories may be connected to the electronic device of the method for monitoring abnormal cargo via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0076] The electronic device of the cargo abnormality monitoring method may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0077] The input device 703 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device of the method for abnormal cargo monitoring, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 704 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0078] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0079] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0080] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0081] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0082] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short).

[0083] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0084] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0085] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for monitoring abnormal cargo, characterized in that: include: Obtain video or image data of goods placed in the first warehouse; Generate a detection result based on the video or image data and the trained target detection model, wherein the target detection model is used to detect abnormal placement of goods in the warehouse, and the detection result includes the location of the goods and the probability of abnormal placement, and the trained target detection model is jointly trained by federated learning based on sample video or image data of the placement of goods in the first warehouse and the second warehouse; The method further comprises: Obtaining sample video or image data of goods placed in the first warehouse and manually marked goods positions and corresponding placement states, wherein the placement states include abnormal placement or normal placement; According to the sample video or image data and the cargo position and the corresponding placement status, the target detection model to be trained is trained to obtain a candidate target detection model; Sending model parameters in the candidate target detection model to a server; Receiving aggregated model parameters sent by the server, where the aggregated model parameters are obtained by the server by aggregating model parameters received from multiple warehouses; The candidate target detection model is updated according to the aggregated model parameters to obtain the trained target detection model, where the trained target detection model is an offline model corresponding to the first warehouse.

2. The method according to claim 1, characterized in that The generating a detection result according to the video or image data and the trained target detection model includes: Preprocessing the video or image data; The preprocessed video or image data is input into the trained target detection model to generate the detection result.

3. The method according to claim 1, characterized in that The training of the target detection model to be trained according to the sample video or image data and the cargo position and the corresponding placement state includes: Preprocessing the sample video or image data; The target detection model to be trained is trained according to the preprocessed sample video or image data and the cargo position and the corresponding placement status.

4. The method according to claim 1, characterized in that Also includes: The detection result is sent to the storage center of the first warehouse, so that the storage center can send abnormal monitoring information to the first warehouse according to the detection result.

5. The method according to claim 2 or 3, characterized in that: The preprocessing includes data cleaning and / or feature engineering.

6. A device for monitoring abnormal cargo, characterized in that: include: A first acquisition module is configured to acquire video or image data of goods placed in the first warehouse; A first generating module is configured to generate a detection result according to the video or image data and a trained target detection model, wherein the target detection model is used to detect abnormal placement of goods in the warehouse, and the detection result includes the location of the goods and the probability of abnormal placement, and the trained target detection model is jointly trained by federated learning based on sample video or image data of the placement of goods in the first warehouse and the second warehouse; A second acquisition module is configured to acquire sample video or image data of the placement of goods in the first warehouse and manually marked positions of goods and corresponding placement states, wherein the placement states include abnormal placement or normal placement; A training module is configured to train the target detection model to be trained according to the sample video or image data and the cargo position and corresponding placement status to obtain a candidate target detection model; A second generating module is configured to generate the trained target detection model according to the candidate target detection model; The second generation module comprises: A sending unit, configured to send the model parameters in the candidate target detection model to a server; A receiving unit is configured to receive the aggregated model parameters sent by the server, where the aggregated model parameters are obtained by the server by aggregating the model parameters of multiple warehouses received; An updating unit is configured to update the candidate target detection model according to the aggregated model parameters to obtain the trained target detection model, where the trained target detection model is an offline model corresponding to the first warehouse.

7. The device according to claim 6, characterized in that The first generation module comprises: A first preprocessing unit, configured to preprocess the video or image data; The generating unit is configured to input the preprocessed video or image data into the trained target detection model to generate the detection result.

8. The device according to claim 6, characterized in that The training module includes: A second preprocessing unit, configured to preprocess the sample video or image data; The training unit is configured to train the target detection model to be trained according to the preprocessed sample video or image data and the cargo position and corresponding placement status.

9. The device according to claim 6, characterized in that Also includes: The sending module is configured to send the detection result to the storage center of the first warehouse, so that the storage center can send abnormal monitoring information to the first warehouse according to the detection result.

10. The device according to claim 7 or 8, characterized in that The preprocessing includes data cleaning and / or feature engineering.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for monitoring abnormal cargo according to any one of claims 1 to 5.

12. A computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the method for monitoring abnormal cargo according to any one of claims 1 to 5.

Images