Discharging control method, device and system

Abstract

The invention discloses an unloading control method, device and system, and relates to the field of control. Based on material weight information of mineral powder loaded by a vehicle loaded with materials, the unloading time is calculated in advance, after it is determined that the vehicle is parked in a parking space of a corresponding unloading hole groove and unloading is started, the pre-calculation time is waited, and the unloading time is shortened; the vehicle residual material state is determined through image detection, and the vehicle is controlled so that the vehicle can conduct sudden entering and sudden braking of the target number of times matched with the vehicle residual material state, so that refined control over residual material discharging can be achieved, the situation that sudden entering and sudden braking control is conducted repeatedly is avoided to a certain degree, and the vehicle residual material discharging efficiency is improved. The residual material state detection is started after the pre-calculation duration, and compared with the scheme of real-time detection in the discharging process, the platform calculation power is saved; in addition, in the unloading process through the unloading device, participation of a driver and factory ground crew is not needed, and manpower resources can be saved.

Description

Technical Field

[0001] The present application relates to the field of control technology, and in particular to a material discharging control method, device and system. Background Art

[0002] Currently, a driver usually drives a vehicle loaded with iron ore powder into a steel plant, and then the driver and ground staff cooperate to complete the iron ore powder unloading task.

[0003] In addition, steel mills often receive many types of iron ore powder, such as lead concentrate powder PB powder, Yangdi powder, super special powder, iron magnesium FMG mixed powder, Brazilian concentrate powder, Indian powder, etc. Different types of iron ore powder often have different physical and chemical properties, which makes it difficult for some iron ore powder to flow cleanly. In order to reduce energy consumption, the current method is for ground staff to check the unloading status of iron ore powder in real time, and instruct the vehicle driver to control the vehicle according to the unloading status, so that the ore powder loaded on the vehicle can flow cleanly.

[0004] Based on this, during the unloading process, the driver needs to wait for instructions from the ground staff in the vehicle, and the ground staff needs to check the iron ore powder unloading status in real time, which is time-consuming and laborious. Summary of the invention

[0005] In view of the above problems, the present application provides a discharging control method, device and system to realize intelligent iron ore powder discharging tasks and reduce the possibility of human errors.

[0006] The specific plan is as follows:

[0007] The first aspect of the present application provides a discharge control method, which is applied to a discharge control platform. The method comprises:

[0008] Obtaining material weight information of the mineral powder loaded on the unloading vehicle, and calculating an estimated unloading time based on the material weight information as the pre-calculated time;

[0009] After determining that the unloading vehicle has parked in the parking space of the target unloading slot and has started unloading, waiting for the pre-calculated time; the target unloading slot is a unloading slot in the unloading area that matches the material type information of the mineral powder loaded by the unloading vehicle;

[0010] Acquire a current residual material state image acquired by an image acquisition device disposed at the target unloading slot;

[0011] Calling a pre-trained residual material state detection model to process the current residual material state image to obtain the residual material remaining ratio of the vehicle to be unloaded; wherein the residual material state detection model is trained using residual material state sample images marked with residual material remaining ratios;

[0012] Determine the target number of rapid advances and sudden brakes required to make the residual materials flow out completely based on at least the remaining proportion of the residual materials in the vehicle to be unloaded;

[0013] If the determined target number is greater than zero, a residual material processing instruction is sent to the vehicle to be unloaded, and the residual material processing instruction is used to control the vehicle to be unloaded to perform the target number of rapid advances and sudden brakes. After the vehicle to be unloaded completes the rapid advances and sudden brakes, the current residual material status image captured by the image acquisition device set at the target unloading hole slot is obtained, and the command returns to execute the call of the pre-trained residual material status detection model to process the current residual material status image.

[0014] A second aspect of the present application provides a discharge control device, which is applied to a discharge control platform, and the device comprises:

[0015] The unloading time calculation unit is used to obtain the material weight information of the mineral powder loaded on the unloading vehicle, and calculate the estimated unloading time according to the material weight information as the pre-calculated time;

[0016] A material unloading monitoring unit is used to wait for the pre-calculated time after determining that the vehicle to be unloaded has parked in the parking space of the target unloading slot and started unloading, obtain the current residual material status image acquired by the image acquisition device set at the target unloading slot, call the pre-trained residual material status detection model to process the current residual material status image, and obtain the residual material remaining ratio of the vehicle to be unloaded; wherein, the target unloading slot is a unloading slot in the unloading area that matches the material type information of the mineral powder loaded by the vehicle to be unloaded, and the residual material status detection model is trained using residual material status sample images marked with residual material remaining ratios;

[0017] The unloading monitoring unit is further used to determine a target number of sudden advances and sudden brakes required to clear the residual material based on at least the remaining proportion of the residual material on the vehicle to be unloaded; if the determined target number is greater than zero, a residual material processing instruction is sent to the vehicle to be unloaded, and the residual material processing instruction is used to control the vehicle to be unloaded to perform the target number of sudden advances and sudden brakes, and after the vehicle to be unloaded completes the sudden advances and sudden brakes, obtain the current residual material status image captured by the image acquisition device set at the target unloading hole slot, and return to execute and call the pre-trained residual material status detection model to process the current residual material status image.

[0018] The third aspect of the present application provides a material unloading control system, comprising: a vehicle, plant equipment and a material unloading control platform; the plant equipment comprises: image acquisition devices respectively arranged at each material unloading slot in the material unloading area;

[0019] The unloading control platform obtains the material weight information of the mineral powder loaded on the vehicle, and calculates the estimated unloading time according to the material weight information as the pre-calculated time;

[0020] After determining that the vehicle has parked in the parking space of the target unloading slot and has started unloading, the unloading control platform waits for the pre-calculated time to obtain the current residual material state image acquired by the image acquisition device provided at the target unloading slot; wherein the target unloading slot is the unloading slot in the unloading area that matches the material type information of the mineral powder loaded by the vehicle;

[0021] The unloading control platform calls a pre-trained residual material state detection model to process the current residual material state image to obtain the residual material remaining ratio of the vehicle; wherein the residual material state detection model is trained using residual material state sample images marked with residual material remaining ratios;

[0022] The unloading control platform determines the target number of sudden advances and sudden brakes required to make the residual materials flow out at least according to the residual material remaining ratio of the vehicle; if the determined target number is greater than zero, a residual material processing instruction is sent to the vehicle, and after the vehicle completes the sudden advance and sudden brake, a current residual material state image captured by an image acquisition device set at the target unloading hole slot is obtained, and the pre-trained residual material state detection model is called back to execute and process the current residual material state image;

[0023] The vehicle performs the target number of rapid advances and sudden brakes in response to the residual material handling instruction.

[0024] By means of the above technical scheme, the present application pre-calculates the unloading time based on the material weight information of the mineral powder loaded on the vehicle loaded with materials, and then after determining to park in the parking space corresponding to the unloading slot and starting to unload, waits for the pre-calculated time, determines the residual material status of the vehicle through image detection and recognition, controls the vehicle to make the vehicle perform a target number of rapid advances and sudden brakes, and realizes the residual material unloading by colliding the vehicle box with the rear tailgate of the vehicle box. Since the target number of times matches the residual material status of the vehicle, the present application can realize the refined control of the unloading of the remaining residual material, and to a certain extent avoids the situation of repeated rapid advances and sudden brakes control. In addition, the present application starts to detect the residual material status after the expected unloading time. Compared with the scheme of real-time detection during the unloading process, the present application saves platform computing power.

[0025] In the process of unloading using this application, there is no need for the participation of drivers and ground staff in the factory. In other words, the unloading control solution provided by this application realizes the intelligent and unmanned iron ore powder unloading task to a certain extent through automatic control, which helps to save human resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0027] Figure 1 A schematic diagram of a flow chart of a discharge control method provided in this application;

[0028] Figure 2 A schematic diagram of the structure of a discharge control device provided in this application;

[0029] Figure 3 A schematic diagram of the hardware structure of a discharge control platform provided in this application;

[0030] Figure 4 A structural schematic diagram of a discharge control system provided in this application;

[0031] Figure 5 The EE architecture diagram of an unmanned tractor for unloading iron ore powder is shown as an example;

[0032] Figure 6 The following is a schematic diagram showing the communication architecture of video surveillance data within the unloading control system. DETAILED DESCRIPTION

[0033] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation mode of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. It is known to those skilled in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0034] The present application provides a discharging control method, device and system, which can be applied to realize intelligent iron ore powder discharging tasks.

[0035] Figure 1 This is a flow chart of a method for controlling a material unloading according to an embodiment of the present application. The material unloading control scheme provided by the present application can be applied to a material unloading control platform, which can be a terminal device with data processing and communication capabilities, such as a computer, a server, a cloud, etc., and the present application does not make any restrictions. The following is an example of a material unloading control process of a vehicle, which can be called a target vehicle.

[0036] Combination Figure 1 As shown, the unloading control method may include the following steps:

[0037] Step S101, obtaining material weight information of the mineral powder loaded on the to-be-unloaded vehicle, and calculating the unloading time according to the material weight information as the pre-calculated time;

[0038] It should be noted that the unloading speed has no direct relationship with the lifting angle of the car body, and is affected by various factors such as the performance of the unloading equipment and the characteristics of the iron ore powder. Based on this, in one possible implementation, a calculation model for the unloading speed can be constructed based on various factors that have a greater impact on the unloading speed to obtain an estimated value of the unloading speed that matches the actual situation, so as to calculate the estimated value of the unloading time required to complete the unloading. In another possible implementation, an empirical value of the unloading speed can be set, and the estimated value of the unloading time can be calculated based on the same empirical value of the unloading speed in the future, so as to simplify the complexity of data processing and improve control efficiency.

[0039] Step S102: After determining that the vehicle to be unloaded has parked in the parking space of the target unloading slot and has started unloading, wait for the pre-calculated time.

[0040] It should be noted that the unloading area is often provided with unloading slots for receiving different types of mineral powders; the target unloading slot is the unloading slot in the unloading area that matches the material type information of the mineral powder loaded on the vehicle to be unloaded.

[0041] Step S103, obtaining a current residual material state image captured by an image capture device disposed at the target unloading slot.

[0042] Exemplarily, the empirical value of the unloading speed can be set to 1 ton / minute. On this basis, assuming that the material weight information is expressed as material weight = ω tons, then: the estimated value of the unloading time = ω tons ÷ 1 ton / minute = ω minutes; in this case, the current residual material status image of the vehicle obtained for the first time can be the residual material status image ω minutes after the lifting angle of the vehicle body to be unloaded reaches the preset angle.

[0043] Step S104: calling a pre-trained residual material state detection model to process the current residual material state image to obtain the residual material remaining ratio of the vehicle to be unloaded.

[0044] The residual material state detection model is trained using residual material state sample images annotated with residual material remaining ratios.

[0045] Step S105, determining a target number of rapid advances and sudden brakes required to clear the residual material at least based on the remaining proportion of the residual material in the vehicle to be unloaded.

[0046] Step S106, when the target number is determined to be greater than zero, a residual material processing instruction is sent to the vehicle to be unloaded, and after the vehicle to be unloaded completes rapid advancement and sudden braking, the current residual material status image captured by the image acquisition device set at the target unloading hole is obtained, and the process returns to step S104, calling the pre-trained residual material status detection model to process the current residual material status image.

[0047] Among them, determining a target number greater than zero can indicate that there are a lot of residual materials remaining in the current car box, and the iron ore powder is difficult to flow out on its own, and auxiliary measures need to be taken to achieve the residual material unloading; determining a target number of zero can indicate that there are a few residual materials remaining in the current car box, which can be ignored, or there are no residual materials in the current car box, that is, the vehicle has completed unloading. Among them, the residual material processing instruction can be used to control the vehicle to be unloaded to perform the target number of rapid advances and sudden brakes. By rapid advances and sudden brakes, the rear tailgate of the car box can collide with the car box to assist in the flow of the remaining residual materials.

[0048] This embodiment pre-calculates the unloading time based on the material weight information of the mineral powder loaded on the vehicle loaded with materials, and then waits for the pre-calculated time after determining to park in the parking space corresponding to the unloading slot and starting to unload, and determines the residual material status of the vehicle through image detection and recognition, and controls the vehicle to make the vehicle perform a target number of rapid advances and sudden brakes, and realizes the residual material unloading by colliding the vehicle box with the tailgate of the vehicle box. Since the target number of times matches the residual material status of the vehicle, the present application can realize the refined control of the unloading of the remaining residual material, and to a certain extent avoids the situation of repeated rapid advances and sudden brakes control. In addition, the present application starts the residual material status detection after the estimated unloading time. Compared with the real-time detection scheme during the unloading process, the present application saves the platform computing power.

[0049] In the process of unloading using this application, there is no need for the participation of drivers and ground staff in the factory. In other words, the unloading control solution provided by this application realizes the intelligent and unmanned iron ore powder unloading task to a certain extent through automatic control, which helps to save human resources.

[0050] The following is an exemplary description of the training process of the residual material state detection model provided in the embodiment of the present application. In a possible implementation, the residual material state detection model can be trained through the following steps:

[0051] The first step is to collect video stream data and extract sample images of residual material status;

[0052] The total number of sample images may be no less than 3 million, and the total space occupied may be no less than 4.5 TB. For the collected video stream, static key frames may be extracted to form a key image sequence as a residual state sample image.

[0053] The second step is to preprocess the data.

[0054] Specifically, the videos and images obtained in the first step that are of poor quality, blurry, and have no scene recognition significance are filtered and deleted, and the videos and images after filtering and deletion are enhanced, that is, a series of random transformations are performed on the original data to expand the data set. Exemplary data enhancement methods may include: flipping, rotating, scaling, cropping, adding noise, etc. Through data preprocessing, the amount of sample data can be effectively increased, providing a basis for preventing model overfitting and improving the generalization ability of model reasoning.

[0055] Step 3: Configure the training data set.

[0056] Specifically, the static image is annotated with a detection frame, and the residual material remaining proportion η represented by the annotated data is added.

[0057] Step 4: Configure the model.

[0058] Exemplarily, it may include: creating a Conv2D convolution layer, the input is a static image of 28×28 pixels, the layer has 32 convolution kernels, the convolution kernel size is 3, and the convolution step is 1; creating a MaxPool2D maximum pooling layer, the pooling window size is 2, and the pooling step is 2; creating a Conv2D convolution layer, the layer has 64 convolution kernels, the convolution kernel size is 3, and the convolution step is 1; creating a MaxPool2D maximum pooling layer, the pooling window size is 2, and the pooling step is 2; creating a fully connected layer Flatten, flattening the model, flattening the input 2D image vector into a 1D image vector, and splicing it with the numerical data η, to achieve The fusion of image and numerical data is now realized; a fully connected layer Dense is created to reduce the image dimension from 784 to 256, and a non-linear activation function relu is used; a fully connected layer Dense is created to reduce the image dimension from 256 to 10, and a non-linear activation function relu is used; Adam is used as the training optimizer, sparse_categorical_crossentropy is used as the multi-category classification cross entropy loss function, and accuracy is used to measure the model fit; batchsize=128, epoch=6000 are set, and the model is trained using the PyTorch engine and annotated dataset.

[0059] Step 5: Generate model.

[0060] Exemplarily, after training is completed, the weight and structure information of the current model can be saved in the file model.pth as the generated model.

[0061] Step 6. Use the PyTorch engine and validation data samples independent of the training samples in the training dataset to verify and evaluate the generated model.

[0062] Step 7. Continuously iterate based on the above steps to continuously optimize the model weight parameters and train a model with satisfactory reasoning accuracy as the pre-trained residual material state detection model.

[0063] In one or more embodiments provided in the present application, the above step S105, determining the target number of rapid advances and sudden brakes required to make the residual material flow out at least according to the residual material remaining ratio of the vehicle to be unloaded, may include:

[0064] According to the material type information and the remaining proportion of the residual material in the vehicle to be unloaded, the target number of rapid advances and sudden brakes required to clear the residual material is determined.

[0065] It should be noted that different materials have different particle sizes and fluidities. Based on this, determining the number of sudden advances and sudden brakes according to the material type information and the remaining proportion of residual material can achieve more refined residual material unloading control tasks and improve the intelligence of unloading control.

[0066] In one or more embodiments provided in the present application, before obtaining the material weight information of the mineral powder loaded on the unloading vehicle, the method may further include:

[0067] Step S001, obtaining unloading task information.

[0068] The unloading task information may include the vehicle identification information of the vehicle to be unloaded and the material type information and material weight information of the mineral powder loaded by the vehicle to be unloaded. Exemplarily, the vehicle identification information may include the license plate number and the vehicle identification number (VIN). The vehicle to be unloaded has completed the entry and weighing, and the vehicle to be unloaded is connected to the factory wireless network, that is, after the target vehicle has completed the entry and weighing, it is called the vehicle to be unloaded.

[0069] In a possible implementation, a port loading operation management system may be provided at the loading port side to manage the loading data of vehicles loading from the current port, which may specifically include the vehicle identification information of the vehicle and the type and weight data of the materials loaded. On this basis, the above information may be obtained from the loading operation management system, and then after the target vehicle completes the weighing at the factory, it constitutes the unloading task information of the target vehicle.

[0070] Step S002: According to the pre-configured correspondence between the unloading slots and the material types, a unloading slot matching the mineral powder loaded by the unloading vehicle is determined from the unloading slots in the unloading area as a target unloading slot.

[0071] Exemplarily, after determining the target unloading slot, the state of the target unloading slot can be detected first, for example, the task state of the previous unloading task corresponding to the target unloading slot can be detected; if the previous unloading task corresponding to the target unloading slot is completed, the target unloading slot is idle and can support the vehicle to unload; if the previous unloading task corresponding to the target unloading slot is not completed, the target unloading slot is occupied and does not support the vehicle to unload. In this case, the task state of the unloading task corresponding to the target unloading slot can be monitored until the task is completed and the target unloading slot is idle. It should be noted that the order of unloading tasks corresponding to any unloading slot can be related to the time when the corresponding vehicle arrives at the factory and is connected to the network, that is, they are sorted according to the order in which the corresponding vehicles arrive at the factory and are connected to the network.

[0072] Optionally, after determining the target discharge slot, the method may further include:

[0073] When the previous unloading task corresponding to the target unloading slot is not completed, a waiting unloading preparation instruction is sent to the vehicle to be unloaded. The waiting unloading preparation instruction is used to control the vehicle to be unloaded to drive to an empty parking space in the factory parking lot in an unmanned manner and park in the empty parking space.

[0074] Based on this, the vehicle to be unloaded can automatically go to the parking lot in response to the waiting unloading preparation instruction, park in an empty parking space, and turn off the engine to wait for the unloading preparation instruction.

[0075] On the basis of the above, the method may further include:

[0076] A real-time image of the parking lot captured by an image acquisition device installed in the factory parking lot is obtained, and a pre-trained empty parking space detection model is called to process the acquired real-time image of the parking lot to obtain an empty parking space recognition result in the factory parking lot, so as to provide a discharge control platform with scheduling vehicles waiting for unloading due to occupied unloading slots, such as configuring a waiting unloading preparation instruction for the unloading control platform.

[0077] The obtained empty parking space identification result may be represented by the number of the empty parking space, the location information, and other information that can identify the empty parking space.

[0078] Step S003: When it is determined that the target unloading slot is in an idle state, a unloading preparation instruction is sent to the vehicle to be unloaded.

[0079] The unloading preparation instruction is used to control the unloading vehicle to drive to the parking space of the target unloading slot in an unmanned manner and park at the parking space of the target unloading slot. Correspondingly, if the unloading vehicle receiving the unloading preparation instruction is in a ignition-off waiting state, the unloading preparation instruction can be used to remotely control the unloading vehicle to remotely ignite and start, and to control the unloading vehicle to drive to the parking space of the target unloading slot in an unmanned manner and park at the parking space of the target unloading slot.

[0080] Step S004: after determining that the vehicle to be unloaded has parked in the parking space of the target unloading slot based on the unloading task information, sending an unloading instruction to the vehicle to be unloaded.

[0081] Optionally, determining that the to-be-unloaded vehicle has parked in the parking space of the target unloading slot based on the unloading task information may include:

[0082] Step A, obtaining a vehicle image captured by an image capture device provided at the target unloading slot, wherein the captured vehicle image includes an image area corresponding to the license plate number;

[0083] Step B, calling a pre-trained license plate number recognition model to process the collected vehicle image to obtain a license plate number recognition result, wherein the license plate number recognition model is trained using a sample image of a vehicle marked with a license plate number;

[0084] Step C, when it is determined that the unloading task information matches the obtained license plate number recognition result and the information of the target unloading slot, it is determined that the mineral powder loaded by the current vehicle on the parking space of the target unloading slot can enter the target unloading slot, that is, it is determined that the vehicle to be unloaded has parked in the parking space of the target unloading slot.

[0085] It should be noted that the unloading task information matches the obtained license plate number recognition result and the information of the target unloading slot, indicating that the unloading task currently executed by the unloading control platform is the unloading task of the vehicle waiting to be unloaded at the target unloading slot. Exemplarily, the vehicle's entry unloading reservation record or the vehicle's loading data can be obtained based on the recognized license plate number and the vehicle VIN obtained from the vehicle, which may include the type of materials loaded by the vehicle, the weight of the materials, the weight of the vehicle itself, and other information. On the basis of the above information, combined with the information of the unloading slot at the location where the image acquisition device for collecting vehicle images is set, which may specifically include the slot number and the type of materials acceptable to the slot, the information of the actual unloading task currently being performed can be obtained, and the information of the actual unloading task is matched and checked with the platform's unloading task record to determine whether the vehicle to be unloaded has parked in the parking space of the target unloading slot.

[0086] The unloading instruction can be used to control the vehicle to be unloaded to perform an unloading operation.

[0087] Wherein, the unloading operation at least includes lifting the upper loading box to an oblique position, and the upper loading box after lifting is in a triangle with the ground. It should be noted that the lifting angle of the box during vehicle unloading can be configured according to the repose angle of the mineral powder to increase the possibility of the mineral powder flowing out independently; under normal circumstances, the standard lifting angle of the box during unloading is about 45°. For some sticky or easily sticky mineral powders, a larger lifting angle may be required, such as 50°~55°. Optionally, when the vehicle to be unloaded is provided with a rainproof cloth, the unloading operation may also include: pulling open the rainproof cloth before lifting the upper loading box. After the unloading operation of the vehicle, the iron ore powder automatically flows out of the box and flows into the unloading hole groove on the ground.

[0088] On the basis of the above, the step of determining that the vehicle to be unloaded parks in the parking space of the target unloading slot and starts unloading may include: upon determining that the vehicle to be unloaded responds to the unloading instruction and starts to perform the unloading operation, determining that the vehicle to be unloaded parks in the parking space of the target unloading slot and starts unloading.

[0089] In one or more embodiments provided in this application, the method may further include:

[0090] If the determined target number is equal to zero, a unloading restoration instruction is sent to the vehicle to be unloaded; after it is determined that the vehicle to be unloaded has completed the unloading restoration operation, it is determined that the unloading task of the vehicle to be unloaded is completed.

[0091] The unloading restoration instruction can be used to control the unloading vehicle to perform an unloading restoration operation, wherein the unloading restoration operation at least includes restoring the upper loading box to a horizontal position. Optionally, when the unloading vehicle is provided with a rainproof sheet, the unloading restoration operation can also include: closing the rainproof sheet after restoring the upper loading box to a horizontal position, that is, closing the rainproof sheet to cover the entire upper loading box.

[0092] Based on the above content, after a vehicle loaded with materials arrives at the factory and is connected to the network, this embodiment determines the unloading slot that can receive the loaded mineral powder, that is, the target unloading slot, based on the material type information of the mineral powder loaded by the vehicle; then, when the slot is idle, the vehicle is controlled so that the vehicle reaches the parking space of the slot in an unmanned manner and waits for unloading, thereby reducing the possibility of the vehicle parking and unloading in the wrong slot; after determining that the vehicle has parked at the parking space of the slot, the vehicle is controlled to lift the upper loading box of the vehicle, the vehicle starts unloading, and the unloading status of the vehicle is monitored. When the determined target number is zero, that is, there is no need for sudden advance and sudden braking control, the vehicle is controlled to restore the upper loading box of the vehicle, and it is determined that the vehicle unloading is complete.

[0093] This embodiment realizes the iron ore powder unloading task through automatic control, improves the degree of intelligence and unmanned operation, and reduces the possibility of human error.

[0094] In one or more embodiments provided in the present application, before step S001, obtaining unloading task information, the method may further include the following steps S201-S203:

[0095] Step S201: Send a fully loaded weighing preparation instruction to the fully loaded vehicle to be weighed.

[0096] The fully loaded vehicle to be weighed may refer to a target vehicle that has entered the factory but has not completed the weighing. The fully loaded weighing preparation instruction may be used to control the fully loaded vehicle to be weighed to drive to the weighing platform in an unmanned manner.

[0097] Step S202: when it is determined that the fully loaded vehicle to be weighed has driven to the weighing platform, obtaining and recording the fully loaded weight value of the fully loaded vehicle to be weighed from the weighing platform.

[0098] In a possible implementation, the above-mentioned determination that the fully loaded vehicle to be weighed has driven to the weighing platform may include:

[0099] Step D, obtaining a vehicle image captured by an image capture device provided at the weighing platform, wherein the captured vehicle image contains an image area corresponding to the license plate number;

[0100] Step E: calling the pre-trained license plate number recognition model to process the collected vehicle image to obtain the license plate number recognition result;

[0101] Step F: When it is determined that the obtained license plate number recognition result matches the vehicle identification information of the fully loaded vehicle to be weighed, it is determined that the fully loaded vehicle to be weighed has driven to the weighing platform.

[0102] In addition, since the unloading control solution provided in the embodiment of the present application is driven by remote control of the vehicle, that is to say, during the unloading process after entering the factory, there is no need for the driver to drive the vehicle. Based on this, when the vehicle drives to the weighing platform for weighing, when the driver is not on the vehicle, a dynamic or static weighing method can be adopted, that is, the vehicle can stop or not when weighing. In a possible implementation, the weighing platform can be provided with a dynamic weighing sensor to collect the weight parameters of the vehicle. Based on this, the unloading control platform can receive the weight parameters reported by the dynamic weighing sensor, and convert the received weight parameters into parameters of the specified weight unit and then record and store them to achieve unified management of the weight parameters.

[0103] Step S203: Determine that the weighing task of the fully loaded vehicle to be weighed is completed, and determine that the fully loaded vehicle to be weighed is the vehicle to be unloaded.

[0104] In one or more embodiments provided in the present application, before obtaining the unloading task information, the method may further include:

[0105] Step G: In response to the image of the vehicle entering the factory captured by the image capture device set at the factory entrance, the pre-trained license plate number recognition model is called to process the image of the vehicle entering the factory to obtain the license plate number recognition result of the vehicle entering the factory.

[0106] The license plate number recognition model is trained using sample vehicle images labeled with license plate numbers.

[0107] Step H: searching the reservation list for a reservation record that matches the license plate number recognition result.

[0108] It should be noted that the driver of a vehicle loaded with iron ore powder can make an appointment to enter the factory for unloading through a mobile phone app, etc. After the appointment is successful, a corresponding appointment record will be generated for acquisition by the unloading control platform. Specifically, the appointment list may include the appointment records of vehicles to be unloaded in the factory, and the appointment records contain the vehicle's license plate number information and the material type and weight of the ore powder loaded by the vehicle.

[0109] If the search is successful, then step I is executed. The search is successful, which indicates that the current vehicle is successfully authenticated and is allowed to enter the factory area to wait for unloading.

[0110] Step I: Control the gate pole set at the factory entrance to lift up so that the inbound vehicles can enter the factory.

[0111] Optionally, after step I, the method may also include: after the incoming vehicle stops at a preset handover position, determining that the incoming vehicle is the fully loaded vehicle to be weighed, and executing the step of sending a fully loaded weighing preparation instruction to the fully loaded vehicle to be weighed.

[0112] The above-mentioned handover position refers to the position where the man-machine handover takes place in the factory. After the driver drives the vehicle into the factory, it stops at the handover position, the driver gets off, and the subsequent vehicle enters the unmanned driving mode. The vehicle entering the unmanned driving mode can periodically report its own driving status data to the unloading control platform. Based on this, the unloading control platform can obtain the vehicle's VIN number, license plate number, type of materials loaded by the vehicle and other information.

[0113] In one or more embodiments provided in the present application, after determining that the unloading task of the vehicle to be unloaded is completed, the method may further include:

[0114] Step S301, determining that the vehicle to be unloaded is a vehicle to be washed.

[0115] Step S302: Send a car wash preparation instruction to the vehicle to be washed.

[0116] The car wash preparation instruction may be used to control the vehicle to be washed to drive to the car wash station and park at the car wash station in an unmanned manner.

[0117] Step S303: after determining that the vehicle to be washed has stopped at the car wash station, sending a water spray control instruction to a sprinkler provided at the car wash station.

[0118] In a possible implementation, the above-mentioned determination that the vehicle to be washed has stopped at the car wash station may include: receiving a car wash readiness notification sent by the vehicle to be washed after entering the car wash station and stopping, thereby determining that the vehicle to be washed has stopped at the car wash station.

[0119] The water spray control instruction can be used to control the sprinkler to perform a water spray operation for a first preset time. For example, the first preset time can be 1-2 minutes; the water spray operation can be from bottom to top and side spray. After the water spray time reaches the first preset time, the sprinkler switch is turned off and the sprinkler stops spraying water.

[0120] Step S304: after determining that the water sprinkler has completed the water spraying operation, determining that the vehicle washing task is completed.

[0121] In one or more embodiments provided in the present application, after step S304, determining that the car washing task of the vehicle to be washed is completed, the method may further include:

[0122] Step S401: Determine that the vehicle to be washed is a vehicle to be blown.

[0123] Step S402: Send an air blowing preparation instruction to the vehicle to be air blown.

[0124] The blowing preparation instruction can be used to control the vehicle to be blown to drive to the blowing station and park at the blowing station in an unmanned manner. It should be noted that the vehicle is in an off state when the car wash is completed. Based on this, the blowing preparation instruction can be used to remotely control the vehicle to be blown to remotely ignite and start, and to control the vehicle to be blown to drive to the blowing station and park at the blowing station in an unmanned manner.

[0125] Step S403: after determining that the vehicle to be blown has stopped at the blow-drying station, sending a blow-drying control instruction to the vehicle blower arranged at the blow-drying station.

[0126] In a possible implementation, the above-mentioned determination that the vehicle to be air-dried has stopped at the air-dried station may include: receiving an air-dried ready notification sent by the vehicle to be air-dried after entering the air-dried station and stopping, thereby determining that the vehicle to be air-dried has stopped at the air-dried station.

[0127] The blowing control instruction may be used to control the vehicle hair dryer to perform a blowing operation for a second preset time. For example, the second preset time may be 1-2 minutes. After the blowing time reaches the second preset time, the vehicle hair dryer switch is turned off, and the vehicle hair dryer stops blowing.

[0128] Step S404: after determining that the vehicle hair dryer completes the blowing operation, determining that the blowing task of the vehicle to be blown is completed.

[0129] In one or more embodiments provided in the present application, after step S404, determining that the air blowing task of the vehicle to be air-blown is completed, the method may further include:

[0130] Step S501: Determine that the vehicle to be blown is an unloaded vehicle to be weighed.

[0131] It should be noted that after the blowing operation, the moisture in the vehicle is blown dry. In this case, weighing can obtain a more accurate vehicle weight.

[0132] Step S502: Send an empty-load weighing preparation instruction to the empty vehicle to be weighed.

[0133] The unloaded weighing preparation instruction can be used to control the unloaded vehicle to be weighed to drive to the weighing platform in an unmanned manner. It should be noted that the vehicle is in an off state when the air blowing is completed. Based on this, the unloaded weighing preparation instruction can be used to remotely control the unloaded vehicle to be weighed to remotely ignite and start, and to control the unloaded vehicle to be weighed to drive to the weighing platform in an unmanned manner.

[0134] Step S503: after determining that the empty vehicle to be weighed has driven to the weighing platform, obtaining and recording the empty weight value of the empty vehicle to be weighed from the weighing platform, and determining that the empty weighing task of the empty vehicle to be weighed is completed.

[0135] The above content provides a solution for obtaining the fully loaded weight value and the unloaded weight value of the target vehicle through automatic control. In other possible implementations, the fully loaded vehicle or the unloaded vehicle can be manually driven for weighing to obtain the weight value of the corresponding state.

[0136] On the basis of the above, after determining that the air blowing task of the vehicle to be air-blown is completed, the method may further include:

[0137] Step S601, obtaining the empty weight value of the weighed empty vehicle and the corresponding fully loaded weight value of the weighed empty vehicle.

[0138] It should be noted that after the unloaded vehicle has completed the unloaded weighing task, it can be called a weighed unloaded vehicle; wherein, the method for obtaining the unloaded weight value of the weighed unloaded vehicle can refer to the above steps S501-S503. The method for collecting the fully loaded weight value corresponding to the weighed unloaded vehicle can refer to the above steps S201-S203.

[0139] Step S602: Calculate the difference between the full-load weight value and the empty-load weight value as the actual material weight value.

[0140] Step S603, determine whether the actual material weight value matches the material weight information within a preset limit, if so, execute step S604, if not, execute step S605.

[0141] Step S604: determine that the weighed empty vehicle is a vehicle to be shipped, send a shipment preparation instruction to the vehicle to be shipped, and output task completion information to the driver of the vehicle to be shipped, so as to instruct the driver to take over the vehicle to be shipped at a preset handover position.

[0142] It should be noted that the degree of match between the actual material weight value and the material weight information is within the preset limit, which can indicate that the weight of the mineral powder loaded on the vehicle is normal. Based on this, the unloading control platform can configure the completion record of the unloading task into the factory, and the current vehicle can be prepared to leave the factory, such as sending a notification of completion of the unloading task to the driver of the current vehicle to instruct the driver to get on the vehicle and take over the vehicle.

[0143] The delivery preparation instruction may be used to control the vehicle to be delivered from the factory to drive to the handover position and park at the handover position in an unmanned manner.

[0144] Step S605: output warning information.

[0145] The warning information is used to indicate that the weight of the mineral powder loaded on the vehicle is abnormal. In this case, the driver may have stolen the material, that is, the driver's stealing behavior can be handled based on the warning information.

[0146] In one or more embodiments provided in the present application, after the vehicle to be shipped stops at the handover position, the method may further include:

[0147] Step S701, in response to the out-of-factory vehicle image captured by the image capture device arranged at the factory exit, calling the license plate number recognition model, processing the out-of-factory vehicle image, and obtaining the out-of-factory vehicle license plate number recognition result.

[0148] Step S702: When it is determined that the leaving vehicle is the waiting leaving vehicle, the gate pole provided at the factory exit is controlled to be raised so as to allow the waiting leaving vehicle to exit the factory.

[0149] Based on the above content, an intelligent, unmanned and standardized unloading control solution is provided for the entire process including fully loaded vehicles entering the factory, fully loaded weighing, unloading, car washing, air blowing, empty load weighing and leaving the factory. This solution greatly reduces the degree of manual participation.

[0150] In one or more embodiments provided in the present application, the method may further include:

[0151] Step L, in response to an image of a vehicle requested to enter the preset position acquired by an image acquisition device disposed at the preset position, calling the license plate number recognition model to process the image of the vehicle requested to enter the preset position, and obtaining a license plate number recognition result of the vehicle requested to enter the preset position;

[0152] Step M: When it is determined that the vehicle requesting to enter the preset position is to perform a preset task corresponding to the preset position, the gate rod set at the preset position is controlled to be lifted so that the vehicle requesting to enter the preset position can enter the preset position.

[0153] Among them, the preset position is any position among the weighing platform, the car washing platform and the blowing platform; the preset task corresponding to the weighing platform is a full-load weighing task or an empty-load weighing task, the preset task corresponding to the car washing platform is a car washing task, and the preset task corresponding to the blowing platform is a blowing task.

[0154] With the above solution, barriers are set at the entrances of different task areas within the factory. The unloading control platform automatically releases vehicles that meet the entry conditions through license plate detection and recognition, and intercepts vehicles that do not meet the entry conditions, thereby improving the reliability of the entire process control of entering the factory for unloading.

[0155] In one or more embodiments provided in the present application, when the vehicle entering the factory is a vehicle supporting automatic driving, the following may also be included in the case of a search hit:

[0156] A factory map is sent to the vehicle supporting autonomous driving.

[0157] It should be noted that the steel plant has built high-precision map data of the factory area in advance, which may include the location data of buildings, obstacles, lane lines, traffic markings, steel plant entrances and exits, unloading areas, parking lots / parking spaces, unloading holes, weighing platforms, car wash stations, and hair dryer stations that affect the traffic environment.

[0158] Based on this, after the unloading vehicle enters the factory and connects to the factory wireless network, the unloading control platform can send a high-precision factory map to the connected vehicle to assist the vehicle in environmental perception and high-precision positioning. In addition, since the steel plant is a closed traffic environment, the road and building information is relatively fixed, so there is no need for frequent map collection and repeated map construction.

[0159] In the above case, the preparation instruction sent to the vehicle supporting autonomous driving may include: location information corresponding to the preparation instruction.

[0160] Among them, the preparation instruction can be used to control the vehicle that supports autonomous driving to perform autonomous driving operations corresponding to the preparation instruction based on the factory map and the location information.

[0161] The preparation instruction is any one of the unloading preparation instruction, the car washing preparation instruction, the blowing preparation instruction, the full load weighing preparation instruction, the empty load weighing preparation instruction and the factory preparation instruction; the position information corresponding to the unloading preparation instruction is the position information of the parking space of the target unloading hole slot, the position information corresponding to the car washing preparation instruction is the position information of the car washing station, the position information corresponding to the blowing preparation instruction is the position information of the blowing station, the position information corresponding to the full load weighing preparation instruction and the empty load weighing preparation instruction is the position information of the weighing station, and the position information corresponding to the factory preparation instruction is the position information of the handover position.

[0162] It should be noted that, in the case where barriers are set at the entrances of different mission areas, the above-mentioned position information may include: the position information of the barrier set at the entrance of the mission area corresponding to the preparation instruction and the position information of the mission area corresponding to the preparation instruction. Based on the above-mentioned position information, the vehicle can automatically drive to the barrier set at the entrance of the corresponding area, wait for the barrier to be released, and automatically drive to the corresponding area after the barrier is released.

[0163] Based on the above content, the unmanned driving method described in this application may refer to an automatic driving method; in another possible implementation, it may also refer to a remote driving method.

[0164] The unloading control device provided in the embodiment of the present application is described below. The unloading control device described below and the unloading control method described above can be referenced to each other.

[0165] Figure 2 This is a schematic diagram of the structure of a discharge control device disclosed in an embodiment of the present application. The device can be applied to a discharge control platform. Figure 2 As shown, the device may include:

[0166] The unloading time calculation unit 11 is used to obtain the material weight information of the mineral powder loaded on the unloading vehicle, and calculate the unloading time according to the material weight information as the pre-calculated time;

[0167] The unloading monitoring unit 12 is used to wait for the pre-calculated time after determining that the vehicle to be unloaded has parked in the parking space of the target unloading slot and started unloading, obtain the current residual material state image acquired by the image acquisition device set at the target unloading slot, call the pre-trained residual material state detection model to process the current residual material state image, and obtain the residual material remaining ratio of the vehicle to be unloaded; wherein, the target unloading slot is a unloading slot in the unloading area that matches the material type information of the mineral powder loaded by the vehicle to be unloaded, and the residual material state detection model is trained using residual material state sample images marked with residual material remaining ratios;

[0168] The unloading monitoring unit 12 is further used to determine the target number of sudden advances and sudden brakes required to clear the residual material based on at least the remaining proportion of the residual material in the vehicle to be unloaded; if the determined target number is greater than zero, a residual material processing instruction is sent to the vehicle to be unloaded, and the residual material processing instruction is used to control the vehicle to be unloaded to perform the target number of sudden advances and sudden brakes, and after the vehicle to be unloaded completes the sudden advances and sudden brakes, obtain the current residual material status image captured by the image acquisition device set at the target unloading hole slot, and return to execute the call of the pre-trained residual material status detection model to process the current residual material status image.

[0169] In one or more embodiments provided in the present application, the device may further include:

[0170] An information acquisition unit is used to acquire unloading task information, wherein the unloading task information includes vehicle identification information of the vehicle to be unloaded and material type information and material weight information of the mineral powder loaded by the vehicle to be unloaded; the vehicle to be unloaded has completed the weighing at the factory, and the vehicle to be unloaded is connected to the factory wireless network;

[0171] A unloading control unit is used to determine, from the unloading slots in the unloading area, a unloading slot that matches the mineral powder loaded by the unloading vehicle according to the pre-configured correspondence between the unloading slots and the material types, as a target unloading slot; when it is determined that the target unloading slot is in an idle state, a unloading preparation instruction is sent to the unloading vehicle, and the unloading preparation instruction is used to control the unloading vehicle to drive to the parking space of the target unloading slot in an unmanned manner and park at the parking space of the target unloading slot; after it is determined based on the unloading task information that the unloading vehicle has parked in the parking space of the target unloading slot, a unloading instruction is sent to the unloading vehicle, and the unloading instruction is used to control the unloading vehicle to perform an unloading operation, and the unloading operation at least includes lifting the upper loading box to an oblique position;

[0172] On the basis of the above, the process in which the unloading monitoring unit 12 determines that the vehicle to be unloaded parks in the parking space of the target unloading slot and starts unloading may include: when it is determined that the vehicle to be unloaded responds to the unloading instruction and starts to perform the unloading operation, determining that the vehicle to be unloaded parks in the parking space of the target unloading slot and starts unloading.

[0173] In one or more embodiments provided in this application,

[0174] The unloading control unit is also used to send an unloading restoration instruction to the vehicle to be unloaded when the target number determined by the unloading monitoring unit is zero, that is, the residual material has been completely flowed out. The unloading restoration instruction is used to control the vehicle to be unloaded to perform an unloading restoration operation, and the unloading restoration operation at least includes restoring the upper loading box to a horizontal position; after determining that the vehicle to be unloaded has completed the unloading restoration operation, it is determined that the unloading task of the vehicle to be unloaded is completed.

[0175] In one or more embodiments provided in the present application, the process in which the unloading monitoring unit 12 determines the target number of rapid advances and sudden brakes required to make the residual material flow out completely based on at least the residual material remaining ratio of the vehicle to be unloaded may include:

[0176] According to the material type information and the remaining proportion of the residual material in the vehicle to be unloaded, the target number of rapid advances and sudden brakes required to clear the residual material is determined.

[0177] In one or more embodiments provided in the present application, the device may further include:

[0178] A car wash control unit is used to determine that the vehicle to be unloaded is a vehicle to be washed after determining that the unloading task of the vehicle to be unloaded is completed; send a car wash preparation instruction to the vehicle to be washed, and the car wash preparation instruction is used to control the vehicle to be washed to drive to the car wash station in an unmanned manner and park at the car wash station; after determining that the vehicle to be washed has stopped at the car wash station, send a water spray control instruction to the sprinkler provided at the car wash station, and the water spray control instruction is used to control the sprinkler to perform a water spray operation for a first preset time; after determining that the sprinkler has completed the water spray operation, it is determined that the car wash task of the vehicle to be washed is completed.

[0179] In one or more embodiments provided in the present application, the device may further include:

[0180] A blowing control unit is used to determine that the vehicle to be washed is a vehicle to be blown after determining that the car washing task of the vehicle to be washed is completed; send a blowing preparation instruction to the vehicle to be blown, and the blowing preparation instruction is used to control the vehicle to be blown to drive to the blowing station in an unmanned manner and park at the blowing station; after determining that the vehicle to be blown has stopped at the blowing station, send a blowing control instruction to the vehicle blower set at the blowing station, and the blowing control instruction is used to control the vehicle blower to perform a blowing operation for a second preset time; after determining that the vehicle blower has completed the blowing operation, it is determined that the blowing task of the vehicle to be blown is completed.

[0181] In one or more embodiments provided in the present application, the device may further include:

[0182] The material weight monitoring unit is used to obtain the empty weight value of a weighed empty vehicle and the full load weight value corresponding to the weighed empty vehicle; calculate the difference between the full load weight value and the empty weight value as the actual material weight value; if the degree of matching between the actual material weight value and the material weight information is within a preset limit, it is determined that the weight of the mineral powder loaded on the weighed empty vehicle is normal; if the degree of matching between the actual material weight value and the material weight information is not within the preset limit, an alarm message is output, and the alarm message is used to indicate that the weight of the mineral powder loaded on the vehicle is abnormal.

[0183] A factory delivery control unit is used to determine that the weighed empty vehicle is a vehicle to be shipped if the weight of the mineral powder loaded on the empty vehicle to be weighed is normal, send a factory delivery preparation instruction to the vehicle to be shipped and output task completion information to the driver of the vehicle to be shipped, so as to instruct the driver to take over the vehicle to be shipped at a preset handover position; the factory delivery preparation instruction is used to control the vehicle to be shipped to drive to the handover position in an unmanned manner and stop at the handover position.

[0184] In one or more embodiments provided in the present application, the device may further include:

[0185] An entrance control unit is used to respond to the image of the incoming vehicle collected by the image acquisition device set at the factory entrance, call the pre-trained license plate number recognition model, process the image of the incoming vehicle, and obtain the license plate number recognition result of the incoming vehicle; the license plate number recognition model is trained using the sample image of the vehicle marked with the license plate number; search for the reservation record matching the license plate number recognition result from the reservation list; the reservation list includes the reservation records of the vehicles to be unloaded in the factory, and the reservation records contain the vehicle's license plate number information and the material type and weight of the mineral powder loaded by the vehicle; if the search is successful, control the gate pole set at the factory entrance to be lifted to allow the incoming vehicle to enter the factory; and after the incoming vehicle stops at the handover position, determine that the incoming vehicle is the fully loaded vehicle to be weighed, and send a fully loaded weighing preparation instruction to the fully loaded vehicle to be weighed; the fully loaded weighing preparation instruction is used to control the fully loaded vehicle to be weighed to drive to the weighing platform in an unmanned manner;

[0186] The factory control unit is also used to, after the vehicle to be shipped stops at the handover position, respond to the image of the factory vehicle captured by the image capture device set at the factory exit, call the license plate number recognition model, process the image of the factory vehicle, and obtain the license plate number recognition result of the factory vehicle; when it is determined that the factory vehicle is the vehicle to be shipped, control the gate pole set at the factory exit to lift up so that the vehicle to be shipped can leave the factory.

[0187] In one or more embodiments provided in the present application, the inlet control unit may also be used for:

[0188] In response to an image of a vehicle requested to enter the preset position acquired by an image acquisition device disposed at the preset position, the license plate number recognition model is called to process the image of the vehicle requested to enter the preset position to obtain a license plate number recognition result of the vehicle requested to enter the preset position;

[0189] When it is determined that the vehicle requesting to enter the preset position is to perform a preset task corresponding to the preset position, controlling the gate rod set at the preset position to be lifted so that the vehicle requesting to enter the preset position can enter the preset position;

[0190] Among them, the preset position is any position among the weighing platform, the car washing platform and the blowing platform; the preset task corresponding to the weighing platform is a full-load weighing task or an empty-load weighing task, the preset task corresponding to the car washing platform is a car washing task, and the preset task corresponding to the blowing platform is a blowing task.

[0191] In one or more embodiments provided by the present application, when the inbound vehicle is a vehicle supporting autonomous driving, the preparation instruction sent to the vehicle supporting autonomous driving includes: location information corresponding to the preparation instruction, the preparation instruction is used to control the vehicle supporting autonomous driving to perform an autonomous driving operation corresponding to the preparation instruction based on a pre-acquired plant map and the location information; the plant map is sent to the vehicle supporting autonomous driving by the unloading control platform when a hit is found;

[0192] Among them, the preparation instruction is any one of the unloading preparation instruction, the car washing preparation instruction, the blowing preparation instruction, the full load weighing preparation instruction, the empty load weighing preparation instruction and the factory preparation instruction; the position information corresponding to the unloading preparation instruction is the position information of the parking space of the target unloading hole slot, the position information corresponding to the car washing preparation instruction is the position information of the car washing station, the position information corresponding to the blowing preparation instruction is the position information of the blowing station, the position information corresponding to the full load weighing preparation instruction and the empty load weighing preparation instruction is the position information of the weighing station, and the position information corresponding to the factory preparation instruction is the position information of the handover position.

[0193] The unloading control device provided in the embodiment of the present application can be applied to the unloading control platform, such as a terminal with data processing capability: a computer, a server, etc. Optionally, Figure 3 The hardware structure diagram of the unloading control platform is shown in Figure 3 ,The hardware structure of the unloading control platform may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0194] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0195] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0196] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), etc., such as at least one disk memory;

[0197] The memory is used to store computer programs, and the processor is used to execute the computer programs, so that the unloading control platform can implement any of the unloading control methods mentioned above.

[0198] The embodiment of the present application also provides a discharge control system. The discharge control system described below and the discharge control scheme described above can refer to each other.

[0199] The unloading control system may include: a vehicle, plant equipment and a unloading control platform; the plant equipment includes: image acquisition devices respectively arranged at each unloading slot in the unloading area;

[0200] The unloading control platform obtains the material weight information of the mineral powder loaded on the vehicle, and calculates the unloading time according to the material weight information as the pre-calculated time;

[0201] After determining that the vehicle has parked in the parking space of the target unloading slot and has started unloading, the unloading control platform waits for the pre-calculated time to obtain the current residual material state image acquired by the image acquisition device provided at the target unloading slot; wherein the target unloading slot is the unloading slot in the unloading area that matches the material type information of the mineral powder loaded by the vehicle;

[0202] The unloading control platform calls a pre-trained residual material state detection model to process the current residual material state image to obtain the residual material remaining ratio of the vehicle; wherein the residual material state detection model is trained using residual material state sample images marked with residual material remaining ratios;

[0203] The unloading control platform determines the target number of sudden advances and sudden brakes required to make the residual materials flow out at least according to the residual material remaining ratio of the vehicle; if the determined target number is greater than zero, a residual material processing instruction is sent to the vehicle, and after the vehicle completes the sudden advance and sudden brake, a current residual material state image captured by the image acquisition device set at the target unloading hole slot is obtained, and the pre-trained residual material state detection model is called back to execute and process the current residual material state image; if the determined target number is equal to zero, a unloading restoration instruction is sent to the vehicle;

[0204] The vehicle performs the target number of rapid advances and sudden brakes in response to the residual material handling instruction.

[0205] In addition, the vehicles in the above process can be called vehicles to be unloaded.

[0206] Before obtaining the material weight information of the mineral powder loaded on the vehicle, the unloading control platform may further execute:

[0207] Obtaining unloading task information, the unloading task information including vehicle identification information of the vehicle and material type information and material weight information of the mineral powder loaded by the vehicle; the vehicle has completed the weighing at the factory, and the vehicle is connected to the factory wireless network;

[0208] According to the correspondence between the pre-configured discharge slots and the material types, a discharge slot matching the mineral powder loaded by the vehicle is determined from the discharge slots in the discharge area as a target discharge slot;

[0209] When it is determined that the target unloading slot is in an idle state, a unloading preparation instruction is sent to the vehicle, wherein the unloading preparation instruction is used to control the vehicle to drive to the parking space of the target unloading slot in an unmanned manner and park at the parking space of the target unloading slot;

[0210] After determining that the vehicle has parked in the parking space of the target unloading slot based on the unloading task information, sending an unloading instruction to the vehicle, the unloading instruction is used to control the vehicle to perform an unloading operation, and the unloading operation at least includes lifting the upper loading box to an oblique position;

[0211] The determining that the vehicle parks in a parking space of a target unloading slot and starts unloading includes: when it is determined that the vehicle starts to perform the unloading operation in response to the unloading instruction, determining that the vehicle parks in a parking space of a target unloading slot and starts unloading.

[0212] In one or more embodiments provided in the present application, the unloading control system may further include: a port loading operation management system and a steel plant unloading operation management system.

[0213] Among them, the port loading operation management system is arranged on the port side, and is used to manage the loading weight data. Optionally, the port loading operation management system may include a loading weight data management module. The steel plant unloading operation management system is arranged on the steel plant side, and the steel plant unloading operation management system is interconnected with the port loading operation management system. For example, a VPN (Virtual Private Network) secure tunnel may be opened on a dedicated network or the Internet to perform synchronous management of loading and unloading weight data. Optionally, the steel plant unloading operation management system may include a unloading weight data management module, which is used to communicate and interact with the loading weight data management module in the port loading operation management system to obtain information about the unloading tasks to be executed, such as vehicle information and corresponding loading weight information. In addition, the steel plant unloading operation management system is also interconnected with the unloading control platform to exchange information with the unloading control platform.

[0214] In one or more embodiments provided in the present application, the steel plant unloading operation management system may further include: an incoming unloading task management module, a plant equipment information management module and a unloading slot information management module.

[0215] Specifically, the factory unloading task management module can uniformly manage the tasks of a vehicle entering the factory to unload materials. The factory unloading task information recorded by the factory unloading task management module may include: order number, order status, vehicle identification information (such as license plate number, vehicle VIN code), scheduled unloading time, vehicle queuing status, material type and material weight of the mineral powder loaded on the vehicle, etc.

[0216] The factory equipment information management module can record the equipment information of each factory equipment, such as equipment ID, equipment name, setting location, basic equipment attributes, purchase time, validity period, manufacturer information, etc.

[0217] The unloading slot information management module can record the slot information of each unloading slot, such as the slot number, position, type of material received, etc.

[0218] On the basis of the above, the unloading control platform can collect the status data of the connected vehicles in real time through wireless networks (such as 5G wireless networks) and issue control instructions to the connected vehicles; the platform can also interact with factory equipment, specifically including: collecting data from various factory equipment, such as data from image acquisition devices, and issuing control instructions to factory equipment.

[0219] Taking the unloading control platform deployed in the cloud and controlling the vehicle in an autonomous driving mode as an example, the unloading control platform can include a control center, a V2N (Vehicle to Network) service module, a MEC (Mobile Edge Computing) capability service module, a scheduling service module, a real-time acquisition monitoring module, and an intelligent detection and analysis module. The following is an example of each module.

[0220] For the control center, the control center can be used to interact with other modules to implement the unloading control solution provided in this application. The control center can request the steel plant unloading operation management system to obtain unloading task information and slot information, and determine the corresponding relationship between the unloading slot and the material type.

[0221] For the V2N service module, it should be noted that the vehicle and the unloading control platform can interact with each other through the 5G-CV2X (Cellular Vehicle-To-Everything, a technology that uses cellular networks to connect vehicles to everything, CV2X) wireless network. Based on this, the V2N service module can be used to send instructions to the vehicle in response to the dispatch of the control center or other modules, such as: sending a remote start instruction to a vehicle in a parking state; sending a driving mode switching instruction to the vehicle, which may specifically include an entry / exit instruction for an unmanned driving mode or a manual takeover mode; sending control instructions to the vehicle, such as unloading preparation instructions, unloading instructions, and residual material processing instructions. Correspondingly, the V2N service module can also be used to receive response feedback sent by the vehicle in response to the platform instruction, such as a vehicle status completion instruction or an operation completion instruction, and forward the received completion instruction to the control center for the control center to execute subsequent steps.

[0222] The MEC capability service module can specifically include a map module and a positioning module. After the vehicle is successfully connected to the Internet, the control center can call the MEC capability service module to obtain a high-precision factory map and send the factory map to the vehicle to support the vehicle's automatic driving. The positioning module mainly involves GNSS (Global Navigation Satellite System, Global Navigation Satellite System, GNSS) differential RTK (Real-Time Kinematic, carrier phase differential technology, RTK), which can be composed of four parts: GNSS system (or BD satellite system), reference base station, mobile station (ie vehicle) and communication equipment. On this basis, a GNSS receiver can be first set up as a reference base station at a known point with high precision and reliability. The reference base station has a wide field of view and high observation conditions. Then, a vehicle is connected to the point to be measured, and the reference base station and the vehicle observe the BD satellite at the same time. After capturing the satellite signal, the receiver of the reference base station solves its own position, compares the solution result with the known coordinates, calculates the error value, and then reversely solves the satellite positioning error based on the error value of the coordinates. Since the GNSS receiver of the reference base station cannot know the number of satellites that the vehicle receiver can receive, the reference base station receiver will lock all satellites in the field of view, calculate the positioning error of each satellite, and compile it into a telegram in a standard format, which is sent to the vehicle via a communication link, so that the vehicle can respond to the received telegram and make error corrections based on the satellites it observes, thereby obtaining accurate positioning results.

[0223] As for the dispatch service module, the module may include: a vehicle unloading command and dispatch module. The vehicle unloading command and dispatch module may dispatch the networked vehicles in the factory area in a certain order. For example, it may determine the need to issue corresponding dispatch instructions to the vehicle based on factors such as the order of the vehicles entering the factory, the types of materials loaded on the vehicles, and the status of the unloading slots corresponding to the material types, so as to instruct the vehicle to enter the unloading area for unloading or instruct the vehicle to wait in the parking lot.

[0224] Optionally, the dispatch service module of the unloading control platform may also include a parking lot dispatch module, which is used to determine the target empty parking space for the vehicle to park and wait from the empty parking spaces in the parking lot in response to the empty parking space demand of the control center. Based on this, when the control center determines that the vehicle needs to go to the parking lot to wait for unloading, it can call the parking lot dispatch module to determine the target empty parking space, and then call the V2N service module to send the location information of the target empty parking space to the vehicle, so that the vehicle can plan the path with the help of the factory map obtained in advance or stored locally, and automatically drive to the target empty parking space to park and turn off the engine to wait.

[0225] For the real-time acquisition monitoring module, the module may include: a video monitoring module for receiving the environmental monitoring video stream collected in real time by each image acquisition device (such as a camera) set in the factory area. Exemplarily, the environmental monitoring video stream collected by the image acquisition device can be sent to the unloading control platform through a large-bandwidth optical fiber local area network so that the platform can perform unified monitoring and intelligent detection and analysis. In addition, the RTP+RTCP protocol can be used when transmitting real-time video streams, wherein the RTP protocol (Real-Time Transport Protocol, Real-time Transport Protocol, RTP) is based on the User Datagram Protocol (User Datagram Protocol, UDP) and is encapsulated and transmitted on the Internet Protocol (Internet Protocol, IP). RTP utilizes the connectionless characteristics of UDP, so it does not need to establish and maintain complex connection states to provide streaming media efficient real-time transmission, delay control, fault tolerance mechanism and other functions; RTCP, Real-Time Control Protocol, a protocol for controlling and monitoring in a real-time transport protocol RTP session, can provide quality feedback, flow control and information synchronization for the RTP session, and provide a basis for ensuring the smooth progress of multimedia sessions. Optionally, the image acquisition device may be a 2.4 GHz / 5.8 GHz wireless camera, based on which the video monitoring module may receive an environment monitoring video stream via a wireless network.

[0226] As for the intelligent detection and analysis module, the module may include: a discharge state detection and analysis module, which is used to monitor and analyze the residual material state of the vehicle during discharge in response to the call of the control center through a neural network model (i.e., residual material state detection model) to detect whether the remaining residual material has flowed out of the vehicle box; and to feed back the detection results, i.e., the residual material remaining proportion, to the control center for the control center to determine the target number of times. Based on this, the control center can determine the target number of times, and execute the residual material processing process when the target number of times is not zero, i.e., call the V2N service module to send a residual material processing instruction to the vehicle, and after determining that the vehicle has completed the sudden advance and sudden braking operation, call the discharge state detection and analysis module to perform residual material state detection again; execute the discharge restoration process when the target number of times is zero, i.e., call the scheduling service module to send a discharge restoration instruction to the vehicle, and then determine that the vehicle has completed the discharge restoration operation after determining that the vehicle has completed the discharge restoration operation.

[0227] On the basis of the above, the intelligent detection and analysis module can also include an empty parking space detection and recognition module, which is used to respond to the call of the parking lot scheduling module, detect and recognize the parking lot image through the neural network model, so as to realize the real-time detection and recognition task of empty parking spaces; and feedback the detection results, i.e., empty parking space information, to the parking lot scheduling module. Based on this, the target empty parking space information in the empty parking space information is finally sent to the unloading vehicle that needs to park and wait, so that the vehicle can perform path planning and unmanned parking.

[0228] In one or more embodiments provided in the present application, the unloading control system may also include: a remote security takeover system that communicates and interacts with the unloading control platform through a local area network, a CA security certification center (Certificate Authority, CA) and a vehicle fault diagnosis and prediction system.

[0229] It should be noted that, under normal circumstances, after the vehicle enters the factory and connects to the Internet, it will adopt unmanned driving and unmanned unloading operations. On this basis, the vehicle fault diagnosis and prediction system can be used to: receive the fault code reported by the vehicle in the unmanned driving mode when it learns the fault information of the vehicle itself, and conduct a re-inspection of the fault information; after determining the fault, implement remote automatic fault elimination; if the fault cannot be eliminated, send a fault notification to the control center of the unloading control platform so that the unloading control platform can execute the fault handling process.

[0230] Optionally, the vehicle can also report its operating status and fault information to the unloading control platform in real time, so that the intelligent detection and analysis module of the unloading control platform can perform remote fault monitoring and diagnosis prediction based on the received vehicle data to improve the accuracy and efficiency of fault diagnosis.

[0231] Optionally, the real-time acquisition monitoring module may also include: a vehicle data monitoring module, which is used to receive vehicle data reported by the vehicle regularly (such as every 100 milliseconds) through the 5G wireless network and the message queue telemetry transmission protocol MQTT (Message Queuing Telemetry Transport, MQTT) protocol, so that the unloading control platform can monitor the vehicle in real time. Optionally, in order to ensure network security, the MQTT used can be supplemented with the transport layer security protocol TLS (Transport Layer Security, TLS) service to provide encryption, identity authentication and data integrity protection, wherein the TLS certificate can be uniformly issued by a trusted CA security certification center. Exemplarily, the vehicle data reported by the vehicle may include: basic vehicle information, such as vehicle model, vehicle identification code VIN, engine model, vehicle production date, etc.; vehicle driving data, such as vehicle speed, steering angle, acceleration and deceleration, mileage, power consumption, etc.; vehicle location data, such as the real-time location latitude and longitude coordinates of the vehicle, driving trajectory, etc.; vehicle status data, such as engine status, brake status, door status, etc.; vehicle environment data, such as vehicle external temperature, humidity, air pressure, etc.

[0232] CA certification refers to a third-party organization that uses the public key infrastructure (PKI) public key infrastructure technology to provide network identity authentication services and is responsible for issuing and managing digital certificates. The CA center is responsible for the approval, issuance, archiving, and revocation of digital certificates. The digital certificates issued by the CA have the digital signature of the CA, and other organizations cannot make unnoticed changes. Based on this, the two network element nodes must obtain each other's digital certificates for mutual security verification to ensure the security of mutual communication. Based on this, the CA security certification center can be used in steel plant service systems, such as unloading control platforms, remote security takeover systems, vehicle fault diagnosis and prediction systems, etc., to generate valid digital certificates, and the CA center will digitally sign them. When a terminal, such as a vehicle, plant equipment, client monitoring system, etc., initiates a connection request, the server will send its own certificate to the terminal for verification. The terminal verifies the server certificate to confirm whether the server's identity is legal and credible. If the certificate is invalid or the verification fails, the terminal will receive a warning message and refuse to connect to the server.

[0233] The remote safety takeover system can include an industrial computer system and a parallel driving kit, including multiple large screens for first-person video surveillance, small screens for digital twin tracking areas, web displays for the steel mill unloading operation management system, all-in-one touch screens, real-time maps, simulated cockpits / steering wheels / gear positions / accelerator pedals / brake pedals, etc. In addition, the unloading control system can include multiple remote safety takeover systems to meet the task of remotely taking over multiple vehicles at the same time.

[0234] On the basis of the above, the dispatching service module of the unloading control platform may also include: a vehicle remote takeover dispatching module.

[0235] It should be noted that, under normal circumstances, after the vehicle enters the factory and is connected to the network, it will adopt an unmanned driving and unmanned unloading operation mode. If a vehicle failure is determined, the unloading control platform will issue a command to exit the unmanned driving mode, coordinate the vehicle remote takeover scheduling module and the scheduling remote safety takeover system, and remotely take over the vehicle manually.

[0236] In one or more embodiments provided in the present application, the steel plant unloading operation management system may further include: a plant equipment status monitoring module, which is used to receive heartbeat packets regularly reported by each plant equipment to implement the heartbeat detection task, thereby providing a basis for checking and judging whether the working status of the plant equipment is normal. Optionally, the plant equipment status monitoring module can output an early warning signal to prompt the staff to replace the equipment when determining that the working status of any plant equipment is abnormal, thereby improving the reliability of unloading control.

[0237] In one or more embodiments provided in the present application, the factory equipment may further include: an image acquisition device and a barrier pole arranged at the factory entrance.

[0238] Correspondingly, the unloading control platform may also include a plant equipment IO (Input-Output) control module, which may include: a barrier pole lifting and lowering control module. The intelligent detection and analysis module may also include: a license plate number recognition module.

[0239] On the basis of the above, the control center can obtain the image of the incoming vehicle captured by the image acquisition device set at the entrance of the plant, and call the license plate number recognition module; the license plate number recognition module responds to the call of the control center, and uses the pre-trained license plate number recognition model to process the image of the incoming vehicle, obtains the license plate number recognition result of the incoming vehicle and feeds back to the control center; then the control center obtains the scheduled in-plant unloading task information, that is, the appointment list, from the steel plant unloading operation management system, and searches for the appointment record matching the license plate number recognition result from the appointment list; if the search is successful, the gate pole lifting and lowering control module is called to issue instructions to the gate pole set at the entrance of the plant, lift the gate pole set at the entrance of the plant, so that the incoming vehicle can enter the plant, and determine that the vehicle is a fully loaded vehicle to be weighed.

[0240] In one or more embodiments provided in the present application, the factory equipment may further include: an image acquisition device and a barrier rod arranged at the entrance of the weighing platform.

[0241] On the basis of the above, the control center can obtain the image acquisition device set at the entrance of the weighing platform to collect the vehicle image of the fully loaded vehicle to be weighed, and call the license plate number recognition module; the license plate number recognition module responds to the call of the control center, and uses the pre-trained license plate number recognition model to process the vehicle image, obtains the license plate number recognition result of the vehicle requesting to enter the weighing platform and feeds back to the control center; then, after the verification is passed, the control center calls the barrier pole lifting and lowering control module to issue instructions to the barrier pole set at the entrance of the weighing platform, and lifts the barrier pole set at the entrance of the weighing platform to allow the fully loaded vehicle to enter the weighing platform for weighing.

[0242] On the basis of the above, the factory equipment may also include: dynamic weighing sensor; the real-time acquisition monitoring module may also include a weighing data acquisition and calculation module; based on this, the control center may call the weighing data acquisition and calculation module to collect data from the dynamic weighing sensor as the weight value of the vehicle to be weighed when it is determined that the vehicle has driven to the weighing platform.

[0243] In one or more embodiments provided in the present application, the factory equipment may also include: a sprinkler, an image acquisition device and a barrier pole arranged at the car wash station.

[0244] Correspondingly, the plant equipment IO control module may also include: a sprinkler control module.

[0245] On this basis, after determining that the unloading task of the vehicle is completed and that the vehicle to be unloaded is a vehicle to be washed, the control center can call the V2N service module to send a car wash preparation instruction to the vehicle, which may specifically include the location information of the car wash station; the vehicle responds to the car wash preparation instruction, automatically drives to the entrance of the car wash station and stops at the entrance of the car wash station; the control center obtains the vehicle image captured by the image acquisition device set at the entrance of the car wash station, and calls the license plate number recognition module; the license plate number recognition module responds to the call of the control center, processes the vehicle image using the pre-trained license plate number recognition model, and obtains the license plate number of the vehicle requesting to enter the car wash station The control center receives the number recognition result and feeds back the result to the control center; after the verification is passed, the control center calls the barrier pole lifting and lowering control module to issue instructions to the barrier pole set at the entrance of the car wash station, and lifts the barrier pole set at the entrance of the car wash station to allow the vehicle to enter the car wash station for washing; after determining that the vehicle to be washed has stopped at the car wash station, the control center calls the sprinkler control module to send a water spraying control instruction to the sprinkler set at the car wash station; the sprinkler responds to the water spraying control instruction and performs a water spraying operation for a first preset time; after determining that the sprinkler has completed the water spraying operation, the control center determines that the car washing task of the vehicle to be washed is completed.

[0246] In one or more embodiments provided in the present application, the factory equipment may further include: a vehicle blower, an image acquisition device and a barrier pole arranged at the blowing station.

[0247] Correspondingly, the plant equipment IO control module may also include: a vehicle blower control module.

[0248] On this basis, the control center can call the V2N service module to send a blowing preparation instruction to the vehicle after determining that the car washing task of the vehicle to be washed is completed and that the vehicle to be washed is a vehicle to be blown; the vehicle responds to the blowing preparation instruction, automatically drives to the entrance of the blowing station and stops at the entrance of the blowing station; the control center obtains the vehicle image captured by the image acquisition device set at the entrance of the blowing station, and calls the license plate number recognition module; the license plate number recognition module responds to the call of the control center, processes the vehicle image using the pre-trained license plate number recognition model, obtains the license plate number recognition result of the vehicle requesting to enter the blowing station, and reports it to the control center feedback; then, after the verification is passed, the control center calls the barrier pole lifting and lowering control module to issue instructions to the barrier pole set at the entrance of the blowing station, and lifts the barrier pole set at the entrance of the blowing station to allow the vehicle to be blown to enter the blowing station for blowing; after determining that the vehicle to be blown has stopped at the blowing station, the control center calls the vehicle blower control module to send a blowing control instruction to the vehicle blower set at the blowing station; the vehicle blower responds to the blowing control instruction and performs a blowing operation for a second preset time; after determining that the vehicle blower has completed the blowing operation, the control center determines that the blowing task of the vehicle to be blown is completed.

[0249] After determining that the blowing task of the vehicle to be blown is completed, the control center can determine that the vehicle to be blown is an empty vehicle to be weighed. The weighing process of the empty vehicle to be weighed can refer to the weighing process of the fully loaded vehicle to be weighed. Based on this, the control center can call the weighing data acquisition and calculation module, and based on the collected empty weight value of the weighed empty vehicle and the full weight value corresponding to the weighed empty vehicle, calculate the difference between the full weight value and the empty weight value as the actual material weight value; then the control center feeds back the actual unloading weight data to the steel plant unloading operation management system; and when the degree of matching between the actual material weight value and the material weight information is not within the preset limit, output an alarm message, and the alarm message is used to characterize the abnormal weight of the mineral powder loaded on the vehicle; when the degree of matching between the actual material weight value and the material weight information is within the preset limit, determine that the weighed empty vehicle is a vehicle to be shipped, call the V2N service module to send a factory preparation instruction to the vehicle to be shipped, and output the task completion information to the driver of the vehicle to be shipped, so as to instruct the driver to take over the vehicle to be shipped at the preset handover position. In response to the factory preparation instruction, the vehicle to be shipped automatically drives to the handover position and stops at the handover position.

[0250] In one or more embodiments provided in the present application, the factory equipment may further include: an image acquisition device and a barrier pole arranged at the factory exit.

[0251] On the basis of the above, the control center can obtain the image of the outgoing vehicle collected by the image acquisition device installed at the factory exit, and call the license plate number recognition module; the license plate number recognition module responds to the call of the control center, and uses the pre-trained license plate number recognition model to process the image of the outgoing vehicle, obtains the license plate number recognition result of the outgoing vehicle and feeds back to the control center; then, after the verification is passed, the control center calls the gate pole lifting and lowering control module to issue instructions to the gate pole installed at the factory exit, lift the gate pole installed at the factory exit, so that the vehicle to be shipped can leave the factory, and determine that the vehicle has completed the task of entering the factory to unload.

[0252] Based on the above, Figure 4 An example of a structural diagram of a discharge control system is shown in Figure 1. Figure 4 As shown, the unloading control system may include: unloading control platform, plant equipment, steel plant unloading operation management system, remote safety takeover system, CA safety certification center, vehicle fault diagnosis and prediction system, port loading operation management system and vehicles.

[0253] The unloading control platform may include a control center, a V2N service module, an MEC capability service module, a dispatching service module, a real-time acquisition monitoring module, an intelligent detection and analysis module, and a plant equipment IO control module. Among them, the MEC capability service module may include a map module and a positioning module, the dispatching service module may include a vehicle unloading command and dispatch module, a parking lot parking dispatch module, and a vehicle remote takeover dispatch module, the real-time acquisition monitoring module may include a video monitoring module, a vehicle data monitoring module, and a weighing data acquisition and calculation module, the plant equipment IO control module may include a gate pole lifting and lowering control module, a sprinkler control module, and a vehicle blower control module, and the intelligent detection and analysis module may include an unloading state detection and analysis module, an empty parking space detection and recognition module, and a license plate number recognition module.

[0254] The factory equipment may include: image acquisition devices, i.e., cameras, installed at the factory entrance, factory exit, parking lot, unloading area, weighing platform, car washing station and hair drying station, barrier poles installed at the factory entrance, factory exit, weighing platform, car washing station and hair drying station, dynamic weighing sensors installed at the weighing platform, water sprinklers installed at the car washing station, and vehicle hair dryers installed at the hair drying station.

[0255] The unloading control platform interacts with the vehicle through a wireless network; the unloading control platform obtains data from factory equipment such as cameras and dynamic weighing sensors, and issues control instructions to factory equipment such as gate poles, sprinklers, and vehicle blowers. In addition, the unloading control platform interacts with the steel plant unloading operation management system, remote security takeover system, CA certification center, and vehicle fault diagnosis and prediction system through local area network communication; the steel plant unloading operation management system interacts with the port loading operation management system through a dedicated network or VPN.

[0256] The vehicle described in this application can be called an unmanned tractor for unloading iron ore powder. After the vehicle enters the factory and is connected to the network, it can perform unmanned and unmanned unloading operations according to the specific factory traffic environment and unloading operation scenario ODD (Operational Design Domain). Exemplarily, after the vehicle enters the factory and is connected to the network, it can receive V2N service instructions from the platform, download high-precision maps, obtain centimeter-level positioning and correction services on the cloud, accept unloading, remote takeover, parking lot and other scheduling services, report the vehicle's operating status information and fault diagnosis information in real time, and allow the remote parallel driving platform to perform safe manual takeover.

[0257] Based on the above, Figure 5 The EE architecture (Electronic and Electrical Architecture, EE architecture, electronic and electrical architecture) diagram of the unmanned tractor for iron ore powder unloading is shown in the figure. Figure 5 As shown in the figure, the architecture takes into account multiple redundant technologies such as unmanned driving domain controllers, multimodal sensors, high-bandwidth vehicle networks, chassis wire-controlled steering, chassis wire-controlled braking, high-precision fusion positioning modules, and takes into account functional safety degradation and network security protection strategies.

[0258] Combination Figure 5 As shown in the figure, the architecture may involve intelligent central computing gateway, multimodal sensors, unmanned driving domain control, high-precision fusion positioning, chassis wire-controlled braking system, chassis wire-controlled steering system, VCU (Vehicle Control Unit, VCU) and power system, body domain controller and unloading operation domain controller. Each part is described in turn below.

[0259] For the intelligent central computing gateway: To reduce costs, the traditional vehicle gateway and the TBOX (Telematics BOX, communication box, Tbox) supporting 5G-CV2X can be integrated to provide vehicle-side Gigabit Ethernet, CAN (Controller Area Network, CAN), CANFD (CAN with Flexible Data-Rate, CANFD) and other vehicle bus communication interfaces for other related systems on the vehicle side. The intelligent central computing gateway encapsulates the device drivers and software protocol stacks of Gigabit Ethernet, CAN, CANFD, and 5G-CV2X, and realizes the transparent transmission and forwarding of uplink and downlink instructions of vehicle-cloud communication, diagnostic routing, multi-protocol analysis and conversion, firewall and intrusion detection defense, network status management, security key authentication, OTA (Over The Air Technology, OTA) management and other functions.

[0260] For multimodal sensors, since the trailer of the iron ore powder unloading tractor is a movable rigid body connected to the front of the vehicle, and the vehicle box needs to be raised and lowered during the unloading process, it is impossible to install and deploy unmanned driving sensors on both sides of the vehicle box. Based on this, the sensors are deployed at the head and tail of the vehicle. Exemplarily, it can be set up according to the 3L-5R-13V-8U scheme, specifically, 1 forward semi-solid 128-wire Lidar (L) in front of the front of the vehicle, 1 side rear-facing full-solid 64-wire Lidar on the left and right sides of the front of the vehicle; 1 forward three-transmitter and four-receiver Radar (R) in front of the front of the vehicle, and 1 three-transmitter and four-receiver Radar on the left front corner / right front corner / left rear corner / right rear corner of the vehicle; 1 forward wide-view 8MP camera (V) + 1 forward long-view 8MP camera in front of the front of the vehicle; 2 side-view 2MP cameras on the left side of the vehicle + 2 side-view 2MP cameras on the right side of the vehicle; 1 rear-facing 2MP camera at the rear; 1 remote driving side rear-view 2MP camera on each side of the vehicle; 3 270° AVM (Around View Monitoring, AVM) surround-view 3MP fisheye cameras at the front, and 1 AHD (Analog High Definition, analog high definition, AHD) rear surround view 2MP camera; 4 ultrasonic radars UPA (Ultrasonic Parking Assistance, UPA) at the front and rear of the car.

[0261] For unmanned driving domain control, dual redundant configuration of domain control hardware is carried out, with built-in high-precision map engine and data, and functional safety degradation strategies (such as dual network redundancy, intelligent gateway downtime or main network abnormality, and the use of private CAN network to control chassis safe parking) and network security protection strategies are considered. A combination of high-computing power SoC (System on Chip, SoC) + MCU (Microcontroller Unit, MCU) chips is configured, and automotive-grade unmanned driving OS (Operating System, OS), middleware, service framework and functional application layer are developed. In order to achieve efficient and timely processing of video streams, images and point cloud big data, Gigabit Ethernet interfaces can be supported.

[0262] For high-precision fusion positioning, a dual-redundancy configuration of high-precision fusion positioning modules is performed, and a built-in heartbeat detection mechanism is used to achieve GNSS+RTK+INS (Inertial Navigation System, INS) combined navigation and high-precision positioning.

[0263] For the chassis brake-by-wire system, a dual-redundancy configuration of the brake-by-wire system EMB (Electronic Mechanical Brake) is performed, and a heartbeat detection mechanism is built in. Two CAN interfaces are provided to the outside world, one connected to the chassis domain CAN and the other connected to the unmanned driving domain control private CAN.

[0264] For the chassis steer-by-wire system, a dual-redundancy configuration of the EPS (Electric Power Steering) is performed, and a heartbeat detection mechanism is built in. Two CAN interfaces are connected to the outside, one to the chassis domain CAN and the other to the unmanned driving domain control private CAN.

[0265] For the VCU and power system, the VCU is connected to two CAN interfaces, one connected to the chassis domain CAN and the other connected to the power domain CAN. A series of power domain systems are also mounted on the power domain CAN, such as the motor controller, high-voltage battery management module, high-voltage power distribution unit, transmission control unit, DC-DC converter DCDC, etc.

[0266] For the body domain controller, it is connected to the intelligent central computing gateway through the body domain CAN.

[0267] The unloading operation domain controller may specifically include a rainproof cloth motor controller and a vehicle box lifting motor controller, both of which can be connected to the intelligent central computing gateway through the same unloading operation domain CAN.

[0268] Figure 6The communication architecture diagram of the video surveillance data in the unloading control system is shown in the figure. Figure 6 As shown, the communication architecture may involve: roadside unit cameras and edge computing units, cloud-controlled video streaming service system, video content analysis service system, and large-screen visual monitoring system.

[0269] Among them, the cloud-controlled video stream service system sends control streams to the roadside unit cameras and edge computing units, and the roadside unit cameras and edge computing units respond to the video stream control feedback to the cloud-controlled video stream service system; the cloud-controlled video stream service system sends original video frames to the video content analysis service system, and the video content analysis service system performs AI (Artificial Intelligence, artificial intelligence, AI) real-time analysis and detection, combines the detection results with the original video frames, and returns the combined and processed video frames to the cloud-controlled video stream service system; the large-screen visual monitoring system sends control streams to the cloud-controlled video stream service system, and the cloud-controlled video stream service system responds to the video stream control feedback to the large-screen visual monitoring system; the large-screen visual monitoring system performs real-time rendering and playback of video streams.

[0270] The following describes each part in turn.

[0271] The roadside unit camera and edge computing unit can adopt an embedded system layered architecture, whose underlying software dependencies include OpenVX, VXLib, VPacket, and Serialize / Deserialize.

[0272] Among them, OpenVX is an open cross-platform acceleration standard for computer vision applications, which has the characteristics of high efficiency, scalability, cross-platform, standardization and customizability, and supports performance and power consumption optimized computer vision processing. OpenVX provides an open API (Application Programming Interface, API) that allows users to develop and deploy computer vision applications on various platforms. OpenVX supports various operating systems and processor architectures, including ARM (Advanced RISC Machines, microprocessors using ARM technology intellectual property IP cores, ARM), MIPS (Million Instructions Per Second, millions of machine language instructions processed per second, MIPS), PowerPC (Performance Optimized With Enhanced RISC Personal Computer, Enhanced RISC Personal Computer Optimized Performance, PowerPC), etc.

[0273] VXLib is an open source cross-platform acceleration standard for computer vision applications, designed to achieve performance and power optimized computer vision processing, especially in embedded and real-time application cases. VXLib can be seen as an embedded version of OpenCV. VXLib provides some standard APIs for image processing, allowing third-party vendors to use hardware to implement image processing functions, thereby improving efficiency. VXLib is designed with an emphasis on optimizing performance and power consumption, making it particularly suitable for resource-constrained environments, such as embedded systems and real-time applications.

[0274] VPacket is used to store compressed video data, which contains rich parameters and data information, such as the memory area for storing media data, a pointer to media data, the length of media data, the display timestamp of the data packet, the decoding timestamp of the data packet, the duration of the data packet, the index number of the media data stream, the encoding and decoding characteristics for specific encoders and formats, non-media ancillary data, the number of ancillary data, the properties of the data packet (such as the content of the data packet is damaged / needs to be discarded, etc.), the position of the data packet in the video stream, private data used by the user, and the time base used by the data packet.

[0275] Serialize, or serialization of video streams, refers to converting video data into a format that can be stored, transmitted, or shared. This process involves converting video frames into binary strings or other formats for transmission over the network or local storage. Deserialize, or deserialization, is the reverse process of serialization, which is to convert the binary string back to the original data structure or object. For video streams, this means that the serialized data received from the storage medium or network can be restored to the original video frame sequence for correct display in the player.

[0276] The protocols of the roadside unit camera and edge computing unit rely on MQTT, RTCP, RTP, and UDP.

[0277] MQTT is a machine-to-machine / IoT connection protocol. It is an extremely lightweight publish / subscribe message transmission protocol suitable for remote connection scenarios that require a small code footprint and / or where network bandwidth is very valuable. It is designed for constrained devices and low-bandwidth, high-latency or unreliable networks. For real-time video streaming between surveillance cameras and video streaming servers, and requiring high performance and quality, the RTP / RTCP dedicated video transmission protocol will be used. It is used in conjunction with the MQTT protocol at the application layer to transmit some additional control messages or metadata, such as device status, alarm information, etc.

[0278] RTCP is a real-time transport control protocol used to manage transmission quality and exchange control information between current application processes. During an RTP session, each participant periodically transmits RTCP packets, which contain statistics such as the number of packets sent and the number of packets lost. Based on this, the server can use the above information to dynamically change the transmission rate and even the payload type. The main functions of RTCP include: using feedback information to provide the transmission quality of distributed data; providing a permanent CNAME transport layer mark for the RTP source; adjusting the sending rate of RTCP packets according to the number of participants; and transmitting session control information.

[0279] RTP is a transmission protocol for multimedia data streams on the Internet. RTP is defined to work in one-to-one or one-to-many transmission situations. Its purpose is to provide time information and achieve stream synchronization. RTP itself can only guarantee the transmission of real-time data, and cannot provide a reliable transmission mechanism for sequential transmission of data packets, nor does it provide flow control or congestion control. It relies on RTCP to provide these services. The main functions of RTP include: adding timestamps to real-time IP packets in order; serializing real-time IP packets in order; monitoring and management. RTP includes data and control parts. The latter is called RTCP (a real-time control protocol that manages transmission quality and provides QoS information, mainly monitoring delay and bandwidth. Once the bandwidth of the transmitted multimedia information stream changes, the sender is notified to change the symbol identification code and encoding parameters); the header of each RTP packet contains a payload type identifier, including static types (such as encoding identifiers) and dynamic types (dynamically negotiated and determined by the control protocol).

[0280] UDP is a connectionless transport layer protocol that provides a simple, unreliable data transmission service. It is suitable for transmitting a small amount of data at a time, but not for transmitting a large amount of data. The UDP protocol does not have a confirmation retransmission mechanism. If the message cannot be sent to the other party due to a network failure, or the other party receives the message but it is out of order during the transmission process, and the other party fails to verify and loses the out-of-order packet, the UDP protocol will not give any error feedback information to the application layer. When UDP transmits data, it sends out one by one in datagrams, and then receives them one by one, which makes it impossible for the upper application layer to flexibly control the number and quantity of data read and write. The RTP / RTCP used in this application is also based on UDP at the bottom layer.

[0281] The application layer of the roadside unit camera and edge computing unit includes video stream acquisition, RTP / RTCP real-time feedback, control signaling synchronization, H.264 / H.265 compression encoding, and video variable sampling residual processing.

[0282] Among them, video stream acquisition is a special type of data acquisition method, which samples and quantizes the video signal output by the surveillance camera to convert it into digital data. Important components include data collection equipment, data transmission equipment, data collection and sorting equipment, etc. The video stream data acquisition method considers a dedicated image acquisition chip to automatically complete image acquisition, frame memory address generation, and image data refresh. In addition to setting the acquisition mode, the main processor does not participate in the acquisition process. It has the advantages of video stream acquisition not taking up CPU time, good real-time performance, and adaptability to the acquisition of moving images, but the circuit is more complex and the cost is higher.

[0283] For RTCP real-time feedback, the Real-time Transport Control Protocol is a standard protocol for sending control packets between the sender and receiver of a data stream, which is used to provide feedback during data transmission, such as packet loss and network congestion, so that the sender can adjust its data transmission strategy. RTCP real-time feedback usually involves the following steps: At the beginning of the data stream, the sender and the receiver negotiate and establish an RTP session, and initialize the RTCP protocol instance; the sender starts to send SR (Sender Report) regularly, which contains information such as timestamp and sequence number; after receiving the RTP data packet, the receiver calculates the packet loss rate and sends RR (Receiver Report) regularly, which contains its own SSRC (Synchronization Source) and corresponding SR information; the sender and the receiver exchange information through RTCP messages to provide real-time feedback on the network status.

[0284] Video control signaling synchronization mainly involves the transmission and management of signaling, ensuring that control instructions during video transmission can be accurately and timely received and executed. Control information can include the start and end of the video stream, encoding format, transmission rate adjustment, etc., to provide a basis for ensuring video quality and improving transmission efficiency. Signaling synchronization evaluates the network status and receiving end status through a real-time feedback mechanism to adjust the transmission rate and even the payload type in a timely manner to solve the delay or jitter problems that may occur in real-time data transmission. This can be achieved through feedback mechanisms such as RTCP to ensure the synchronization and quality of video transmission.

[0285] H.264 is a high-performance video codec technology. Its biggest advantage is its high data compression ratio. Under the condition of equal image quality, the compression ratio of H.264 is more than twice that of MPEG-2 (Moving Pictures Experts Group, MPEG), and 1.5~2 times that of MPEG-4. H.264 divides the system framework into two layers, namely the video coding layer VCL and the network abstraction layer NAL. VCL is responsible for effectively representing the video data, while NAL is responsible for encapsulating the data representing the video in an abstract sense. The video compression technology of H.264 includes: intra-frame prediction, inter-frame prediction, discrete cosine transform DCT, quantization, and entropy coding.

[0286] H.265 (HEVC, High Efficiency Video Coding) is a video compression standard designed to transmit high-quality video with higher compression efficiency. It is one of the potential successors of H.264, providing about 2 times the data compression ratio, or significantly improving video quality at the same bit rate. The design goal of H.265 is to reduce the required storage space and transmission bandwidth while maintaining video quality. The main features of H.265 include: tree-structured prediction and residual block segmentation, extended prediction block size, flexible block transform, integer inverse transform, intra-frame prediction and prediction type based on 33 directions, adaptive motion vector prediction selection mechanism, Merge mode, and Skip mode based on Merge mode.

[0287] Video variable sampling residual processing mainly involves video compression sensing technology, which realizes the simultaneous sampling and compression of signals, independent observation of each video frame, and joint reconstruction. The prediction-residual reconstruction mode has the best reconstruction performance. Multi-hypothesis prediction residual reconstruction based on the pixel domain is performed again on the basis of the reconstructed video frame, which can effectively weaken the block effect and improve the reconstruction quality of the video. In addition, considering the relatively violent motion of some video sequences, an inter-frame / intra-frame adaptive multi-hypothesis prediction mode and a regional fast search scheme are proposed.

[0288] The cloud-controlled video streaming service system can adopt a server-side video system layered architecture, and its underlying software dependencies include Nginx, Janus, Kubernates, and MQ (Message Queue).

[0289] Nginx is a high-performance HTTP (Hypertext Transfer Protocol) and reverse proxy web server. Nginx is designed with efficiency in mind and features low memory usage, extremely fast startup, and high concurrency. This application can use Nginx to build a video streaming server cluster.

[0290] Janus is a client-side load balancing library designed specifically for Java developers. It provides full support for the HTTP protocol (including synchronous and asynchronous modes) and WebSocket (WebSocket, a protocol for full-duplex communication on a single TCP connection). Janus provides an open source web server for real-time video streaming and provides services in the form of Restful API. Restful API is a design style of HTTP that enables developers to operate resources through HTTP. The main goal of Janus is to help developers effectively manage client connections in distributed systems, improve system availability and performance, and is suitable for Java applications that handle a large number of concurrent requests and need to run in a multi-service environment. The core functions and features of Janus include: service discovery, HTTP and WebSocket support, multiple load balancing strategies, custom extensions, fault tolerance mechanisms, performance optimization, easy integration, etc.

[0291] Kubernates, or K8S for short, is an open source container orchestration platform designed for automated deployment, expansion, and management of containerized applications. The main functions of Kubernates include: service discovery and load balancing, self-healing, elastic scaling, automatic deployment and rollback, etc. These functions together ensure the reliable operation and management of containerized applications. The core concepts of Kubernetes include Pod (the smallest deployment unit), Service (providing a stable access entry), Namespace (resource isolation), Deployment (defining the deployment and update strategy of Pod), etc. Its architectural design includes Master nodes and Node nodes, where the Master node is responsible for the scheduling, management, and operation of the cluster, while the Node node is responsible for running containers. The characteristics of Kubernates include portability, scalability, automation, and high availability. It supports public cloud, private cloud, hybrid cloud, and multi-cloud environments, and is easy to expand and integrate with other systems through modular and plug-in design.

[0292] MQ message queues (such as RabbitMQ, Kafka, etc.) are important tools for asynchronous task processing and data transmission in video live broadcast systems. Their main uses include, for example, storing information about video transcoding completion events; before the introduction of message queues, the video encoding module must wait for the output of the video upload module. After the introduction of message queues, the video encoding module no longer needs to wait for the output of the video upload module. If there are events in the message queue, the video encoding module can execute these tasks in parallel. The main role of message queues in the communication architecture of steel plant unloading video monitoring is application decoupling, asynchronous communication, current limiting and peak shaving, etc. The characteristics of message queues include message orderliness, duplicate message processing, reliability, etc. Message queue modes include point-to-point mode and publish / subscribe mode. In point-to-point mode, the message queue middleware can store messages produced by multiple producers. When a message is successfully consumed by a consumer, the message will be removed to achieve the purpose of non-repeatable messages. If a consumer fails to process a message, the message will be consumed again. In the publish / subscribe mode, only consumers who have completed registration and subscription can consume the corresponding messages. In this mode, the same consumer can consume multiple messages, and the messages produced by one producer can be consumed by multiple consumers.

[0293] The protocols of the cloud-controlled video streaming service system rely on MQTT, RTCP, RTP, and UDP. The specific description is the same as above and will not be repeated here.

[0294] The application layer of the cloud-controlled video streaming service system includes video transcoding, video forwarding, ZLMediaKit video stream processing components, task scheduling and load balancing, video storage, and SDP (Streaming Distribution Protocol) cache.

[0295] Among them, video transcoding refers to converting a compressed and encoded video stream into another video stream to adapt to different network bandwidths, different terminal processing capabilities and different user needs. Transcoding is essentially to temporarily decode the video first, and then re-encode it into the required format and data encoding speed. The streams before and after the conversion may or may not follow the same video encoding standard. From a technical point of view, video transcoding technology is needed to solve many problems such as video format conversion, bit rate reduction, and time and spatial resolution reduction. The camera video service of steel plant unloading involves recording, live broadcast, playback, analysis and other functions. Its video transcoding service generally includes: scheduling core module, transcoding processing module, transcoding node module, transcoding merging module, transcoding task queue, transcoding request, transcoding result storage space, cluster distributed shared storage module, real-time video stream and video source file.

[0296] Video forwarding involves streaming media transmission and encoding and decoding technologies, which work together to enable video content to be efficiently transmitted from a server to multiple terminal devices. In order to adapt to different network environments and device performance, the present invention adopts adaptive bitrate streaming (Adaptive Bitrate, adaptive bit rate, ABR), that is, the video quality is adjusted according to the real-time bandwidth changes during the transmission process to ensure smooth playback. At the same time, the streaming media protocol RTP / RTCP is responsible for defining how data is transmitted in the form of a stream on the network, taking into account aspects such as error control, synchronization and data recovery.

[0297] ZLMediaKit video stream processing component is an open source streaming media server developed based on C++11. It supports multiple streaming protocols, including RTSP (Real-Time Streaming Protocol), RTMP (Real Time Messaging Protocol), WebRTC (Web Real-Time Communications), HLS (HTTP Live Streaming), HTTP-FLV (Flash Video over HTTP, using HTTP protocol to transmit Flash Video format video data, HTTP-FLV), etc. It has low latency and high concurrency processing capabilities. The main features of ZLMediaKit include: high performance and high concurrency, multi-protocol support, cross-platform support, low latency, flexible expansion, and open source community.

[0298] For task scheduling and load balancing, in video streaming services, the main role of balancing is to disperse user requests, optimize resource utilization and improve user experience. It avoids overloading of a single server by distributing incoming video streaming requests to multiple servers, and ensures high availability and reliability. Through task scheduling and load balancing, resource utilization is optimized, ensuring that each server operates within the effective working range, avoiding the situation where some servers are idle while other servers are overloaded, thereby improving the work efficiency and response speed of the entire system. Policy algorithms can consider polling, minimum connection, source address hashing, etc.

[0299] The process of video storage is divided into two steps: encoding and compression. The purpose of encoding is to convert the original video signal into a digital signal, and compression is to process the encoded digital signal to reduce the file size for easy transmission and storage. During the encoding process, the video signal is sampled and quantized and converted into a digital signal. Sampling is to sample a continuous analog signal at a certain time interval to obtain a discrete digital signal, so that the sample is an image frame. Quantization is to convert the continuous value obtained by sampling into a discrete value for subsequent processing. The compression process is divided into lossy compression and lossless compression. Lossy compression will lose part of the video data, sacrificing part of the image quality to reduce the file size. Lossless compression can save all video data completely, but the file size is larger. After encoding and compressing, the video data is saved in binary form on the storage medium. When playing, the video data is decoded and output to the display device, presenting a continuous dynamic picture.

[0300] SDP (Streaming Distribution Protocol, SDP) cache is a text protocol for multimedia data stream distribution. SDP itself is not a cache mechanism, but it can be used to control and manage the cache of media streams. The implementation logic of the SDP cache mechanism includes: when the server receives a PLAY request from the client, it generates an SDP file; the server provides the media stream according to the SDP file; if the same client requests the same media stream again, it can consider providing the media stream from the cache to reduce the need to generate a new SDP file.

[0301] The video content analysis service system can adopt a server-side AI computing layered architecture. Its underlying software relies on graphics card drivers, CUDA, and the GPU acceleration library cuDNN for deep neural networks.

[0302] Among them, the graphics driver is a software designed specifically for the GPU (Graphics Processing Unit, GPU), which is the bridge between the computer system and the GPU. The graphics driver enables the operating system to identify and communicate with the GPU, thereby unleashing the full functionality and performance of the GPU. It includes key components such as starting GPU hardware acceleration, controlling GPU operation, optimizing graphics performance, providing API support, power management, and heat dissipation control. Without a suitable driver, the GPU cannot work properly.

[0303] CUDA is a parallel computing platform and programming model that includes the CUDA instruction set architecture and the parallel computing engine inside the GPU. Developers can use C language to write programs for the CUDA architecture, and the programs written can run at ultra-high performance on CUDA-supported processors. CUDA is a complete GPU solution that greatly improves the performance of graphics processing and compute-intensive tasks. CUDA provides a direct access interface to the hardware, without having to rely on the graphics API to access the GPU as in traditional methods. CUDA's complete toolset includes compilers, math libraries, drivers, and platform component development environments.

[0304] cuDNN is a deep neural network library optimized for GPU devices. cuDNN provides more efficient and lower memory usage forward and back propagation algorithms, which are essential for the training and reasoning of deep learning models. cuDNN supports interfaces of multiple deep learning frameworks, allowing developers to take advantage of its optimization capabilities without deep low-level programming. The main functions and features of cuDNN include: convolution operation optimization, RNN (Recurrent Neural Networks, RNN) and LSTM (Long Short-Term Memory, LSTM) support, standard neural network layer optimization, FP16 half-precision floating point calculation support (Floating Point 16, half-precision floating point, FP16), cross-platform and interface support.

[0305] The AI ​​frameworks of the video content analysis service system include TensorFlow, Keras, and CNN (Convolutional Neural Network) model.

[0306] TensorFlow is an open source software library based on data flow graphs for research and application of numerical computing, machine learning, and deep neural networks. The core advantage of TensorFlow lies in its flexibility and scalability. Its workflow includes: data preprocessing, model building, model training, and prediction. TensorFlow has a multi-level structure and can be deployed on various servers, PC terminals, and web pages, and supports high-performance numerical computing on GPUs and TPUs (Tensor Processing Units).

[0307] Keras is an open source artificial neural network library written in Python. It provides a high-level API that makes it easier to design, debug, evaluate, apply, and visualize deep learning models. Keras supports multi-GPU parallel computing under multiple operating systems. Keras is a high-level API of TensorFlow. Keras can run on TensorFlow and can take advantage of TensorFlow's underlying computing power and optimization algorithms.

[0308] The CNN model, or convolutional neural network, is a deep learning model or a multi-layer perceptron similar to an artificial neural network, used for image recognition, object detection, natural language processing, etc. Classic CNN models include LeNet-5, AlexNet, ResNet, VGG, GoogleNet / Inception, etc.

[0309] The application layer of the video content analysis service system includes nonlinear activation function, normalized exponential function, pooling calculation, discarding calculation, and convolution calculation.

[0310] Among them, the nonlinear activation function is a key component of the neural network. Its role is to decide whether to pass the signal to the next layer of neurons after the neuron receives the input signal. Nonlinear activation functions, such as ReLU (Rectified Linear Unit, ReLU), S-shaped growth curve Sigmoid, Tanh (Hyperbolic Tangent Function, Tanh), etc., enable the neural network to learn and represent complex functions and decision boundaries. They can increase the nonlinear fitting ability of the neural network, thereby improving the expressive power of the model.

[0311] The normalized exponential function, also known as the Softmax function, is a data function that is used to "compress" a K-dimensional vector containing arbitrary real numbers into another K-dimensional real vector so that the range of each element is between (0, 1) and the sum of all elements is 1. The function of the normalized exponential function is to convert a set of numerical values ​​into a probability distribution, which is used for probability analysis and decision-making in multi-classification problems.

[0312] Pooling is a common operation in deep learning. It is used to reduce the number of network parameters, reduce computational complexity, and extract the invariance of features. Pooling represents each local area in the input feature map with a summary statistical value, such as maximum pooling or average pooling. Pooling is generally performed after the convolution layer of a convolutional neural network to compress the size of the feature map, reduce the number of parameters, and enhance the robustness of the features.

[0313] Dropout is a regularization technique used to prevent overfitting of neural networks during training. In each iteration of training, a number of neurons in each layer are randomly blocked with a certain probability, and the network consisting of the remaining neurons continues to be trained. This random discarding process is not performed during the test or inference phase to ensure the stability of the output. Dropout introduces randomness into the training process of the neural network, making the model more robust to perturbations of the input data. That is, the introduction of this randomness is equivalent to adding noise between layers, thereby enhancing the smoothness of the model and reducing the risk of overfitting.

[0314] Convolution refers to a mathematical operation in functional analysis that generates a third function through two functions f and g. Its essence is a special integral transformation, which represents the integral of the overlapping function value product of the functions f and g after flipping and translating over the overlapping length. Convolution is an operation performed on a specific window in the spatial domain and is a basic calculation method used in image smoothing and sharpening.

[0315] The large-screen visual monitoring system can adopt a layered architecture of the desktop client video playback system. Its underlying software dependencies include Windows, FFmpeg, OpenGL (Open Graphics Library, OpenGL), and CUDA (Compute Unified Device Architecture, CUDA).

[0316] Windows is the operating system commonly used in large-screen visual monitoring systems.

[0317] FFmpeg is an open source, cross-platform multimedia processing framework that includes tools and libraries for processing various multimedia data such as audio, video, and streaming media. FFmpeg's functions include video encoding and decoding, transcoding, editing, splicing, filter effects, screenshots, thumbnail generation, etc. It also supports the transmission and forwarding of streaming media, and can play or push audio and video data in real time over the network.

[0318] OpenGL is a cross-programming language and cross-platform programming graphics programming interface. It uses graphics card hardware to accelerate graphics rendering of 2D and 3D vector graphics APIs, providing high-performance graphics processing services. OpenGL uses a client-server architecture design, with the application as the client and the image hardware device as the server. The client is responsible for submitting OpenGL commands, and the server executes these commands and renders the image. The main function of OpenGL is to provide a set of functions for creating and controlling graphic objects; it also provides shader programming for calculating the color value of each pixel; in addition, there are rich texture, lighting, shadow and depth testing functions to better present 3D scenes.

[0319] The protocols of the large-screen visual monitoring system rely on MQTT, RTCP, RTP, and UDP. The specific description is the same as above and will not be repeated here.

[0320] The application layer of the large-screen visual monitoring system includes QoS (Quality of Service) guarantee, FEC forward error correction (FEC), GPU decoding and rendering, cyclic redundancy, NACK (Negative Acknowledgment, negative acknowledgment, NACK), video bit rate adaptive control, and Jitter Buffer.

[0321] Among them, QoS guarantee is a set of network design principles and technologies used to ensure the quality of data transmitted in the network. The QoS quality assurance parameters in video transmission mainly include bandwidth, delay, packet loss rate and jitter. Through traffic shaping (controlling the sending rate and number of data packets to achieve the purpose of optimizing network performance), priority queue (sorting and sending according to the priority of data packets to ensure that important data is transmitted first), flow control (controlling the rate and bandwidth of traffic by setting restrictions to avoid network congestion), congestion control (detecting indicators such as network load status and delay changes, dynamically adjusting the rate and bandwidth of traffic to reduce congestion), packet loss recovery (using forward error correction code or retransmission mechanism to recover lost data packets to ensure the integrity of data transmission) and other technologies, these parameters can be effectively optimized to provide a better video transmission experience.

[0322] FEC (Forward Error Correction) is an effective data error correction method used to improve the reliability of data transmission and ensure the integrity and accuracy of data. By adding a certain amount of redundant error correction code at the sending end and sending it together with the original data, it ensures that the receiving end can detect and correct errors when receiving the data. FEC is applicable to one-way communication systems and reduces the bit error rate through a predetermined algorithm, thereby reducing errors in data transmission. In video playback, FEC technology can effectively combat network instability, packet loss and other problems, and ensure the continuity and quality of video streaming. FEC technology is particularly suitable for situations where the network environment is unstable or the bandwidth is low.

[0323] For GPU decoding and rendering, it is a basic requirement to watch multiple HD video images in real time on the large screen in the background of the steel plant. Each video image needs to be decoded and rendered. If the decoding and rendering work are handed over to the CPU, the CPU load will soar immediately. In serious cases, the CPU will run full and cause the system to freeze. In addition, the CPU is not good at image processing. For H.264 / H.265 videos, decoding and rendering are both professional skills of GPU graphics cards. To decode H.264 / H.265 video graphics, the GPU must first copy them to the video memory and then decode them by the GPU. The image data obtained after decoding (usually in YUV format, Y brightness component, U and V represent chrominance components) is also in the video memory. At this time, the data is directly submitted to the GPU for rendering to achieve high performance.

[0324] Regarding cyclic redundancy, in live video, cyclic redundancy mainly involves video compression and error handling mechanisms. In live video systems, video compression involves removing redundant information to improve compression efficiency and save storage space. Redundant information mainly includes data redundancy, visual redundancy, spatial redundancy, temporal redundancy, image area redundancy, etc. In addition, error handling and redundant transmission mechanisms in live broadcast systems are also important aspects to ensure live broadcast quality. This includes error handling of live broadcast slices to ensure accurate data transmission, and at the same time, handling redundant transmission through certain mechanisms to further optimize data transmission efficiency.

[0325] NACK is a QoS strategy for video processing. When the receiver detects a lost data packet, it can send a NACK message to notify the sender that the data packet needs to be retransmitted.

[0326] Adaptive video bitrate control means that the video stream is allowed to dynamically adjust the bitrate according to network conditions, device capabilities or user preferences to optimize video quality and user experience. This technology ensures that the video can be played in the best way in various environments by real-time monitoring and adjusting the encoding parameters of the video. Its key technologies include: adaptive bitrate control algorithms (such as content adaptive rate control algorithm CABR), bandwidth estimation and adaptive bitrate switching, neural network prediction and adaptive modulation, and user experience optimization.

[0327] Jitter Buffer is a kind of jitter, which means different delays when sending from the source address to the destination address, that is, delay variation. Jitter will make the video playback unstable, and its solution is to increase the delay. The main function of Jitter Buffer is to cache unstable data packets in network transmission to smooth the jitter in network transmission, so as to ensure that the video data can be played in the correct order at the receiving end, ensuring the continuity and stability of the video data. Network jitter factors include network congestion, route changes, bandwidth limitations, etc. Jitter Buffer caches the received video data packets and sorts them according to their timestamps and sequence numbers to ensure that the data packets are played in the correct order. When a data packet arrives, Jitter Buffer calculates the time difference between its arrival time and the previous data packet (that is, jitter) and stores it in the cache. When a data packet needs to be played, Jitter Buffer determines whether it needs to wait for more data packets to arrive based on the data packets already in the cache.

[0328] Based on the above content, this embodiment uses cloud computing, big data, AI deep learning neural network, video streaming technology, 5G-CV2X communication technology, high-precision maps, unmanned driving technology, information security technology, vehicle-side software and hardware embedded technology, etc., and configures an integrated unloading control platform, remote safety takeover system, CA security certification center, vehicle fault diagnosis and prediction system, steel plant unloading operation management system, factory equipment, etc., to realize unmanned vehicle driving and unloading operations in the factory, which helps to realize the intelligent, unmanned and standardized operation of steel plant unloading.

[0329] A storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any unloading control method provided in the embodiment of the present application.

[0330] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the unloading control methods provided in the embodiments of the present application.

[0331] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0332] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0333] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for controlling unloading, characterized in that: Applied to a discharge control platform, the method comprises: Obtaining the material weight information of the mineral powder loaded on the vehicle to be unloaded, and calculating the unloading time according to the material weight information as the pre-calculated time; After determining that the unloading vehicle has parked in the parking space of the target unloading slot and has started unloading, waiting for the pre-calculated time; the target unloading slot is a unloading slot in the unloading area that matches the material type information of the mineral powder loaded by the unloading vehicle; Acquire a current residual material state image acquired by an image acquisition device disposed at the target unloading slot; Calling a pre-trained residual material state detection model to process the current residual material state image to obtain the residual material remaining ratio of the vehicle to be unloaded; wherein the residual material state detection model is trained using residual material state sample images marked with residual material remaining ratios; Determine the target number of rapid advances and sudden brakes required to make the residual materials flow out completely based on at least the remaining proportion of the residual materials in the vehicle to be unloaded; If the determined target number is greater than zero, a residual material processing instruction is sent to the vehicle to be unloaded, and the residual material processing instruction is used to control the vehicle to be unloaded to perform the target number of rapid advances and sudden brakes. After the vehicle to be unloaded completes the rapid advances and sudden brakes, the current residual material status image captured by the image acquisition device set at the target unloading hole slot is obtained, and the command returns to execute the call of the pre-trained residual material status detection model to process the current residual material status image.

2. The unloading control method according to claim 1, characterized in that: At least according to the remaining proportion of the residual materials in the vehicle to be unloaded, the target number of rapid advances and sudden brakes required to make the residual materials flow out completely is determined, including: According to the material type information and the remaining proportion of the residual material in the vehicle to be unloaded, the target number of rapid advances and sudden brakes required to clear the residual material is determined.

3. The unloading control method according to claim 1 or 2, characterized in that: Before obtaining the material weight information of the mineral powder loaded on the unloading vehicle, the method further includes: Obtaining unloading task information, the unloading task information including vehicle identification information of the vehicle to be unloaded and material type information and material weight information of the mineral powder loaded by the vehicle to be unloaded; the vehicle to be unloaded has completed the weighing at the factory, and the vehicle to be unloaded is connected to the factory wireless network; According to the correspondence between the pre-configured unloading slots and the material types, a unloading slot matching the mineral powder loaded by the unloading vehicle to be unloaded is determined from the unloading slots in the unloading area as a target unloading slot; When it is determined that the target unloading slot is in an idle state, a unloading preparation instruction is sent to the vehicle to be unloaded, wherein the unloading preparation instruction is used to control the vehicle to be unloaded to drive to the parking space of the target unloading slot in an unmanned manner and park at the parking space of the target unloading slot; After determining that the vehicle to be unloaded has parked in the parking space of the target unloading slot based on the unloading task information, sending an unloading instruction to the vehicle to be unloaded, wherein the unloading instruction is used to control the vehicle to be unloaded to perform an unloading operation, and the unloading operation at least includes lifting the upper loading box to an oblique position; The step of determining that the vehicle to be unloaded parks in a parking space of a target unloading slot and starts unloading comprises: upon determining that the vehicle to be unloaded starts to perform the unloading operation in response to the unloading instruction, determining that the vehicle to be unloaded parks in a parking space of a target unloading slot and starts unloading.

4. The unloading control method according to claim 3, characterized in that: The method further includes: If the determined target number is equal to zero, a unloading restoration instruction is sent to the unloading vehicle, wherein the unloading restoration instruction is used to control the unloading vehicle to perform an unloading restoration operation, wherein the unloading restoration operation at least includes restoring the upper loading box to a horizontal position; After it is determined that the vehicle to be unloaded has completed the unloading restoration operation, it is determined that the unloading task of the vehicle to be unloaded is completed.

5. The unloading control method according to claim 4, characterized in that: After determining that the unloading task of the vehicle to be unloaded is completed, the method further includes: Determining that the vehicle to be unloaded is a vehicle to be washed; Sending a car wash preparation instruction to the vehicle to be washed, wherein the car wash preparation instruction is used to control the vehicle to be washed to drive to a car wash station and park at the car wash station in an unmanned manner; After determining that the vehicle to be washed has stopped at the car wash station, sending a water spray control instruction to a water sprayer provided at the car wash station, wherein the water spray control instruction is used to control the water sprayer to perform a water spray operation for a first preset time period; After it is determined that the water sprinkler completes the water spraying operation, it is determined that the vehicle washing task of the vehicle to be washed is completed.

6. The unloading control method according to claim 5, characterized in that: After determining that the vehicle washing task for the vehicle to be washed is completed, the method further includes: Determining that the vehicle to be washed is a vehicle to be blown; Sending a blowing preparation instruction to the vehicle to be blown, wherein the blowing preparation instruction is used to control the vehicle to be blown to drive to the blowing station in an unmanned manner and stop at the blowing station; After determining that the vehicle to be blown has stopped at the blow-drying station, sending a blow-drying control instruction to a vehicle blower provided at the blow-drying station, wherein the blow-drying control instruction is used to control the vehicle blower to perform a blow-drying operation for a second preset time; After determining that the vehicle blower completes the blowing operation, it is determined that the blowing task of the vehicle to be blown is completed.

7. The unloading control method according to claim 6, characterized in that: After determining that the air blowing task of the vehicle to be air-blown is completed, the method further includes: Obtaining an empty weight value of an unloaded vehicle that has been weighed and a fully loaded weight value corresponding to the unloaded vehicle that has been weighed; Calculate the difference between the full-load weight value and the empty-load weight value as the actual material weight value; If the degree of matching between the actual material weight value and the material weight information is within a preset limit, the weighed empty vehicle is determined to be a vehicle to be shipped, a shipment preparation instruction is sent to the vehicle to be shipped, and task completion information is output to the driver of the vehicle to be shipped, so as to instruct the driver to take over the vehicle to be shipped at a preset handover position; the shipment preparation instruction is used to control the vehicle to be shipped to drive to the handover position in an unmanned manner and park at the handover position; If the degree of matching between the actual material weight value and the material weight information is not within a preset limit, an alarm message is output, wherein the alarm message is used to indicate that the weight of the mineral powder loaded on the vehicle is abnormal.

8. The unloading control method according to claim 7, characterized in that: Before obtaining the unloading task information, including: In response to the images of vehicles entering the factory collected by the image acquisition device set at the factory entrance, a pre-trained license plate number recognition model is called to process the images of vehicles entering the factory to obtain the license plate number recognition results of the vehicles entering the factory; the license plate number recognition model is trained using sample images of vehicles marked with license plates; Searching for a reservation record matching the license plate number recognition result from the reservation list; the reservation list includes reservation records of vehicles to be unloaded at the factory, and the reservation records include the vehicle license plate number information and the material type and weight of the mineral powder loaded by the vehicle; If the search is successful, the gate pole at the factory entrance is controlled to be lifted to allow the vehicle to enter the factory; after the vehicle stops at the handover position, it is determined that the vehicle is a fully loaded vehicle to be weighed, and a fully loaded weighing preparation instruction is sent to the fully loaded vehicle to be weighed; the fully loaded weighing preparation instruction is used to control the fully loaded vehicle to be weighed to drive to the weighing platform in an unmanned manner; After the vehicle to be shipped stops at the handover position, the method further includes: In response to the image of the out-of-factory vehicle collected by the image collection device arranged at the factory exit, the license plate number recognition model is called to process the image of the out-of-factory vehicle to obtain the license plate number recognition result of the out-of-factory vehicle; When it is determined that the leaving vehicle is the waiting leaving vehicle, the gate pole arranged at the factory exit is controlled to be lifted so as to allow the waiting leaving vehicle to drive out of the factory.

9. A discharge control device, characterized in that: Applied to the unloading control platform, the device includes: The unloading time calculation unit is used to obtain the material weight information of the mineral powder loaded on the unloading vehicle, and calculate the unloading time according to the material weight information as the pre-calculated time; A material unloading monitoring unit is used to wait for the pre-calculated time after determining that the vehicle to be unloaded has parked in the parking space of the target unloading slot and started unloading, obtain the current residual material status image acquired by the image acquisition device set at the target unloading slot, call the pre-trained residual material status detection model to process the current residual material status image, and obtain the residual material remaining ratio of the vehicle to be unloaded; wherein, the target unloading slot is a unloading slot in the unloading area that matches the material type information of the mineral powder loaded by the vehicle to be unloaded, and the residual material status detection model is trained using residual material status sample images marked with residual material remaining ratios; The unloading monitoring unit is further used to determine a target number of sudden advances and sudden brakes required to clear the residual material based on at least the remaining proportion of the residual material on the vehicle to be unloaded; if the determined target number is greater than zero, a residual material processing instruction is sent to the vehicle to be unloaded, and the residual material processing instruction is used to control the vehicle to be unloaded to perform the target number of sudden advances and sudden brakes, and after the vehicle to be unloaded completes the sudden advances and sudden brakes, obtain the current residual material status image captured by the image acquisition device set at the target unloading hole slot, and return to execute and call the pre-trained residual material status detection model to process the current residual material status image.

10. A material discharging control system, characterized in that: include: Vehicles, plant equipment and unloading control platforms; The plant equipment includes: image acquisition devices respectively arranged at each discharge hole slot in the discharge area; The unloading control platform obtains the material weight information of the mineral powder loaded on the vehicle, and calculates the unloading time according to the material weight information as the pre-calculated time; After determining that the vehicle has parked in the parking space of the target unloading slot and has started unloading, the unloading control platform waits for the pre-calculated time to obtain the current residual material state image acquired by the image acquisition device provided at the target unloading slot; wherein the target unloading slot is the unloading slot in the unloading area that matches the material type information of the mineral powder loaded by the vehicle; The unloading control platform calls a pre-trained residual material state detection model to process the current residual material state image to obtain the residual material remaining ratio of the vehicle; wherein the residual material state detection model is trained using residual material state sample images marked with residual material remaining ratios; The unloading control platform determines the target number of sudden advances and sudden brakes required to make the residual materials flow out at least according to the residual material remaining ratio of the vehicle; if the determined target number is greater than zero, a residual material processing instruction is sent to the vehicle, and after the vehicle completes the sudden advance and sudden brake, a current residual material state image captured by an image acquisition device set at the target unloading hole slot is obtained, and the pre-trained residual material state detection model is called back to execute and process the current residual material state image; The vehicle performs the target number of rapid advances and sudden brakes in response to the residual material handling instruction.

Images