Vehicle control method and system, storage medium and vehicle

Abstract

The invention discloses a vehicle control method and system, a storage medium and a vehicle. The method comprises the steps that in the process that a vehicle executes an operation task in a target area, image information around the vehicle is obtained, and the image content of the image information at least comprises obstacles around the vehicle and an operation object corresponding to the operation task; a recognition model is utilized to recognize contour features of the operation object in the image information to obtain a recognition result of the operation object, and the recognition model is obtained by training a convolutional neural network model by utilizing an image information sample containing an operation object sample and a recognition result sample of the operation object sample; based on the identification result, determining a control strategy for the to-be-executed job task of the job object; and in response to a control instruction corresponding to the control strategy, controlling the vehicle to execute an operation task on the operation object at the target operation position. The technical problem that the control accuracy of the vehicle is low is solved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a vehicle control method, system, storage medium and vehicle. Background Art

[0002] Currently, when vehicles are performing tasks, such as identifying and moving goods in a warehouse, if they cannot accurately identify the location and type of goods, it may lead to incorrect handling or missing goods. If the vehicle collides with other obstacles or vehicles during movement, it may cause damage or personal injury. Therefore, it is very important to ensure the accuracy of the vehicle's tasks when it is unmanned.

[0003] In the related art, machine vision can be used in vehicles to identify work objects. For example, barcode scanning and radio frequency identification (RFID) methods can be used to identify and carry work objects. However, for the barcode scanning method, its recognition ability will be greatly reduced when there is no barcode or the barcode is blocked or worn. For the RFID method, a higher standard RFID system may be required in scenarios that require high-frequency reading and precise positioning. Through the above analysis, these two methods still have shortcomings in recognition ability, adaptability, and flexibility. Therefore, there is still a technical problem of low vehicle control accuracy.

[0004] With respect to the technical problem of low control accuracy of the above-mentioned vehicles, no effective solution has been proposed so far. Summary of the invention

[0005] Embodiments of the present invention provide a vehicle control method, system, storage medium and vehicle to at least solve the technical problem of low vehicle control accuracy.

[0006] According to one aspect of an embodiment of the present invention, a vehicle control method is provided. The method may include: in the process of the vehicle performing a work task in a target area, obtaining image information around the vehicle, wherein the image content of the image information at least includes obstacles around the vehicle and a work object corresponding to the work task; using a recognition model, identifying the contour features of the work object in the image information to obtain a recognition result of the work object, wherein the recognition model is obtained by training a convolutional neural network model using image information samples containing work object samples and recognition result samples of the work object samples, and the recognition result samples are obtained by classifying the work object samples according to the contour feature samples in the image information samples; based on the recognition result, determining a control strategy for the work task to be performed on the work object, wherein the control strategy is used to instruct the vehicle to perform the work task while avoiding obstacles; responding to a control instruction corresponding to the control strategy, controlling the vehicle to perform the work task on the work object at the target work position.

[0007] Optionally, the recognition model includes a contour feature enhancement layer and a standard convolution layer, wherein the recognition model is used to identify the contour features of the work object in the image information to obtain a recognition result of the work object, including: in response to the recognition model receiving image information, the image information is input into the standard convolution layer, and the image information is recognized by the standard convolution layer to obtain an initial recognition result, wherein the initial recognition result is used to represent the visual features of the image information; the initial recognition result is input into the contour feature enhancement layer, and the initial recognition result is enhanced by the contour feature enhancement layer to obtain the enhanced initial recognition result, wherein the signal strength of the contour signal corresponding to the contour feature in the enhanced initial recognition result is greater than the signal strength of the contour signal corresponding to the contour feature before the enhancement; based on the initial recognition result and the enhanced initial recognition result, the recognition result is determined.

[0008] Optionally, in response to the recognition model receiving image information, the image information is input into a standard convolution layer, and the image information is recognized using the standard convolution layer to obtain an initial recognition result, including: in response to the standard convolution layer receiving image information, a convolution operation is performed on the image information, weight information and bias information of the standard convolution layer to obtain an initial recognition result; the initial recognition result is input into a contour feature enhancement layer, and the initial recognition result is enhanced using the contour feature enhancement layer to obtain an enhanced initial recognition result, including: in response to the contour feature enhancement layer receiving the initial recognition result, a convolution operation is performed on the initial recognition result, the weight information and bias information of the contour feature enhancement layer to obtain the enhanced initial recognition result.

[0009] Optionally, based on the initial recognition result and the initial recognition result after enhancement processing, the recognition result is determined, including: obtaining weight data corresponding to the initial recognition result after enhancement processing; using the weight data to fuse the initial recognition result and the initial recognition result after enhancement processing to obtain the recognition result.

[0010] Optionally, the method also includes: obtaining a convolutional neural network model to be trained, and classification error data and contour error data corresponding to the convolutional neural network model; using the classification error data and contour error data to train the convolutional neural network model to obtain a recognition model.

[0011] Optionally, the recognition result samples include category recognition samples, and the category recognition samples correspond to classification error data, wherein obtaining the classification error data corresponding to the convolutional neural network model includes: obtaining the category recognition samples output by the convolutional neural network model based on the image information samples, and the real category samples corresponding to the image information samples; using the category recognition samples and the real category samples to determine the classification error data.

[0012] Optionally, the recognition result sample includes a contour recognition sample, and the contour recognition sample corresponds to the contour error data, wherein obtaining the contour error data corresponding to the convolutional neural network model includes: obtaining a standard convolution layer in the convolutional neural network model, based on an initial recognition result sample output by the image information sample, and a contour feature enhancement layer in the convolutional neural network model, based on a contour recognition sample output by the initial recognition result sample; and determining the contour error data using the initial recognition result sample and the contour recognition sample.

[0013] Optionally, the classification error data and the contour error data are used to train the convolutional neural network model to obtain a recognition model, including: determining the loss function of the convolutional neural network model based on the classification error data and the contour error data; using the loss function to train the convolutional neural network model to obtain a trained initial recognition model; in response to the iterative result corresponding to the recognition result sample of the initial recognition model, the difference information between the iterative result and the previous iterative result is less than a difference information threshold, determining the recognition model based on the initial recognition model, wherein the difference information is used to indicate the degree of difference between the iterative result and the previous iterative result.

[0014] Optionally, the control instructions include driving instructions and grabbing instructions, and the work tasks include grabbing tasks and transportation tasks corresponding to the work objects, wherein the control instructions corresponding to the control strategy are responded to, and the vehicle is controlled to be at the target work position to perform the work tasks on the work object, including: responding to the driving path corresponding to the driving instructions, and controlling the vehicle to drive to the target work position; at the target work position, responding to the grabbing instructions, and performing the grabbing task on the work object; in response to the vehicle completing the grabbing task, responding to the driving path, and performing the transportation task on the work object.

[0015] According to another aspect of an embodiment of the present invention, a vehicle control system is also provided. The system may include: an image acquisition unit, used to obtain image information around the vehicle when the vehicle performs a work task in a target area, wherein the image content of the image information at least includes obstacles around the vehicle and a work object corresponding to the work task; a processing unit, used to use a recognition model to identify the contour features of the work object in the image information to obtain a recognition result of the work object, wherein the recognition model is obtained by training a convolutional neural network model using image information samples containing work object samples and recognition result samples of the work object samples, and the recognition result samples are obtained by classifying the work object samples according to the contour feature samples in the image information samples; based on the recognition result, a control strategy for the work task to be performed on the work object is determined, wherein the control strategy is used to instruct the vehicle to perform the work task while avoiding obstacles; a control execution unit, used to respond to a control instruction corresponding to the control strategy, control the vehicle to perform the work task on the work object at the target work position.

[0016] Optionally, the control system is connected to a central control system, which is used to assign corresponding work tasks to the control systems of multiple vehicles corresponding to the multiple vehicles, and plan driving paths in the target area for the control systems of the multiple vehicles.

[0017] According to another aspect of an embodiment of the present invention, a vehicle control device is also provided. The device may include: an acquisition unit, used to acquire image information around the vehicle when the vehicle performs a work task in a target area, wherein the image content of the image information at least includes obstacles around the vehicle and a work object corresponding to the work task; a recognition unit, used to use a recognition model to recognize the contour features of the work object in the image information to obtain a recognition result of the work object, wherein the recognition model is obtained by training a convolutional neural network model using image information samples containing work object samples and recognition result samples of the work object samples, and the recognition result samples are obtained by classifying the work object samples according to the contour feature samples in the image information samples; a determination unit, used to determine a control strategy for the work task to be performed on the work object based on the recognition result, wherein the control strategy is used to instruct the vehicle to perform the work task while avoiding obstacles; and an execution unit, used to respond to a control instruction corresponding to the control strategy, control the vehicle to perform the work task on the work object at the target work position.

[0018] According to another aspect of an embodiment of the present invention, a processor is provided, which is used to run a program, wherein the vehicle control method of the embodiment of the present invention is executed when the program is run.

[0019] According to another aspect of the embodiment of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the vehicle control method of the embodiment of the present invention.

[0020] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the vehicle control method of the embodiment of the present invention.

[0021] According to another aspect of the embodiment of the present invention, a vehicle is provided, which is used to execute the vehicle control method of the embodiment of the present invention.

[0022] In an embodiment of the present invention, when a vehicle performs a work task in a target area, the environment around the vehicle can be monitored in real time to obtain image information containing obstacles around the vehicle or work objects. The contour features of the work object in the image information can be identified using a recognition model to obtain a recognition result of the work object. Based on the recognition result, a control strategy for the work task to be performed on the work object can be determined, and the control strategy can be used for the vehicle to perform the work task while avoiding obstacles. According to the control instructions corresponding to the control strategy, the vehicle can be controlled to perform the work task on the work object at the target work position.

[0023] In this embodiment, in order to improve the accuracy of the identification of the working object, the contour features of the working object are learned and mastered by training the recognition model, so that the target object can be more reliably identified even under partial occlusion or poor light conditions, thereby improving the accuracy of the identification. In order to enhance the adaptability to the environment, the image information of the surrounding environment is obtained during the vehicle's execution of the working task, including the detection of obstacles, so that a quick response can be made to avoid collisions and ensure the safe operation of the operation. In order to improve the flexibility of the operation, the operation plan is dynamically adjusted according to the real-time environmental data and the information of the working object, so that the automated operation is not limited to the preset path and operation, but can more flexibly adapt to the changing working environment and operation requirements, thereby improving the practicality of the automation system. After obtaining the image information containing obstacles or working objects around the vehicle, the contour features of the working object in the image information are identified by using the recognition model, which can help the vehicle to find and avoid obstacles in time when performing the working task, and accurately lock the working object, thereby improving the working efficiency and safety. Based on the recognition result, the control strategy is determined, so that the vehicle can flexibly adjust the way of performing the working task according to the actual situation, thereby improving the accuracy of performing the working task. Thereby solving the technical problem of low control accuracy of the vehicle and achieving the technical effect of improving the control accuracy of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0025] Figure 1 is a flow chart of a vehicle control method according to an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of image data according to an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of a feature map extraction process of a stored goods image according to an embodiment of the present invention;

[0028] Figure 4 is a flow chart of an unmanned pallet truck automated operation using an intelligent identification method for stored goods based on monitoring equipment according to an embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of a control system of a vehicle according to an embodiment of the present invention;

[0030] Figure 6 is a schematic diagram of a vehicle control device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] According to an embodiment of the present invention, an embodiment of a method for charging reminder of a battery in a vehicle is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] Figure 1 is a flow chart of a vehicle control method according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0035] Step S102, while the vehicle is performing a task in the target area, image information around the vehicle is obtained.

[0036] In the technical solution provided in the above step S102 of the present invention, the image information may also be referred to as image data, wherein the image content includes at least obstacles within the vicinity of the vehicle and the work object corresponding to the work task. If the target area is a warehouse, the image information may be an image of stored goods. The vehicle may be an autonomous driving vehicle, for example, an unmanned pallet truck. The target area may be an area where the work task is performed, for example, when carrying goods, the target area may be a warehouse. The work tasks may be transporting goods, pulling goods, picking up goods, and the like. Obstacles may be objects other than the work tasks in the target area that hinder the vehicle from performing the work tasks normally, for example, in a warehouse, obstacles may be shelves, other vehicles, or staff. The work object may be goods to be processed, for example, if the goods being transported are related goods in the vehicle, then they may be auto parts.

[0037] It should be noted that the above-mentioned target areas, work tasks, obstacles, and work objects are only for illustration and are not specifically limited here. As long as the vehicle control method in the embodiment of the present invention can be reasonably utilized to control the above-mentioned information corresponding to the work tasks performed by the vehicle, it is within the protection scope of the embodiment of the present invention.

[0038] In this embodiment, while the vehicle is performing a work task in the target area, image information around the vehicle can be obtained. The image information can help the vehicle perceive the surrounding environment in real time, identify obstacles and avoid them, and also locate the position of the work object to ensure the accurate execution of the work task.

[0039] Optionally, the image information can be obtained by a sensor (vehicle-mounted sensor), such as a vehicle-mounted camera, monitoring equipment, and radar. It should be noted that the method of obtaining the image information here is only for illustration and is not specifically limited. As long as it can be used to obtain image information, it is within the protection scope of the embodiments of the present invention.

[0040] Step S104: using the recognition model, identifying the contour features of the working object in the image information to obtain a recognition result of the working object.

[0041] In the technical solution provided in the above step S104 of the present invention, when the vehicle performs the work task in the target area, after acquiring the image information around the vehicle, the recognition model can be used to recognize the contour features of the work object in the image information to obtain the recognition result of the work object. Among them, the recognition model is obtained by training the convolutional neural network model using the image information samples containing the work object samples and the recognition result samples of the work object samples, and the recognition result samples are obtained by classifying the work object samples according to the contour feature samples in the image information samples. The image information can be a component image. The contour features of the work object can be the edge and contour information of the component. The recognition result can be a model classification result, that is, the recognition model can classify the work object in the input image information. For example, when the component image is input into the recognition model, the corresponding recognition category can be "clutch pressure plate". The recognition result sample can be image data manually labeled with component categories.

[0042] In this embodiment, by identifying the contour features of the work object in the image information through the recognition model, the contour features of the work object can be automatically recognized without human intervention, which can save time and labor costs and ensure automated, highly accurate and high-precision recognition of the work object.

[0043] Optionally, the recognition model can be an improved convolutional neural network structure, which can enhance the sensitivity of the recognition model to edge and contour information, highlight the contour of the work object, and provide the recognition model with richer shape features, thereby improving the accuracy and robustness of recognition. For example, the recognition accuracy of goods such as auto parts can be improved.

[0044] In an embodiment of the present invention, the image information around the vehicle can be collected during the process of the vehicle performing the work task. The real-time image information is input into the trained recognition model, and the recognition model recognizes the contour features in the image. By using the recognition model to recognize the contour features of the work object in the image information, the edge and contour information of the image can be automatically captured and analyzed, thereby achieving accurate classification and positioning of the work object. By using the recognition model to recognize the contour features of the work object, not only automation and efficiency are achieved, but also the accuracy of recognition is improved, and misoperation during the work process is reduced. Thereby solving the technical problem of low control accuracy of the vehicle, and achieving the technical effect of improving the control accuracy of the vehicle.

[0045] Optionally, the recognition model can automatically analyze and identify the contour features of the work object and complete the recognition task without human intervention, which can save time and labor costs. Especially in large-scale warehousing or manufacturing environments, automated recognition can significantly improve overall work efficiency. By using a convolutional neural network model optimized for the contour features of the work object, the edges and contours of the object can be highlighted, and a high recognition accuracy can be maintained even in partial occlusion, poor lighting conditions or changes in viewing angle. The recognition model is based on learning from a large amount of sample data and can adapt to different types of work objects and work scenarios. The vehicle is not limited to identifying a single type of object, but can handle a wide variety of parts, increasing flexibility and versatility. Recognition based on contour features can reduce misidentification caused by similar shapes, reduce the error rate that may occur in operations, and can effectively avoid production delays or cost waste caused by recognition errors.

[0046] Step S106: determining a control strategy for the task to be performed by the operation object based on the recognition result.

[0047] In the technical solution of step S106 of the present invention, after the contour features of the working object in the image information are identified by using the recognition model and the recognition result of the working object is obtained, a control strategy for the working task to be performed by the working object can be determined based on the recognition result. The control strategy is used to instruct the vehicle to perform the working task while avoiding obstacles.

[0048] In this embodiment, the position and shape of the obstacle, as well as the degree of impact of the obstacle on the vehicle's execution of the task, can be determined based on the recognition results. If the obstacle does not pose a threat to the safety of the vehicle and does not affect the completion of the task, the vehicle can continue to perform the task without changing the driving path or stopping the task. If the obstacle poses a threat to the safety of the vehicle or affects the completion of the task, the vehicle needs to adopt a corresponding control strategy and can change the path or stop the task. Determining the control strategy for the task to be performed by the task object based on the recognition results can help ensure the smooth progress of the task and improve accuracy.

[0049] In the embodiment of the present invention, the control strategy can guide the vehicle on how to perform the task, optimize resource management, enhance the safety of performing the task, improve the flexibility of performing the task, and promote continuous performance optimization to ensure the efficient operation and accuracy of the task. Based on the recognition results, the control strategy can dynamically adjust the priority and resource allocation of the task to ensure that key tasks can be given priority while reducing resource waste and waiting time. Based on the task information and environmental conditions, the task can be quickly adjusted, such as re-planning the path, adjusting the grabbing order, etc., thereby improving the adaptability and flexibility.

[0050] Optionally, in order to optimize resource management, in a multi-task environment, the task allocation between vehicles can be dynamically adjusted according to the identification results of each vehicle to optimize resource utilization. For example, the order in which multiple vehicles perform tasks on multiple work objects can be intelligently scheduled according to the type and location of the work object, reducing unnecessary waiting time between vehicles and congestion caused by multiple vehicles performing work tasks on the same work object at the same time, thereby improving overall work efficiency.

[0051] For example, if the recognition result indicates that the work object is a specific type of auto part, such as a clutch pressure plate, the control strategy can define an effective method for grasping the auto part, including the selection of grasping tools, the positioning of grasping points, the control of grasping force, etc. At the same time, the control strategy can also guide the vehicle on how to plan the driving path and transport the object auto part to the destination.

[0052] Step S108, responding to the control instruction corresponding to the control strategy, controlling the vehicle to perform the work task on the work object at the target work position.

[0053] In the technical solution provided in the above step S108 of the present invention, in the process of determining the control strategy for the work task to be performed on the work object based on the recognition result, the vehicle can be controlled at the target work position to perform the work task on the work object according to the control instructions corresponding to the control strategy.

[0054] In this embodiment, by responding to the control instructions corresponding to the control strategy, the vehicle can be controlled to perform automated work tasks accurately, safely and efficiently without human intervention, which can improve the accuracy of work tasks and reduce labor costs and error rates.

[0055] Optionally, when the control strategy for the work task to be performed on the work object is determined based on the recognition result obtained by the recognition model, the central control system of the vehicle, that is, the central control system, can assign corresponding work tasks to the control systems of multiple vehicles corresponding to the multiple vehicles, and plan the driving paths in the target area for the control systems of the multiple vehicles. According to the control instructions corresponding to the control strategy, the vehicle can be controlled to perform the work task on the work object at the target work position. The control instructions can be speed control, steering control, braking control, and operation instructions of the robot arm or grasping tool. For example, when the target work object is identified as a "clutch pressure plate" and its position is determined, a control instruction can be generated to instruct the vehicle to approach the target at a safe speed, while planning the grasping action of the robot arm.

[0056] Optionally, the vehicle can navigate to the target work location based on the driving path information in the control instruction. The built-in navigation system or path planning algorithm can intelligently select the path, avoid obstacles, and quickly reach the destination. When the vehicle reaches the target work location, a positioning operation can be performed to ensure that the relative position of the vehicle and the work object is accurate. The positioning operation can be achieved through sensors, positioning systems, or visual recognition. According to the work task requirements in the control instruction, the vehicle can perform corresponding work tasks on the work object, such as grabbing, carrying, placing, etc.

[0057] In the above steps S102 to S108 of the present application, in the process of the vehicle performing the work task in the target area, the environment around the vehicle can be monitored in real time to obtain image information containing obstacles around the vehicle or the work object. The recognition model can be used to identify the contour features of the work object in the image information to obtain the recognition result of the work object. Based on the recognition result, a control strategy for the work task to be performed on the work object can be determined to instruct the vehicle to perform the work task while avoiding obstacles. According to the control instructions corresponding to the control strategy, the vehicle can be controlled at the target work position to perform the work task on the work object.

[0058] In this embodiment, in order to improve the accuracy of the identification of the working object, the contour features of the working object are learned and mastered through the trained recognition model, so that the target object can be more reliably identified even under partial occlusion or poor light conditions, thereby improving the accuracy of the identification. In order to enhance the adaptability to the environment, the image information of the surrounding environment is obtained during the vehicle's execution of the working task, including the detection of obstacles, so that a quick response can be made to avoid collisions and ensure the safe operation of the operation. In order to improve the flexibility of the operation, the operation plan is dynamically adjusted according to the real-time environmental data and the information of the working object, so that the automated operation is not limited to the preset path and operation, but can more flexibly adapt to the changing working environment and operation requirements, thereby improving the practicality of the automation system. After obtaining the image information containing obstacles or working objects around the vehicle, the contour features of the working object in the image information are identified by using the recognition model, which can help the vehicle to find and avoid obstacles in time when performing the working task, and accurately lock the working object, thereby improving the working efficiency and safety. Based on the recognition result, the control strategy is determined, so that the vehicle can flexibly adjust the way of performing the working task according to the actual situation, thereby improving the accuracy of performing the working task. Thereby solving the technical problem of low control accuracy of the vehicle and achieving the technical effect of improving the control accuracy of the vehicle.

[0059] The above method of this embodiment is further introduced below.

[0060] As an optional implementation method, the recognition model includes a contour feature enhancement layer and a standard convolution layer. Step S104, using the recognition model, identifies the contour features of the work object in the image information to obtain a recognition result of the work object, including: in response to the recognition model receiving the image information, inputting the image information into the standard convolution layer, using the standard convolution layer to recognize the image information, and obtaining an initial recognition result, wherein the initial recognition result is used to represent the visual features of the image information; inputting the initial recognition result into the contour feature enhancement layer, using the contour feature enhancement layer, enhancing the initial recognition result to obtain the enhanced initial recognition result, wherein the signal strength of the contour signal corresponding to the contour feature in the enhanced initial recognition result is greater than the signal strength of the contour signal corresponding to the contour feature before the enhancement; determining the recognition result based on the initial recognition result and the enhanced initial recognition result.

[0061] In this embodiment, in the process of using the recognition model to recognize the contour features of the work object in the image information and obtaining the recognition result of the work object, when the recognition model receives the image information, the image information can be input into the standard convolution layer, and the image information can be recognized by using the standard convolution layer to obtain the initial recognition result, that is, the feature map obtained by the standard convolution layer. The initial recognition result can be input into the contour feature enhancement layer, and the initial recognition result can be enhanced by using the contour feature enhancement layer, that is, the obtained feature map is input into the contour feature enhancement layer. The initial recognition result after the enhancement process is obtained. Based on the initial recognition result and the initial recognition result after the enhancement process, the recognition result can be determined. Among them, the signal strength of the contour signal corresponding to the contour feature in the initial recognition result after the enhancement process is greater than the signal strength of the contour signal corresponding to the contour feature before the enhancement process.

[0062] In an embodiment of the present invention, when the recognition model receives image information, the image is input into a standard convolution layer. The standard convolution layer filters the image through a series of convolution kernels to extract basic visual features, such as color, texture, and simple shapes. The output of this layer constitutes a preliminary analysis of the image, that is, an initial recognition result, which can reflect the visual features of the image. The initial recognition result is passed to the contour feature enhancement layer, which can enhance the contour feature representation in the image, highlight the edges and contours of the work object, and increase the signal strength of the feature. Therefore, the recognition model can more accurately identify the work object in the image information, and improve the accuracy and precision of the recognition. The contour features in the image can also be determined more accurately, so as to better complete the recognition task of the work object.

[0063] As an optional implementation method, in response to the recognition model receiving image information, the image information is input into a standard convolution layer, and the image information is recognized using the standard convolution layer to obtain an initial recognition result, including: in response to the standard convolution layer receiving image information, a convolution operation is performed on the image information, weight information and bias information of the standard convolution layer to obtain an initial recognition result; the initial recognition result is input into a contour feature enhancement layer, and the initial recognition result is enhanced using the contour feature enhancement layer to obtain an enhanced initial recognition result, including: in response to the contour feature enhancement layer receiving the initial recognition result, a convolution operation is performed on the initial recognition result, the weight information and bias information of the contour feature enhancement layer to obtain an enhanced initial recognition result.

[0064] In this embodiment, when the recognition model receives image information, inputs the image information into the standard convolution layer, and uses the standard convolution layer to recognize the image information to obtain an initial recognition result, when the standard convolution layer receives the image information, a convolution operation can be performed on the image information, the weight information and the bias information of the standard convolution layer to obtain an initial recognition result. When the initial recognition result is input into the contour feature enhancement layer, the initial recognition result is enhanced by the contour feature enhancement layer to obtain an enhanced initial recognition result, when the contour feature enhancement layer receives the initial recognition result, a convolution operation can be performed on the initial recognition result, the weight information and the bias information of the contour feature enhancement layer to obtain an enhanced initial recognition result.

[0065] For example, before performing the convolution operation on the image information, the weight information and the bias information of the standard convolution layer, they can be initialized based on the heuristic method of the component shape characteristics, as expressed as:

[0066]

[0067] In the formula, Can represent the initial weight matrix; γ p It can represent the shape sensitivity coefficient and adjust the initial weight to enhance edge detection; n p It can represent the number of elements in the convolution kernel; rand() can represent the generation of a random number between 0 and 1.

[0068] For another example, the convolution operation can be performed on the image information, the weight information and the bias information of the standard convolution layer by the following formula:

[0069]

[0070] In the formula, X p Can represent image information; Can represent the weight information (weight) of the standard convolutional layer; It can represent the bias information (bias) of the standard convolutional layer; ReLU() can be used to represent the Rectified Linear Unit (ReLU) activation function; It can represent the feature map of the lth layer, that is, the feature map of the lth layer output by the standard convolutional layer.

[0071] As an optional example, a convolution operation may be performed on the initial recognition result, the weight information of the contour feature enhancement layer, and the bias information by the following formula:

[0072]

[0073] In the formula, The weight information (weight) of the contour feature enhancement layer can be expressed; The bias information (bias) of the contour feature enhancement layer can be represented; It can represent the contour features of the lth layer, that is, it can be a feature map output by the enhanced layer containing the above-mentioned enhanced contour features.

[0074] Optionally, a kernel function is applied to the feature map to enhance the contour signal and obtain a contour feature enhancement layer. Kernel functions, also known as convolution kernels, are used to extract image features. By designing a specific kernel function, specific types of features in an image can be enhanced, such as edges, textures, or contours.

[0075] Optionally, the standard convolution layer can contain multiple convolution kernels, each of which has corresponding weight information and bias information. The convolution operation performs a weighted summation nonlinear transformation on each region by sliding the convolution kernel on the image information. For example, the ReLU activation function can be used to extract basic visual features in the image, such as edge information, texture, color, and shape. Thus, a preliminary recognition result is obtained to represent the visual features of the image information.

[0076] Optionally, the initial recognition result is input to the contour feature enhancement layer, which can enhance the contour and edge features of the work object in the image. The convolution operation performed by the contour feature enhancement layer uses a specific kernel function, which is designed to highlight and enhance the edge and contour signals in the image. Through the convolution operation, the contour feature enhancement layer can extract and enhance the contour features from the initial recognition result to generate an enhanced initial recognition result. The contour features in the image are enhanced and highlighted, thereby improving the model's recognition ability of the work object.

[0077] As an optional implementation method, based on the initial recognition result and the initial recognition result after enhancement processing, the recognition result is determined, including: obtaining weight data corresponding to the initial recognition result after enhancement processing; using the weight data to fuse the initial recognition result and the initial recognition result after enhancement processing to obtain the recognition result.

[0078] In this embodiment, based on the initial recognition result and the initial recognition result after the enhancement processing, in the process of determining the recognition result, the weight data corresponding to the initial recognition result after the enhancement processing can be obtained, and the initial recognition result and the initial recognition result after the enhancement processing are fused using the weight data to obtain the recognition result. The acquisition and fusion operation of the weight data can help the model better utilize the learned information, thereby more accurately determining the recognition result of the operation object. By comprehensively considering the initial recognition result and the result after the enhancement processing, the recognition accuracy and reliability of the operation object can be further improved.

[0079] Optionally, the weight data may be a weight coefficient of the contour feature. The fusion operation may be to fuse the feature map output from the contour feature enhancement layer with the output of the traditional convolution layer, and the fusion strategy may adopt a weighted method based on feature importance.

[0080] Optionally, a weighted method based on the importance of contour features is used to fuse the feature map output by the contour feature enhancement layer with the feature map output by the standard convolutional layer.

[0081] For example, the feature map output by the contour feature enhancement layer and the feature map output by the standard convolutional layer can be fused by the following formula:

[0082]

[0083] In the formula, Can represent the fused feature map; α p It can represent the weight coefficient of the contour feature, which is used to adjust the proportion of the contour and basic features in the final feature map.

[0084] As an optional implementation method, the method also includes: obtaining a convolutional neural network model to be trained, and classification error data and contour error data corresponding to the convolutional neural network model; using the classification error data and contour error data to train the convolutional neural network model to obtain a recognition model.

[0085] In this embodiment, after obtaining the convolutional neural network model to be trained, and the classification error data and contour error data corresponding to the convolutional neural network model, the classification error data and contour error data can be used to train the convolutional neural network model to obtain a recognition model. By training the convolutional neural network model using the classification error data and contour error data, the recognition model can better understand and recognize contour features, thereby improving the recognition accuracy of the work object.

[0086] Optionally, the convolutional neural network model can be a Visual Geometry Group Network (VGG) structure, a Residual Network (ResNet) structure, or a custom convolutional neural network structure, which can be used as a basic model for training. The selection of the above-mentioned convolutional neural network model is only for illustration and is not specifically limited here. As long as it is a convolutional neural network model that can be used for training, it is within the protection scope of the embodiments of the present invention.

[0087] Optionally, during the training of the convolutional neural network model, an improved loss function can be used to consider not only the classification error but also the contour matching error term, which can be used to evaluate the effect of the convolutional neural network model in maintaining the contour features of the component.

[0088] Optionally, the classification error data can represent the error of the recognition model in the classification task, and is used to guide the learning process of the model to minimize the classification error. It can be obtained by calculating functions such as cross entropy loss and multi-classification loss. The contour error data can represent the error of the recognition model in contour feature recognition, and can enhance the contour recognition ability of the recognition model. It can be calculated by comparing the difference between the contour predicted by the recognition model and the real contour manually annotated, for example, using the intersection over union (IoU) indicator or the edge matching loss function.

[0089] It should be noted that the above-mentioned methods for obtaining classification error data and contour error data are only examples and are not specifically limited here. As long as the error data can be used for convolutional neural network model training, it is within the protection scope of the embodiments of the present invention.

[0090] As an optional implementation method, the recognition result sample includes a category recognition sample, and the category recognition sample corresponds to the classification error data, wherein obtaining the classification error data corresponding to the convolutional neural network model includes: obtaining the category recognition sample output by the convolutional neural network model based on the image information sample, and the real category sample corresponding to the image information sample; using the category recognition sample and the real category sample to determine the classification error data.

[0091] In this embodiment, in the process of obtaining the classification error data corresponding to the convolutional neural network model, the category recognition sample output by the convolutional neural network model based on the image information sample can be obtained, that is, the predicted label can be used i To represent, and the real category samples corresponding to the image information samples, that is, the real labels, can be used Representation; using category recognition samples and true category samples, the classification error data can be determined It can also be called classification error. The recognition result samples include category recognition samples, and the category recognition samples correspond to the classification error data.

[0092] Optionally, the category recognition samples, also called predicted labels, can be obtained by inputting the image information samples into a trained convolutional neural network model. It can include the category labels predicted by the recognition model for each image information sample. That is, what are the specific parts of the vehicle determined based on the contour. The real category samples, also called real labels, can be used to evaluate the performance of the recognition model and the benchmark real category labels for calculating errors, that is, the real categories corresponding to the image information samples provided in the labeled data.

[0093] In the embodiment of the present invention, by obtaining the category recognition samples and the true category samples output by the convolutional neural network model and comparing the two to determine the classification error data, it can help evaluate the accuracy and performance of the recognition model in the category recognition task. The above data can provide guidance for further optimizing the recognition model and help improve the classification accuracy and recognition ability of the recognition model.

[0094] As an optional implementation method, the recognition result sample includes a contour recognition sample, and the contour recognition sample corresponds to the contour error data, wherein obtaining the contour error data corresponding to the convolutional neural network model includes: obtaining a standard convolution layer in the convolutional neural network model, based on an initial recognition result sample output by the image information sample, and a contour feature enhancement layer in the convolutional neural network model, based on a contour recognition sample output by the initial recognition result sample; using the initial recognition result sample and the contour recognition sample to determine the contour error data.

[0095] In this embodiment, in the process of obtaining the contour error data corresponding to the convolutional neural network model, the standard convolution layer in the convolutional neural network model can be obtained, and the initial recognition result sample output based on the image information sample And the contour feature enhancement layer in the convolutional neural network model, based on the contour recognition samples output by the initial recognition result samples The initial recognition result sample and the contour recognition sample can be used to determine the contour error data It can also be called contour error.

[0096] Optionally, the initial recognition result sample can be obtained by inputting the image information sample into the standard convolution layer of the convolutional neural network model, and can represent the result obtained after the recognition model recognizes the image information sample. The contour recognition sample can be obtained by inputting the initial recognition result sample into the contour feature enhancement layer of the convolutional neural network model, and can represent the result obtained after the recognition model recognizes the contour feature. By comparing the initial recognition result sample and the contour recognition sample, the contour error data can be determined, and the above data can be used to evaluate the recognition accuracy and performance of the recognition model for the contour feature.

[0097] In an embodiment of the present invention, by obtaining the initial recognition result samples and contour recognition samples output by the convolutional neural network model, and determining the contour error data by comparing the two, it can help evaluate the accuracy and performance of the recognition model in the contour feature recognition task, and improve the recognition model's recognition ability and accuracy for contour features.

[0098] As an optional implementation method, the convolutional neural network model is trained using classification error data and contour error data to obtain a recognition model, including: determining the loss function of the convolutional neural network model based on the classification error data and the contour error data; using the loss function to train the convolutional neural network model to obtain a trained initial recognition model; in response to the iterative result corresponding to the recognition result sample of the initial recognition model, the difference information between the iterative result and the previous iterative result is less than the difference information threshold, and determining the recognition model based on the initial recognition model, wherein the difference information is used to indicate the degree of difference between the iterative result and the previous iterative result.

[0099] In this embodiment, the convolutional neural network model is trained using the classification error data and the contour error data. In the process of obtaining the recognition model, the loss function of the convolutional neural network model can be determined based on the classification error data and the contour error data; the convolutional neural network model can be trained using the loss function to obtain the trained initial recognition model; the recognition model can be determined based on the initial recognition model when the difference information between the iteration result corresponding to the recognition result sample of the initial recognition model and the previous iteration result is less than the difference information threshold. The difference information can be used to indicate the degree of difference between the iteration result and the previous iteration result, that is, it can be used to indicate the degree of change between the feature graphs generated by the two iteration processes, which can also be called the feature graph change rate.

[0100] Optionally, the effect of the convolutional neural network model on maintaining the contour features of the component can be evaluated by guiding the optimization of the convolutional neural network model parameters through a loss function. The convolutional neural network model can be trained using a determined loss function. During the training process, the convolutional neural network model continuously adjusts parameters according to the classification error data and the contour error data, minimizes the loss function to reduce the loss function, and obtains a trained initial recognition model with better recognition ability and performance. Through continuous iterative training, in response to the difference information between the iterative result corresponding to the recognition result sample of the initial recognition model and the previous iterative result being less than the set difference information threshold, the convolutional neural network model can be gradually optimized until the stopping condition is met. For example, the rate of change of the feature map in several consecutive iterations can be compared to determine whether all important contour features have been stably learned.

[0101] For example, the loss function can be determined by the following formula:

[0102]

[0103] In the formula, can represent the loss function; N can represent the number of samples; l() can represent the classification error; l cf () can represent the contour matching error; y i and can represent the true label and the predicted label respectively; λ p It can represent the coefficient for adjusting the classification error and contour matching error; W p Can represent the updated weight; η p It can represent the learning rate.

[0104] For example, the difference between the results of the previous iteration and this iteration can be expressed as:

[0105]

[0106] Where N can represent the number of samples; t and t-1 can represent the current and previous iterations respectively; It can represent the feature map obtained by the iterative process from 1 to t. It can represent the feature map obtained by the iterative process from 1 to t-1; ΔM p It can represent the rate of change of feature maps.

[0107] If ΔM p <∈ p , that is, the difference information ΔM between the iteration result corresponding to the recognition result sample of the initial recognition model and the previous iteration result p Less than the set difference information threshold ∈ p , then the iteration stops. Among them, ∈ pCan represent a stability threshold.

[0108] In the embodiment of the present invention, the difference information is used to indicate the degree of difference between the iteration result and the previous iteration result, and helps evaluate the convergence and performance of the convolutional neural network model training. The convolutional neural network model is effectively trained and optimized through classification error data and contour error data to obtain a recognition model with better performance. Iterative training can help the model to continuously optimize and improve its recognition accuracy of the work object.

[0109] As an optional implementation method, the control instructions include driving instructions and grabbing instructions, and the work tasks include grabbing tasks and transportation tasks corresponding to the work objects. Step S108 responds to the control instructions corresponding to the control strategy, controls the vehicle to be at the target work position, and performs the work task on the work object, including: responding to the driving path corresponding to the driving instructions, controlling the vehicle to drive to the target work position; at the target work position, responding to the grabbing instructions, and performing the grabbing task on the work object; in response to the vehicle completing the grabbing task, responding to the driving path, and performing the transportation task on the work object.

[0110] In this embodiment, in the process of controlling the vehicle to perform the work task on the work object at the target work position according to the control instructions corresponding to the control strategy, the vehicle can be controlled to travel to the target work position according to the driving path corresponding to the driving instructions; at the target work position, the grabbing task can be performed on the work object according to the grabbing instructions; when the vehicle completes the grabbing task, the transportation task can be performed on the work object according to the driving path. Among them, the control instructions include driving instructions and grabbing instructions, and the work tasks include grabbing tasks and transportation tasks corresponding to the work object.

[0111] Optionally, when the vehicle receives a driving instruction, it can start moving according to the driving path information contained in the instruction using an onboard navigation system, such as laser navigation, visual navigation, or magnetic navigation. The navigation system continuously monitors the position of the vehicle and compares it with the coordinates of the target working position, and adjusts the direction and speed of the vehicle in real time to ensure that the vehicle can accurately drive to the target working position along the given path.

[0112] Optionally, after the vehicle reaches the target work location, the vehicle's control system will parse the grasping instructions, which may include parameters such as the specific location of the work object, the choice of grasping tools, and the grasping force and angle. The robotic arm or other grasping device on the vehicle is activated according to the parameters, accurately locates the work object, and then performs the grasping action. The robotic arm can use a vacuum suction cup, a gripper, or other suitable tools to grasp the work object stably in the most appropriate way according to its shape and weight.

[0113] Optionally, when the grasping task is successfully completed, the vehicle will respond to the driving path instruction again and start the transportation task. The vehicle will move the work object to the designated storage location, assembly line or boxing area. During the whole process, the vehicle will continue to monitor the surrounding environment using sensors, such as LiDAR for obstacle detection, to ensure that there will be no collisions or accidents during transportation. The vehicle's navigation system will also dynamically adjust the path based on real-time environmental information to deal with emergencies, such as the appearance of temporary obstacles.

[0114] In an embodiment of the present invention, when a vehicle performs a work task in a target area, the environment around the vehicle can be monitored in real time to obtain image information containing obstacles around the vehicle or work objects. The contour features of the work object in the image information can be identified using a recognition model to obtain a recognition result of the work object. Based on the recognition result, a control strategy for instructing the vehicle to perform the work task while avoiding obstacles can be determined for the work task to be performed on the work object. According to the control instructions corresponding to the control strategy, the vehicle can be controlled at the target work position to perform the work task on the work object.

[0115] In this embodiment, in order to improve the accuracy of the identification of the working object, the contour features of the working object are learned and mastered through the trained recognition model, so that the target object can be more reliably identified even under partial occlusion or poor light conditions, thereby improving the accuracy of the identification. In order to enhance the adaptability to the environment, the image information of the surrounding environment is obtained during the vehicle's execution of the working task, including the detection of obstacles, so that a quick response can be made to avoid collisions and ensure the safe operation of the operation. In order to improve the flexibility of the operation, the operation plan is dynamically adjusted according to the real-time environmental data and the information of the working object, so that the automated operation is not limited to the preset path and operation, but can more flexibly adapt to the changing working environment and operation requirements, thereby improving the practicality of the automation system. After obtaining the image information containing obstacles or working objects around the vehicle, the contour features of the working object in the image information are identified by using the recognition model, which can help the vehicle to find and avoid obstacles in time when performing the working task, and accurately lock the working object, thereby improving the working efficiency and safety. Based on the recognition result, the control strategy is determined, so that the vehicle can flexibly adjust the way of performing the working task according to the actual situation, thereby improving the accuracy of performing the working task. Thereby solving the technical problem of low control accuracy of the vehicle and achieving the technical effect of improving the control accuracy of the vehicle.

[0116] The following uses an unmanned pallet truck performing an operation task in a warehouse as an example to illustrate the technical solution of the embodiment of the present invention.

[0117] At present, in warehouse management technology, the identification and handling of goods are mainly achieved through technical means such as barcode scanning and RFID, but there are still some limitations. The relevant image recognition technology or barcode scanning system can only provide basic item recognition functions, and it is difficult to handle items with complex or similar appearances, especially when the barcode or label cannot be clearly seen. The adaptability of the relevant automation system is poor, especially when facing environmental changes (such as light changes, changes in cargo stacking density), the stability and reliability of the automation system are often challenged. There is often a lack of flexibility in task scheduling and path planning, resulting in low efficiency of unmanned pallet trucks or robots in performing tasks, and there may be inefficiencies such as roundabout transportation and frequent obstacle avoidance. In summary, the existing warehouse automation technology has obvious deficiencies in handling complex environments and item recognition, and it is difficult to quickly adapt to technological advances, limiting its application potential in efficient and flexible warehouse management systems. Therefore, there is also a technical problem of low vehicle control accuracy.

[0118] In the embodiment of the present invention, by adopting the intelligent recognition method based on the monitoring device, it is specifically deployed in the automated operation system of the unmanned pallet truck. By adding an improved convolutional neural network to the convolutional neural network model, especially adding a contour feature enhancement layer to improve the standard convolutional layer, the improved convolutional neural network model is used for recognition. The recognition accuracy of goods such as auto parts is effectively improved, so that the system can accurately identify objects with complex shapes, thereby improving the recognition accuracy. The operation of the unmanned pallet truck is guided by data obtained by a variety of sensors, including obstacle avoidance and cargo handling. By dynamically adjusting task allocation and path planning, the automated operation system significantly improves the overall operating efficiency. The central control system dynamically adjusts task allocation according to real-time recognition results and vehicle status information, reduces invalid operation and waiting time, and improves resource utilization. The execution results are continuously received through the feedback loop, and the system status and operation strategy are updated accordingly to ensure adaptation to the ever-changing operating environment and needs. Thereby solving the technical problem of low vehicle control accuracy and achieving the technical effect of improving vehicle control accuracy.

[0119] The following takes the storage goods intelligent identification method of the monitoring equipment as an example to further introduce the embodiment of the present invention.

[0120] The present invention proposes a method for intelligent identification of stored goods based on monitoring equipment. The method is deployed in the automated operating system of an unmanned pallet truck, including an on-board processor, multiple sensors such as lidar and cameras, a control execution unit, and a central control system. The automated operating system integrates multi-sensor data, utilizes an improved convolutional neural network structure, and adds a contour feature enhancement layer to improve the recognition accuracy of goods such as auto parts. The automated operating system can dynamically adjust tasks and path planning according to real-time data, optimize operational accuracy, and continuously update and optimize the performance of the automated operating system through a feedback loop after completing a task, which not only improves the accuracy of recognition and operation, but also significantly improves the automation level of warehouse management.

[0121] Among them, the on-board processor can be used to receive and process data from sensors, and execute algorithms for obstacle recognition and avoidance, cargo recognition, and precise operations. A variety of sensors, including lidar and cameras, can be used to monitor the surrounding environment and identify cargo in real time. The control execution unit can perform actual mechanical operations, such as driving the unmanned vehicle to avoid obstacles, and grabbing and placing cargo. The central control system can be used to coordinate the operations of multiple unmanned pallet trucks, ensure the rationality of task allocation, and optimize path planning.

[0122] In the embodiment of the present invention, Figure 4 FIG. 1 is a flow chart of an unmanned pallet truck automated operation based on an intelligent identification method for stored goods using monitoring equipment according to an embodiment of the present invention. Figure 4 As shown, the following steps are included:

[0123] Step 401, collect environment and cargo data.

[0124] In this embodiment, the vehicle-mounted sensors continuously collect data about the surrounding environment and the cargo to be processed, such as the location of obstacles, the size and location of the cargo, etc. For example, the camera monitoring device is used to collect image data, and the collected data mainly includes images of various auto parts and cargo. After collection, the images are stored in a structured data format, specifically in jpg format. On this basis, the data is manually labeled to ensure that each piece of data has a clear labeling category for subsequent training and feature extraction. In this embodiment, Figure 2 FIG. 1 is a schematic diagram of image data according to an embodiment of the present invention. Figure 2 As shown, for this image data, the category can be labeled as "clutch pressure plate".

[0125] Step 402, perform data processing and tasks.

[0126] In this embodiment, after receiving the data, the onboard processor uses the pre-trained convolutional neural network model to perform obstacle recognition and obstacle avoidance decisions, and at the same time recognizes the goods and determines the pick-and-place strategy that meets the requirements. The processing results will guide the control execution unit to perform corresponding operations.

[0127] Although the related convolutional neural network model performs well in the field of image processing, for specific automotive parts images, only using standard convolution layers may not be able to fully capture the detailed features of the parts, especially the edge and contour information. Therefore, the present invention proposes an improved convolutional neural network structure, which adds a contour feature enhancement layer and uses a specific kernel function in conjunction with the standard convolution kernel to highlight the edges and contours of parts in the feature map, thereby better capturing the shape information that is critical to classification.

[0128] In this embodiment, the image processing and training process of the improved convolutional neural network is as follows:

[0129] Optionally, the parameters of the convolutional neural network model are initialized based on a heuristic method based on component shape features. Considering the geometric and structural characteristics of automotive parts, the initialization strategy is initialized using a heuristic method based on component shape features, which is expressed as:

[0130]

[0131] In the formula, Can represent the initial weight matrix; γ p It can represent the shape sensitivity coefficient and adjust the initial weight to enhance edge detection; n p It can represent the number of elements in the convolution kernel; rand() can represent the generation of a random number between 0 and 1.

[0132] Optionally, the input warehouse cargo image passes through a standard convolution layer to extract basic visual features, and then enters a contour feature enhancement layer. In the contour feature enhancement layer, a kernel function is used to act on the feature map to enhance the representation of the contour signal. The processing method is expressed as follows:

[0133]

[0134] In the formula, It can represent the feature map of the lth layer, that is, the feature map of the lth layer output by the standard convolutional layer; and They can represent the weight and bias of the standard convolutional layer, respectively. * can represent the convolution operation; ReLU() can represent the ReLU activation function; and It can represent the weights and biases of the contour feature enhancement layer; It can represent the contour features of the lth layer, that is, it can be a feature map output by the enhanced layer containing the above-mentioned enhanced contour features.

[0135] The kernel function usually refers to the filter or convolution kernel used for convolution operation, which is used to slide on the image and perform local operations to highlight or enhance certain features of the image, such as edges, textures, directions, etc. Kernel functions can be designed in various shapes and sizes to meet different feature extraction requirements. The data features input to the lth layer represent the data feature image input to the lth layer, which contains the visual information of the stored goods. After the above feature map is extracted from the basic visual features by the convolution layer, it is used as the input of the contour feature enhancement layer.

[0136] Figure 3 FIG. 1 is a schematic diagram of a feature map extraction process of a warehouse cargo image according to an embodiment of the present invention. Figure 3 As shown in the figure, the feature map of the warehouse goods image can be extracted through the convolution kernel.

[0137] Optionally, the feature map (feature Figure 2 ), and the feature map (feature Figure 1 ), and the fusion strategy adopts a weighted method based on feature importance, considering the importance of contour features. The fusion method is expressed as:

[0138]

[0139] In the formula, Can represent the fused feature map; α p It can represent the weight coefficient of the contour feature, which is used to adjust the proportion of the contour and basic features in the final feature map.

[0140] Optionally, an improved loss function is used to consider not only the classification error but also the contour matching error term to evaluate the effect of the convolutional neural network model on maintaining the contour features of the components. Furthermore, the parameter update not only uses the gradient descent method but also combines the adaptive adjustment strategy of the component features to better adapt to the characteristics of various components. The calculation method of the loss function including the contour matching error is expressed as:

[0141]

[0142] In the formula, can represent the loss function; N can represent the number of samples; l() can represent the classification error; l cf () can represent the contour matching error; y i and can represent the true label and the predicted label respectively; λ pIt can represent the coefficient for adjusting the classification error and contour matching error; W p Can represent the updated weight; η p It can represent the learning rate.

[0143] Among them, classification error is a measure of the inconsistency between the predicted category and the actual category of the convolutional neural network model, and is usually used in classification tasks, such as image classification, text classification, etc. In deep learning, especially when convolutional neural networks are used for image classification, classification error is a loss function. It also includes cross-entropy loss (Cross-Entropy Loss), mean squared error (Mean Squared Error, applicable to the case of regression conversion classification), etc. Classification error is a commonly used evaluation indicator in machine learning and deep learning, which is used to measure the difference between the predicted results and the true labels of the classification model when performing classification tasks. Specifically, the classification error indicates the degree of inconsistency between the predicted label and the true label when the classification model makes a classification prediction for a set of samples, which can be expressed as the complement of the error rate or accuracy rate.

[0144] Among them, the loss function of the contour matching error can control the convolutional neural network model to better learn and maintain the contour features of the input data, improve the effect of the convolutional neural network model in maintaining the contour features of the parts, and further improve the performance and expression of the convolutional neural network model in image processing tasks.

[0145] Optionally, after a certain number of iterations, a feature stability test is performed to determine whether all important contour features are stably learned by comparing the rate of change of feature maps in consecutive iterations, which can be expressed as:

[0146]

[0147] Where N can represent the number of samples; t and t-1 can represent the current and previous iterations respectively; It can represent the feature map obtained by the iterative process from 1 to t. It can represent the feature map obtained by the iterative process from 1 to t-1; ΔM p It can represent the rate of change of feature maps.

[0148] If ΔM p <∈ p , that is, the difference information ΔM between the iteration result corresponding to the recognition result sample of the initial recognition model and the previous iteration result p Less than the set difference information threshold ∈ p , then the iteration stops. Among them, ∈ p Can represent a stability threshold.

[0149] In this embodiment, after the convolutional neural network model is trained, the trained convolutional neural network model can be used to perform warehouse cargo image recognition and classification.

[0150] Step 403: perform dynamic task adjustment.

[0151] In this embodiment, the vehicle processor sends the image recognition category as an instruction to the central control system, which dynamically adjusts task allocation and path planning based on the status information returned by each vehicle processor to optimize the overall operation accuracy. The system re-evaluates and optimizes the scheduling strategy at regular intervals.

[0152] Step 404: Feedback the task execution result to the central control system.

[0153] In this embodiment, after the operation is completed, the unmanned pallet truck can feed back the task execution results to the central control system, and the central control system updates the system status according to the results and provides data support for subsequent operations.

[0154] In summary, based on the standard convolutional neural network initialization strategy, the present invention takes into account the special structure of automobile parts and adopts a heuristic method based on the shape features of automobile parts for weight initialization. The method may include presetting certain weights to preferentially detect and identify edges or specific shapes, so that it can be sensitive to shapes and edges at the beginning of training, and improve the recognition ability of complex parts. After the training is completed, the convolutional neural network model is used for actual warehouse cargo image recognition, which can quickly classify according to the image content and provide real-time guidance for the operation of unmanned pallet trucks. The precision and accuracy of automobile parts recognition are improved, especially when handling goods with complex shapes or similar appearances. Through the contour feature enhancement layer, weighted feature fusion and improved loss function, it is ensured that the convolutional neural network model can accurately capture and identify the edges and contours of the parts. Combined with the dynamic adjustment of tasks and path planning of the central control system, as well as the continuous feedback optimization mechanism, a complete and intelligent warehouse recognition and operation system is formed. Thereby solving the technical problem of low control accuracy of the vehicle and achieving the technical effect of improving the control accuracy of the vehicle.

[0155] Figure 5 Schematic diagram of a vehicle control system according to an embodiment of the present invention. Figure 5 As shown, the control system 500 of the vehicle may include: an image acquisition unit 502 , a processing unit 504 and a control execution unit 506 .

[0156] The image acquisition unit 502 may be a sensor, which may acquire image information around the vehicle when the vehicle is performing an operation task in a target area.

[0157] The processing unit 504 may be a vehicle-mounted processor, which uses a recognition model to recognize the contour features of the working object in the image information to obtain a recognition result of the working object; based on the recognition result, a control strategy for the working task to be performed by the working object is determined.

[0158] The control execution unit 506 can respond to the control instructions corresponding to the control strategy, control the vehicle to perform the operation task on the operation object at the target operation position.

[0159] The above method of this embodiment is further introduced below.

[0160] As an optional implementation method, the control system 500 can be connected to a central control system, which is used to assign corresponding work tasks to the control systems of multiple vehicles corresponding to the multiple vehicles, and plan driving paths in the target area for the control systems of the multiple vehicles.

[0161] In this embodiment, the central control system can formulate intelligent cargo handling strategies and path planning based on the recognition results and other sensor data. The control system can convert the strategy into specific operations to ensure that the cargo is safely and efficiently identified and handled. The central control system can significantly improve the operating efficiency and safety of the automated warehouse through intelligent decision-making, real-time monitoring and efficient scheduling, while achieving reasonable allocation of resources and continuous optimization of performance.

[0162] In the control system of the above-mentioned vehicle of the present invention, the surrounding environment information is obtained by the image acquisition unit 502, the processing unit 504 uses the recognition model to identify the working object, and then determines the control strategy according to the recognition result, and finally the control execution unit 506 executes the working task on the working object and connects with the central control system, which can realize the collaborative operation between vehicles and improve the accuracy of the working task, thereby solving the technical problem of low control accuracy of the vehicle and achieving the technical effect of improving the control accuracy of the vehicle.

[0163] According to an embodiment of the present invention, a vehicle control device is also provided. It should be noted that the control device of the active suspension in the vehicle can be used to execute the vehicle control method in the above embodiment.

[0164] Figure 6 is a schematic diagram of a vehicle control device according to an embodiment of the present invention. Figure 6 As shown, the vehicle control device 600 may include:

[0165] The acquisition unit 602 is used to acquire image information around the vehicle when the vehicle performs a work task in the target area, wherein the image content of the image information at least includes obstacles around the vehicle and a work object corresponding to the work task.

[0166] The recognition unit 604 is used to use the recognition model to recognize the contour features of the work object in the image information to obtain the recognition result of the work object, wherein the recognition model is obtained by training a convolutional neural network model using image information samples containing the work object samples and recognition result samples of the work object samples, and the recognition result samples are obtained by classifying the work object samples according to the contour feature samples in the image information samples.

[0167] The determination unit 606 is used to determine a control strategy for the work task to be performed by the work object based on the recognition result, wherein the control strategy is used to instruct the vehicle to perform the work task while avoiding obstacles.

[0168] The execution unit 608 is used to respond to the control instruction corresponding to the control strategy, control the vehicle to perform the operation task on the operation object at the target operation position.

[0169] Optionally, the recognition unit 604 may include: a first acquisition module, used to input the image information into a standard convolution layer in response to the recognition model receiving image information, and use the standard convolution layer to recognize the image information to obtain an initial recognition result, wherein the initial recognition result is used to represent the visual features of the image information; a second acquisition module, used to input the initial recognition result into a contour feature enhancement layer, and use the contour feature enhancement layer to enhance the initial recognition result to obtain the enhanced initial recognition result, wherein the signal strength of the contour signal corresponding to the contour feature in the enhanced initial recognition result is greater than the signal strength of the contour signal corresponding to the contour feature before the enhancement; a first determination module, used to determine the recognition result based on the initial recognition result and the enhanced initial recognition result.

[0170] Optionally, the device may also include: a third acquisition module, used to perform a convolution operation on the image information, the weight information and the bias information of the standard convolution layer in response to the standard convolution layer receiving the image information, so as to obtain an initial recognition result; a fourth acquisition module, used to perform a convolution operation on the initial recognition result, the weight information and the bias information of the contour feature enhancement layer in response to the contour feature enhancement layer receiving the initial recognition result, so as to obtain the initial recognition result after enhancement processing.

[0171] Optionally, the above-mentioned first determination module may include: a first acquisition submodule, used to obtain weight data corresponding to the initial recognition result after enhancement processing; a fusion submodule, used to use the weight data to fuse the initial recognition result and the initial recognition result after enhancement processing to obtain a recognition result.

[0172] Optionally, the device may also include: a fifth acquisition module, used to obtain the convolutional neural network model to be trained, and the classification error data and contour error data corresponding to the convolutional neural network model; a first training module, used to train the convolutional neural network model using the classification error data and contour error data to obtain a recognition model.

[0173] Optionally, the fifth acquisition module may include: a second acquisition sub-module, used to obtain category recognition samples output by the convolutional neural network model based on the image information samples, and real category samples corresponding to the image information samples; a determination sub-module, used to determine classification error data using the category recognition samples and the real category samples.

[0174] Optionally, the device may also include: a sixth acquisition module, used to acquire the initial recognition result samples output by the standard convolution layer in the convolutional neural network model based on the image information samples, and the contour feature enhancement layer in the convolutional neural network model, based on the contour recognition samples output by the initial recognition result samples; a second determination module, used to determine the contour error data using the initial recognition result samples and the contour recognition samples.

[0175] Optionally, the device may also include: a third determination module, used to determine the loss function of the convolutional neural network model based on the classification error data and the contour error data; a second training module, used to train the convolutional neural network model using the loss function to obtain a trained initial recognition model; a fourth determination module, used to determine the recognition model based on the initial recognition model in response to the iterative result corresponding to the recognition result sample of the initial recognition model, and the difference information between the iterative result and the previous iterative result is less than the difference information threshold, wherein the difference information is used to indicate the degree of difference between the iterative result and the previous iterative result.

[0176] Optionally, the determination unit 606 may include: a control module, used to respond to the driving path corresponding to the driving instruction, and control the vehicle to drive to the target work position; a first execution module, used to respond to the grabbing instruction at the target work position, and perform the grabbing task on the work object; a second execution module, used to respond to the driving path in response to the vehicle completing the grabbing task, and perform the transportation task on the work object.

[0177] In an embodiment of the present invention, the acquisition unit 602 acquires image information around the vehicle during the process of the vehicle performing the operation task in the target area, wherein the image content of the image information at least includes obstacles around the vehicle and the operation object corresponding to the operation task. The recognition unit 604 uses the recognition model to recognize the contour features of the operation object in the image information to obtain the recognition result of the operation object, wherein the recognition model is obtained by training the convolutional neural network model using the image information sample containing the operation object sample and the recognition result sample of the operation object sample, and the recognition result sample is obtained by classifying the operation object sample according to the contour feature sample in the image information sample. The determination unit 606 determines the control strategy for the operation task to be performed on the operation object based on the recognition result, wherein the control strategy is used to instruct the vehicle to perform the operation task while avoiding obstacles. The execution unit 608 responds to the control instruction corresponding to the control strategy, controls the vehicle at the target operation position, and performs the operation task on the operation object. Thus, the technical problem of low control accuracy of the vehicle is solved, and the technical effect of improving the control accuracy of the vehicle is achieved.

[0178] According to an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein the vehicle control method in the above embodiment is executed when the program is run.

[0179] According to an embodiment of the present invention, a computer-readable storage medium is further provided. The storage medium includes a stored program, wherein the program executes the vehicle control method in the above embodiment.

[0180] The embodiment of the present application further provides a computer program product. Optionally, in the present embodiment, the computer program product may include a computer program, and when the computer program is executed by a processor, the vehicle control method of the embodiment of the present application is implemented.

[0181] According to an embodiment of the present invention, a vehicle is further provided. The vehicle is used to execute the vehicle control method according to the embodiment of the present invention.

[0182] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0184] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0185] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0186] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, Read-Only Memory (ROM), Random Access Memory (RAM), mobile hard disk, magnetic disk or optical disk, etc., various media that can store program codes.

[0187] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A vehicle control method, characterized in that: include: When the vehicle performs a work task in a target area, image information around the vehicle is acquired, wherein the image content of the image information at least includes obstacles around the vehicle and a work object corresponding to the work task; Using a recognition model, the contour features of the work object in the image information are recognized to obtain a recognition result of the work object, wherein the recognition model is obtained by training a convolutional neural network model using an image information sample containing a work object sample and a recognition result sample of the work object sample, and the recognition result sample is obtained by classifying the work object sample according to the contour feature sample in the image information sample; Based on the recognition result, determining a control strategy for the operation task to be performed by the operation object, wherein the control strategy is used to instruct the vehicle to perform the operation task while avoiding the obstacle; In response to the control instruction corresponding to the control strategy, the vehicle is controlled to perform the working task on the working object at the target working position.

2. The method according to claim 1, characterized in that The recognition model includes a contour feature enhancement layer and a standard convolution layer, wherein the contour feature of the work object in the image information is recognized by using the recognition model to obtain a recognition result of the work object, including: In response to the recognition model receiving the image information, the image information is input into the standard convolution layer, and the image information is recognized by using the standard convolution layer to obtain an initial recognition result, wherein the initial recognition result is used to represent the visual features of the image information; Inputting the initial recognition result into the contour feature enhancement layer, and using the contour feature enhancement layer to enhance the initial recognition result to obtain the enhanced initial recognition result, wherein the signal strength of the contour signal corresponding to the contour feature in the enhanced initial recognition result is greater than the signal strength of the contour signal corresponding to the contour feature before the enhanced process; The recognition result is determined based on the initial recognition result and the initial recognition result after enhancement processing.

3. The method according to claim 2, characterized in that In response to the recognition model receiving the image information, inputting the image information into the standard convolution layer, and using the standard convolution layer to recognize the image information to obtain an initial recognition result, including: In response to the standard convolution layer receiving the image information, performing a convolution operation on the image information, the weight information and the bias information of the standard convolution layer to obtain the initial recognition result; Inputting the initial recognition result into the contour feature enhancement layer, and using the contour feature enhancement layer to enhance the initial recognition result to obtain the enhanced initial recognition result, including: In response to the contour feature enhancement layer receiving the initial recognition result, a convolution operation is performed on the initial recognition result, the weight information and the bias information of the contour feature enhancement layer to obtain the enhanced initial recognition result.

4. The method according to claim 2, characterized in that: Determining the recognition result based on the initial recognition result and the initial recognition result after enhancement processing includes: Obtaining weight data corresponding to the initial recognition result after enhancement processing; The weight data is used to fuse the initial recognition result and the initial recognition result after enhancement processing to obtain the recognition result.

5. The method according to claim 1, characterized in that The method further comprises: Obtaining the convolutional neural network model to be trained, and classification error data and contour error data corresponding to the convolutional neural network model; The convolutional neural network model is trained using the classification error data and the contour error data to obtain the recognition model.

6. The method according to claim 5, characterized in that The recognition result samples include category recognition samples, and the category recognition samples correspond to the classification error data, wherein obtaining the classification error data corresponding to the convolutional neural network model includes: Obtain the category recognition sample output by the convolutional neural network model based on the image information sample, and the real category sample corresponding to the image information sample; The classification error data is determined using the category recognition samples and the true category samples.

7. The method according to claim 5, characterized in that The recognition result sample includes a contour recognition sample, and the contour recognition sample corresponds to the contour error data, wherein obtaining the contour error data corresponding to the convolutional neural network model includes: Obtaining a standard convolutional layer in the convolutional neural network model, based on an initial recognition result sample output by the image information sample, and a contour feature enhancement layer in the convolutional neural network model, based on the contour recognition sample output by the initial recognition result sample; The contour error data is determined using the initial recognition result sample and the contour recognition sample.

8. The method according to claim 5, characterized in that The convolutional neural network model is trained using the classification error data and the contour error data to obtain the recognition model, including: Determining a loss function of the convolutional neural network model based on the classification error data and the contour error data; Using the loss function, the convolutional neural network model is trained to obtain a trained initial recognition model; In response to the iterative result corresponding to the recognition result sample of the initial recognition model, the difference information between the iterative result and the previous iterative result is less than the difference information threshold, and the recognition model is determined based on the initial recognition model, wherein the difference information is used to indicate the degree of difference between the iterative result and the previous iterative result.

9. The method according to claim 1, characterized in that: The control instructions include driving instructions and grabbing instructions, and the operation tasks include grabbing tasks and transportation tasks corresponding to the operation objects. In response to the control instructions corresponding to the control strategy, the vehicle is controlled to perform the operation tasks on the operation objects at the target operation position, including: In response to the driving path corresponding to the driving instruction, control the vehicle to drive to the target working position; At the target operation position, responding to the grasping instruction, executing the grasping task on the operation object; In response to the vehicle completing the grasping task and in response to the driving path, the transport task is performed on the work object.

10. A vehicle control system, characterized in that: include: An image acquisition unit, used to acquire image information around the vehicle when the vehicle performs an operation task in a target area, wherein the image content of the image information at least includes obstacles around the vehicle and an operation object corresponding to the operation task; A processing unit is used to identify the contour features of the working object in the image information by using a recognition model to obtain a recognition result of the working object, wherein the recognition model is obtained by training a convolutional neural network model using image information samples containing working object samples and recognition result samples of the working object samples, and the recognition result samples are obtained by classifying the working object samples according to the contour feature samples in the image information samples; based on the recognition result, determine a control strategy for the working task to be performed by the working object, wherein the control strategy is used to instruct the vehicle to perform the working task while avoiding the obstacle; The control execution unit is used to respond to the control instruction corresponding to the control strategy, control the vehicle to perform the operation task on the operation object at the target operation position.

11. The system according to claim 10, characterized in that: The control system of the vehicle is connected to the central control system, which is used to assign corresponding work tasks to the control systems of the multiple vehicles corresponding to the multiple vehicles, and plan driving paths in the target area for the control systems of the multiple vehicles.

12. A processor, characterized in that: The processor is used to run a program, wherein the program, when run by the processor, executes the vehicle control method according to any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the vehicle control method according to any one of claims 1 to 9.

14. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the vehicle control method according to any one of claims 1 to 9.

15. A vehicle, characterized in that: Used to execute the vehicle control method as claimed in any one of claims 1 to 9.

Images