Information processing method and device, paving equipment and storage medium
The camera obtains user face images and generates application authorization information, which solves the problem of inconvenient and easy loss of the remote control of the photovoltaic panel paving equipment, and realizes the safe, convenient operation and efficient task execution of the equipment.
Patent Information
- Application Number
- CN202411189237.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-28
Smart Images

Figure CN118736176B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of control technology, and in particular to an information processing method and device, paving equipment and storage medium. Background Art
[0002] Photovoltaic is the abbreviation of solar photovoltaic power generation system, which is a new power generation system that uses the photovoltaic effect of solar cell semiconductor materials to directly convert solar radiation energy into electrical energy. Solar photovoltaic panel components are a power generation device that generates direct current when exposed to sunlight. Photovoltaic components can be made into different shapes, and the components can be connected to each other to generate more electricity.
[0003] At present, a special remote controller is used to control the operation of the paving equipment on the photovoltaic panels. The special remote controller is inconvenient to carry, and if it fails or is lost, the paving equipment cannot be used. Summary of the invention
[0004] The embodiments of the present application provide an information processing method and device, paving equipment and storage medium, which can authorize the application of operating the paving equipment through the user's facial image, thereby improving the convenience and security of task execution.
[0005] In a first aspect, an embodiment of the present application provides an information processing method, the method is applied to a paving device, the paving device includes a camera, and the method includes:
[0006] In response to a first instruction from a first user, simulation data generated by the first instruction is obtained, and a first facial image of the first user is obtained through the camera, wherein the first instruction is used to simulate the paving equipment performing a first task or a subtask of the first task, and the first task belongs to an operation task set of the paving equipment; an evaluation value of the first user performing the first task is obtained based on the simulation data; if the evaluation value is greater than a preset threshold, application authorization information of the first user is generated based on the first facial image.
[0007] Optionally, before acquiring the first facial image of the first user through the camera, the method further includes: determining location information of the first user; and adjusting the position and / or angle of the camera based on the location information of the first user.
[0008] Optionally, determining the location information of the first user includes: sending a signal test instruction to an electronic device of the first user; receiving a feedback signal from the electronic device; and determining the location information of the first user based on a signal strength and a signal direction of the feedback signal.
[0009] Optionally, the first task is a photovoltaic panel installation task, and the simulation data includes a simulated installation position of the photovoltaic panel and a simulated moving route of the photovoltaic panel; and obtaining the evaluation value of the first user performing the first task based on the simulation data includes:
[0010] Based on the simulated moving route of the photovoltaic panel, a reasonable value of the first instruction is obtained; based on the simulated paving position of the photovoltaic panel, the placement accuracy of the photovoltaic panel is obtained; based on the placement accuracy and the reasonable value, an evaluation value of the first user performing the first task is obtained.
[0011] Optionally, the method further includes: if the evaluation value is less than or equal to the preset threshold, obtaining operation instruction information of the first task; and sending a task test instruction of the first task based on the operation instruction information.
[0012] Optionally, the method also includes: receiving a second instruction from a second user, the second instruction is used to control the paving equipment to perform a second task or a subtask of the second task, the second task belongs to an operation task set of the paving equipment, and the second instruction includes the application authorization information; obtaining a second facial image of the second user through the camera; obtaining the first facial image based on the application authorization information; matching the second facial image with the first facial image, and executing the second instruction when the match is successful.
[0013] Optionally, the method further includes: acquiring an environmental image through the camera; performing feature recognition on the environmental image to obtain an environmental feature set; determining a next instruction of the second instruction based on the second instruction and the environmental feature set; and sending the next instruction of the second instruction.
[0014] In a second aspect, an embodiment of the present application provides an information processing device, including:
[0015] an acquisition unit, configured to respond to a first instruction of a first user, acquire simulation data generated by the first instruction, and collect a first face image of the first user through the camera, wherein the first instruction is used to simulate the paving equipment performing a first task or a subtask of the first task, and the first task belongs to an operation task set of the paving equipment; and based on the simulation data, acquire an evaluation value of the first user performing the first task;
[0016] A generating unit is configured to generate application authorization information of the first user based on the first facial image if the evaluation value is greater than a preset threshold.
[0017] Optionally, the information processing device further includes a determination unit and an adjustment unit, wherein the determination unit is used to determine the location information of the first user; and the adjustment unit is used to adjust the position and / or angle of the camera based on the location information of the first user.
[0018] Optionally, the information processing device also includes a transceiver unit for sending a signal test instruction to the electronic device of the first user; receiving a feedback signal from the electronic device; and a determination unit for determining the location information of the first user based on the signal strength and signal direction of the feedback signal.
[0019] Optionally, the first task is a photovoltaic panel installation task, the simulation data includes a simulated installation position of the photovoltaic panel and a simulated moving route of the photovoltaic panel, and the acquisition unit is specifically used to obtain a reasonable value of the first instruction based on the simulated moving route of the photovoltaic panel; obtain the placement accuracy of the photovoltaic panel based on the simulated installation position of the photovoltaic panel; and obtain the evaluation value of the first user performing the first task based on the placement accuracy and the reasonable value.
[0020] Optionally, the information processing device further includes a transceiver unit, configured to obtain operation instruction information of the first task if the evaluation value is less than or equal to the preset threshold; and send a task test instruction of the first task based on the operation instruction information.
[0021] Optionally, the information processing device also includes a transceiver unit for receiving a second instruction from a second user, the second instruction being used to control the paving equipment to perform a second task or a subtask of the second task, the second task belonging to an operation task set of the paving equipment, and the second instruction including the application authorization information; the acquisition unit is also used to acquire a second facial image of the second user through the camera; acquire the first facial image based on the application authorization information; match the second facial image with the first facial image, and execute the second instruction when the match is successful.
[0022] Optionally, the information processing device further includes a determination unit. The acquisition unit is further used to collect an environmental image through the camera; perform feature recognition on the environmental image to obtain an environmental feature set; the determination unit is used to determine the next instruction of the second instruction based on the second instruction and the environmental feature set; and the transceiver unit is further used to send the next instruction of the second instruction.
[0023] In a third aspect, an embodiment of the present application provides a paving device, comprising a camera, a processor and a memory, wherein the memory is used to store computer-readable instructions, and the processor is used to call the instructions stored in the memory to execute the steps in the method described in the first aspect.
[0024] In a fourth aspect, an embodiment of the present application provides a chip, comprising a data interface and a processor, wherein the processor is used to execute the method in the above-mentioned first aspect and any possible implementation manner of the above-mentioned first aspect.
[0025] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program includes program instructions, which, when executed by a processor, enable the processor to execute the method of the above-mentioned first aspect and any possible implementation method of the above-mentioned first aspect.
[0026] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes program instructions, and when the program instructions are executed by a processor, the processor executes the method of the above-mentioned first aspect and any possible implementation manner of the above-mentioned first aspect.
[0027] Implementing the embodiments of the present application has the following beneficial effects:
[0028] The evaluation value of the first user performing the first task is obtained based on the simulated data generated by the first instruction of the first user. When the evaluation value is greater than the preset threshold, the application authorization information of the first user is generated based on the first facial image of the first user. In this way, the first user can control the paving equipment to work, which can improve the convenience of operating the paving equipment. The application authorization for operating the paving equipment can be performed through the user's facial image, which can improve the safety of task execution. Before authorization, the simulation of the paving equipment performing the task or the subtask of the task can avoid losses caused by accidents of the paving equipment or photovoltaic panels. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0030] Figure 1 A schematic diagram of the architecture of a paving system provided in an embodiment of the present application;
[0031] Figure 2 A flowchart of an information processing method provided in an embodiment of the present application;
[0032] Figure 3 A schematic diagram of the structure of an information processing device provided in an embodiment of the present application;
[0033] Figure 4A schematic diagram of the structure of a paving device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0034] The terms "first" and "second" in the specification, claims and drawings of this application are only used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.
[0035] "At least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three and more than three, and "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".
[0036] In the embodiment of the present application, the paving device is used to move the photovoltaic panel to the target area of the photovoltaic panel in the paving frame. Specifically, it can be applied to the cleaning scene of the photovoltaic panel, the replacement scene and other scenes of moving the photovoltaic panel, which are not limited here.
[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0038] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the architecture of a paving system provided in an embodiment of the present application. Figure 1 As shown, the paving system may include a paving device 10, a paving frame 20 and a photovoltaic panel 30. The paving device 10 may be used as follows Figure 1The form shown is a vehicle, the power system of the vehicle can be an electric system, a fuel system, a hybrid system, etc. The present application does not limit the power type of the vehicle, and does not limit the driving method of the paving equipment 10.
[0039] The driving method of the paving equipment 10 is given below. Figure 1 Two forms not shown, of which:
[0040] Form 1: A driver for operating the power system is provided inside the paving device 10, and the paving device 10 is driven to move by a person holding the driver or by an unmanned driving strategy.
[0041] Form 2: A remote controller compatible with the paving device 10 is provided, and the movement of the paving device 10 is controlled by the remote controller of the paving device 10 .
[0042] Optionally, the remote controller may be a physical remote controller, which may be understood as a special remote controller, on which buttons related to the operation type of the paving device 10 are arranged. For example, movement keys for controlling the paving device to turn left, turn right, move forward, move backward, etc. Another example is a button for controlling the operation of a mechanical arm or other device in the paving device 10. The present application does not limit the button type of the remote controller.
[0043] Optionally, the remote controller may be a virtual remote controller of the electronic device.
[0044] Among them, the electronic devices may be smart phones, tablet computers, laptop computers, PDAs, mobile internet devices (MIDs), wearable devices (such as smart watches, smart bracelets, etc.), smart voice interaction devices, smart home appliances (such as smart TVs, etc.) and vehicle-mounted devices, etc., without limitation here.
[0045] The electronic device may be a server, which may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, without limitation.
[0046] The virtual remote controller may be provided with virtual buttons for the above-mentioned related functions, or may be provided with trigger gestures for related functions, etc., and the present application does not limit the implementation of the virtual remote controller. It can be understood that by using the remote controller to trigger the paving device 10 to operate, the control efficiency can be improved.
[0047] It should be noted that the above two forms are only examples. In fact, there may be other forms of devices to drive the paving device 10 to move.
[0048] In the embodiment of the present application, the paving device 10 is used to place the photovoltaic panel 30 on the paving frame 20. The paving device 10 can also be used to move the photovoltaic panel 30 in the paving frame 20 to other locations, which is not limited here. The paving frame 20 is used to place at least one photovoltaic panel, and the placement area corresponding to the paving frame 20 includes the target area of each photovoltaic panel 30.
[0049] The present application does not limit the location of the target area of the photovoltaic panel 30, which can be determined according to a preset paving order. The preset paving order can be used to determine the placement position of the photovoltaic panel 30 on the paving frame 20. The paving order can be to pave the photovoltaic panels 30 from left to right, or to pave the photovoltaic panels 30 from right to left; or in the case where the paving frame 20 includes multiple layers of paving frames, the photovoltaic panels 30 can be laid layer by layer from left to right, and after the paving of one layer of paving frames is completed, the photovoltaic panels 30 of the next layer can be laid; or the photovoltaic panels 30 can be laid layer by layer from right to left, and after the paving of one layer of paving frames is completed, the photovoltaic panels 30 of the next layer can be laid; or one photovoltaic panel 30 can be laid on each layer and then one photovoltaic panel 30 can be laid on the next layer, etc.
[0050] Optionally, at least two paving sequences are included. For example, the upper layer of the paving frame 20 is paved with the photovoltaic panels 30 in a paving sequence from left to right, and the lower layer of the paving frame 20 is paved with the photovoltaic panels 30 in a paving sequence from right to left, so that the photovoltaic panels 30 of the upper layer can be paved first and then the photovoltaic panels 30 of the lower layer, or the photovoltaic panels 30 of the lower layer can be paved first and then the photovoltaic panels 30 of the upper layer, which is conducive to improving the paving efficiency.
[0051] exist Figure 1 In the embodiment, the paving frame 20 is exemplified by a single-layer frame. In fact, the paving frame 20 can be provided with a two-layer frame or a multi-layer frame so that at least two layers of photovoltaic panels can be placed.
[0052] Optionally, the paving frame 20 can be as follows Figure 1 The support member 201 is shown to keep the photovoltaic panel 30 at a certain distance from the ground. The present application does not limit the form of the support member 201, which can be as follows: Figure 1 Shown in upright position, or can be Figure 1 The form of the tripod not shown can also be in the form of wheels, etc. It can be understood that the paving frame 20 in the form of wheels is conducive to movement. The paving frame 20 in the form of a tripod has high stability.
[0053] In the embodiment of the present application, the paving device 10 may include a camera. The camera is used to collect images, such as environmental images, user images, etc. Among them, the environmental image may include device images of the paving device 10 and the paving frame 20, and the user image may include a face image.
[0054] Optionally, the paving device 10 may further include a carrier and a mechanical arm. The carrier is used to carry at least one photovoltaic panel 30, and specifically, at least one photovoltaic panel 30 may be carried in the gripping area of the carrier. The carrier may be a frame of fixed size, or may be a frame of a size to be modified. The frame may include panels in different directions, and the modification of the frame size may be achieved by triggering the movement of the panel by a telescopic cylinder, etc., which is not limited here.
[0055] The present application does not limit the number of photovoltaic panels 30 in the carrier, and the number of photovoltaic panels 30 can be determined according to the remaining height and / or occupied height of the carrier, and the thickness of the photovoltaic panels 30. Alternatively, the paving device 10 includes a counter and a weighing device. The weighing device can be set at the bottom of the carrier, and the number of photovoltaic panels 30 can be determined by measuring the weight of the photovoltaic panels 30 in the carrier, and then the number of photovoltaic panels 30 that have been paved can be determined by the number of remaining photovoltaic panels 30.
[0056] In the embodiment of the present application, the robot arm is used to grab the photovoltaic panel 30 and move the photovoltaic panel 30. For example, the photovoltaic panel 30 in the carrier is moved to the target area of the laying frame 20. For another example, the photovoltaic panel 30 on the laying frame 20 is moved to the carrier or the cleaning area.
[0057] Optionally, a camera is provided on the robotic arm.
[0058] The present application does not limit the position and number of the camera, which can be located on the upper arm or lower arm of the robot arm, or at the end of the robot arm, etc. It can be understood that by setting the camera on the robot arm, the image range of the camera can be expanded through the movement of the robot arm.
[0059] The present application does not limit the number of mechanical arms and load-bearing parts. For example, the number of mechanical arms is 1 and the number of load-bearing parts is 2. For another example, the number of mechanical arms and load-bearing parts is 2, and one mechanical arm corresponds to one load-bearing part. The present application does not limit the positions of mechanical arms and load-bearing parts. The mechanical arms can be set at the middle position of the paving equipment 10, and the load-bearing parts can be set at the two end positions of the paving equipment 10; or the mechanical arms can be set at the two end positions of the paving equipment 10, and the load-bearing parts can be set at the middle position of the paving equipment 10, etc.
[0060] It should be noted that Figure 1The structure and number of the devices or apparatuses of the system architecture shown are only examples, and the embodiments of the present application are not limited thereto. The figure may also include apparatuses or components not shown. For example, an auxiliary vehicle may also be included to assist the photovoltaic panel 30 in being placed in the paving frame 20. For the sake of simplicity, they are not described one by one in the drawings.
[0061] For another example, the functions of various devices or components can be realized by the control method of the processor. The processor can be located at any position of the paving device 10, or set in the above-mentioned electronic device. After the electronic device establishes a communication connection with the paving device 10, the control is realized by issuing instructions. This application does not limit the form of the processor and the control method.
[0062] The processor may include a control board or a control chip. Exemplarily, the processor may include a battery management system (BMS), a micro control unit (MCU), a central processing unit (CPU), other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a pulse width modulator (PWM) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0063] The working mode of the paving equipment will be described below in combination with the method.
[0064] Please refer to Figure 2 , Figure 2 The following is a flowchart of an information processing method provided in an embodiment of the present application. The method can be executed by an information processing device, which can be a paving device, or can be a device or component in a paving device, or a computer program (including program code, program instructions), etc. Figure 2 As shown, the method includes steps S201 to S203, wherein:
[0065] S201: In response to a first instruction from a first user, the paving equipment obtains simulation data generated by the first instruction, and obtains a first facial image of the first user through a camera, wherein the first instruction is used to simulate the execution of a first task or a subtask of the first task, and the first task belongs to an operation task set of the paving equipment.
[0066] In the embodiment of the present application, the first user is a user who is to be authorized to operate the paving equipment. The operation task set of the paving equipment may include the paving task of the photovoltaic panel, the moving task of the paving equipment, the adjustment task of the position of the bearing member, etc., which are not limited here. Among them, the paving task of the photovoltaic panel may include the steps of grabbing the photovoltaic panel, moving the photovoltaic panel and placing the photovoltaic panel, and may also include the steps of adjusting the position of the photovoltaic panel.
[0067] Moving the paving equipment may include the step of moving the paving equipment to a target position, or may include the step of adjusting the tilt angle of the paving equipment based on the road conditions, which step can prevent the paving equipment from tipping over when moving. Adjusting the position of the carrier may include the step of adjusting the size of the carrier based on the size of the photovoltaic panel, which step can protect the photovoltaic panel in the carrier and prevent the paving equipment from tilting and causing the photovoltaic panel to fall when moving. In the case where the carrier is in two different positions when not paving the photovoltaic panel and when paving the photovoltaic panel, adjusting the position of the carrier may also include the step of setting the carrier to a loading state or a unloading state. The position of the carrier when not paving the photovoltaic panel can facilitate the movement of the paving equipment. The position of the carrier when paving the photovoltaic panel can facilitate the paving of the photovoltaic panel. Adjusting the position of the carrier may also include the step of adjusting the position of the photovoltaic panel in the carrier, for example, after a photovoltaic panel in the gripping area of the carrier is gripped, the next photovoltaic panel in the carrier is moved to the gripping area.
[0068] Each task in the operation task set may include at least one subtask, which may be different steps of the above tasks, or substeps of each step. For example, the first task is the photovoltaic panel installation task, and the subtasks of the first task may include at least one step such as the photovoltaic panel task, the photovoltaic panel task, the photovoltaic panel placement task, and the photovoltaic panel position adjustment task.
[0069] In an embodiment of the present application, the first task may be any task in the operation task set of the paving equipment, that is, when the evaluation value of the first user executing a task in the operation task set is greater than a preset threshold, the application authorization information of the first user is generated. The first task may also be all tasks in the operation task set of the paving equipment, that is, when the evaluation value of the first user executing each task in the operation task set is greater than a preset threshold, the application authorization information of the first user is generated. The first task may also be at least one designated task in the operation task set of the paving equipment, that is, when the evaluation value of the first user executing a designated task is greater than a preset threshold, the application authorization information of the first user is generated.
[0070] This application does not limit the type of designated tasks and the method of determining designated tasks. The designated task can be a photovoltaic panel paving task, which can be determined based on the number of subtasks and the impact value of the task. Among them, the impact value of the task is used to describe whether the task has an impact on the various devices or components of the paving equipment when it is executed, and the degree of impact. The impact value of the task can be determined based on the number of devices involved in the execution of the task, or it can be weighted according to the preset weights of each device and the number of devices, or it can be weighted according to the preset weights of each subtask and the number of devices affected by each subtask, etc., which are not limited here.
[0071] In an embodiment of the present application, the user can be placed within the shooting range of the camera, and the user's facial image can be captured by a camera in a fixed position; or the position of the camera can be moved or the angle of the camera can be adjusted, and the user's facial image can be captured by a camera in a movable position.
[0072] In some feasible examples, before obtaining the first facial image of the first user through the camera, it also includes: determining the location information of the first user; and adjusting the position and / or angle of the camera based on the location information of the first user.
[0073] The location information of the first user may be carried in the first instruction, so that the location information of the first user is determined by the first instruction. Alternatively, the paving device may send a request message to the electronic device of the first user so that the electronic device replies with its location information, or the paving device may measure the location information of the first user, etc. This application does not limit the method for determining the location information of the first user.
[0074] In some feasible examples, the paving equipment determines the location information of the first user, including: the paving equipment sends a signal test instruction to the electronic device of the first user; the paving equipment receives a feedback signal from the electronic device; and determines the location information of the first user based on the signal strength and signal direction of the feedback signal.
[0075] Among them, the signal test instruction can be used to instruct the first user's electronic device to perform a signal test on the communication between the electronic device and the paving device. The feedback signal is a test signal that the electronic device replies to the signal test instruction. The signal strength of the feedback signal can be used to determine the distance of the electronic device relative to the paving device, and the signal direction of the feedback signal can be used to determine the offset position of the electronic device relative to the paving device, so that the location information of the first user can be determined according to the signal strength and signal direction of the feedback signal, thereby improving the accuracy of positioning.
[0076] In other feasible examples, the paving equipment determines the location information of the first user, including: collecting multiple environmental images through a camera, searching for a target image containing the user from the multiple environmental images; and determining the location information of the first user based on the target image.
[0077] Among them, the environmental image may specifically include a structural image of the paving equipment and a scene image of the scene where the paving equipment is located. The scene image may include a structural image of the paving frame, a structural image of the surrounding device, etc. There are at least two images with different angles in the multiple environmental images, or there may be at least two images with different positions. The location information of the environment where the paving equipment is located can be determined based on the environmental images at different angles and different positions collected by the camera. The location information may include the values corresponding to the x-axis, y-axis and z-axis of different reference objects, and may also include position parameters such as the angles corresponding to the x-axis, y-axis and z-axis. The reference object may be an object photographed in the environmental image. It can be understood that the number of target images containing users is 1, and the user in the target image can be used as the first user. If the number of target images containing users exceeds 1, the user in the target image may need to be identified. For example, the target image containing the first user is determined based on the signal direction and signal strength determined by the communication signal between the paving equipment and the electronic device, which can further improve the accuracy of positioning.
[0078] It can be understood that after determining the location information of the first user, the position and / or angle of the camera can be adjusted based on the location information to facilitate the camera to capture the first facial image of the first user.
[0079] The present application does not limit the method for adjusting the position and / or angle of the camera, and the adjustment can be made for the camera, or when the camera is set on the robotic arm, the position and / or angle of the camera can be adjusted for the robotic arm. Optionally, adjusting the position and / or angle of the camera based on the position information of the first user includes: determining a movement route of the robotic arm based on the position information of the first user; and moving the robotic arm based on the movement route so that the position and / or angle of the camera is adjusted.
[0080] The movement of the movement route of the robot arm requires the cooperation of devices with different degrees of freedom. These devices with different degrees of freedom may include the rotation joint of the robot arm itself, and may also include devices related to the rotation joint. It can be understood that by moving the robot arm, the position and / or angle of the camera on it can be adjusted, which can increase the adjustment range of the camera.
[0081] In the embodiment of the present application, simulation data refers to the data generated by simulating the paving equipment to perform the first task or a subtask of the first task. For example, the first instruction is the paving task of photovoltaic panels, and the simulation data may include the simulated paving position of photovoltaic panels and the simulated moving route of photovoltaic panels; the first instruction is to move paving equipment, and the simulation data may include the simulated moving position, simulated moving route, simulated moving duration, etc. of the paving equipment; the first instruction is to adjust the position of the carrier, and the simulation data may include the simulated adjustment position and simulated adjustment angle of the carrier. It can be understood that before authorization, simulating the paving equipment to perform the first task or a subtask of the first task can avoid accidents and losses to the paving equipment or photovoltaic panels.
[0082] S202: The paving equipment obtains an evaluation value of the first user performing the first task based on the simulation data.
[0083] In the embodiment of the present application, the evaluation value of the first user performing the first task is used to describe the proficiency and accuracy of the first user in performing the first task. The simulation data is data generated by simulating the paving equipment performing the first task or a subtask of the first task corresponding to the first instruction, which can be used to measure the level of the first user in performing the first task.
[0084] The present application does not limit the method for obtaining the evaluation value. Taking the first task as the photovoltaic panel paving task as an example, the simulation data includes a simulated paving image of the photovoltaic panel and a simulated moving route of the photovoltaic panel. Step S202 includes the following steps: the paving equipment obtains a reasonable value of the first instruction based on the simulated moving route of the photovoltaic panel; the paving equipment obtains the placement accuracy of the photovoltaic panel based on the simulated paving position of the photovoltaic panel; the paving equipment obtains the evaluation value of the first user performing the first task based on the placement accuracy and the reasonable value.
[0085] The placement accuracy of photovoltaic panels is used to describe the accuracy of placing photovoltaic panels at target locations. The area ratio of the photovoltaic panel in the paving area of the photovoltaic panel and the area ratio of the photovoltaic panel in the paving area of other photovoltaic panels can be determined based on the simulated paving image of the photovoltaic panel, and then the placement accuracy of the photovoltaic panel can be obtained by weighting these two area ratios.
[0086] The reasonable value of the first instruction is used to describe whether the first instruction can execute the first task or a subtask of the first task, and the degree of completion of executing the first task or a subtask of the first task. It is possible to first determine whether the moving route of the photovoltaic panel can move the photovoltaic panel to the target position based on the simulated moving route of the photovoltaic panel, and if so, then determine the matching value between the moving route of the photovoltaic panel and the optimal moving route, and obtain the reasonable value of the first instruction based on the matching value.
[0087] The evaluation value of the first user performing the first task may be a weighted value between the placement accuracy and the reasonable value. The present application does not limit the preset weights of the placement accuracy and the reasonable value, and may be determined according to the amount and data type of various types of simulation data.
[0088] It can be understood that when the first task is the paving task of photovoltaic panels, the reasonable value of the first instruction can be obtained based on the simulated moving route of the photovoltaic panels, and the placement accuracy of the photovoltaic panels can be obtained based on the simulated paving position of the photovoltaic panels. Then, the evaluation value of the first user performing the first task can be obtained based on the placement accuracy and the reasonable value, which can improve the accuracy of obtaining the evaluation value.
[0089] It should be noted that the method of making the types correspond is only an example. In fact, the evaluation value of the first user performing the first task can also be obtained by other methods. For example, when performing the first task, the evaluation score can be obtained based on the simulation data generated by each subtask, and then the evaluation score can be weighted to obtain the evaluation value of the first user performing the first task.
[0090] S203: If the evaluation value is greater than a preset threshold, the paving device generates application authorization information of the first user based on the first face image.
[0091] In an embodiment of the present application, the application authorization information is used to indicate that the first user who sends the first instruction is allowed to operate the paving device, which can specifically be the paving device that receives the first instruction. The application authorization information may include verification information of the authorized user, such as a facial image of the authorized user or a facial feature set of the authorized user.
[0092] The application authorization information may also include the time of application authorization, basic information of the authorized user, information of the authorized device, etc. Among them, the time of application authorization may include the start time of application authorization, and may also include the effective time of application authorization. In this way, the paving equipment can be operated within the effective time, and the user identity needs to be verified again when the effective time expires.
[0093] The basic information of the authorized user includes the authorized user's identification, age, gender, contact number, etc. The identification of the authorized object may include the authorized object's work number, name, account number, email address, contact number, etc. The device information of the authorized device refers to the device information of the paving device authorized by the authorized user, such as device number, device type, device location, etc., which is not limited here.
[0094] Optionally, the application authorization information is a coded image. Thus, by decoding the coded image, the specific content of the application authorization information can be obtained.
[0095] The present application does not limit how to generate the application authorization information. The facial feature points of the first face image may be encoded to obtain an encoded image of the application authorization information.
[0096] This application does not limit the preset threshold. When the evaluation value of the first user performing the first task is greater than the preset threshold, it means that the first user has passed the test of operating the paving equipment, and the application authorization information can be generated. If the evaluation value is less than the preset threshold, it means that the first user has not passed the test of operating the paving equipment and needs to be proficient in the operation of the paving equipment.
[0097] This application does not limit the case where the evaluation value is less than the preset threshold, and other tasks can be performed, or training can be used to learn how to operate the paving equipment. Or in other feasible examples, the method also includes: if the evaluation value is less than or equal to the preset threshold, obtaining the operation instruction information of the first task; and sending the task test instruction of the first task based on the operation instruction information.
[0098] Among them, the operation instruction information of the first task is used to prompt the operation method when executing the first task, which can be specifically the way of using the electronic device during the implementation of the first task, for example, first click the function button corresponding to the first function, then select the target item in the item column corresponding to the second function, and finally submit the function button corresponding to the confirmation function, etc.
[0099] The task test instruction is used to instruct the first user to perform the first task according to the operation instruction information. It can be understood that when the evaluation value is less than or equal to the preset threshold, sending the task test instruction of the first task to the electronic device of the first user based on the first operation instruction information can improve the success rate of the first user's authentication of the first task, and facilitate the first user to master the method of executing the instructions issued by the first task.
[0100] exist Figure 2 In the method shown, the evaluation value of the first user performing the first task is obtained based on the simulated data generated by the first instruction of the first user. When the evaluation value is greater than the preset threshold, the application authorization information of the first user is generated based on the first facial image of the first user. In this way, the first user can control the paving equipment to work, which can improve the convenience of operating the paving equipment. The application authorization for operating the paving equipment can be performed through the user's facial image, which can improve the security of task execution. And before authorization, simulating the paving equipment to perform tasks or subtasks of tasks can avoid losses caused by accidents of paving equipment or photovoltaic panels.
[0101] In some feasible examples, the method also includes: the paving device receives a second instruction from a second user; the paving device obtains a second facial image of the second user through a camera; the paving device matches the second facial image with the first facial image based on application authorization information, and executes the second instruction when the match is successful.
[0102] The second instruction is used to control the paving device to perform the second task or a subtask of the second task. The second task can be any task in the operation task set of the paving device. Optionally, the second task is the first task. The second instruction includes application authorization information, so that it can be determined that the electronic device sending the second instruction has been authorized to operate the paving device, and the identity information of the second user who operates the electronic device to send the second instruction can be identified.
[0103] In this example, a second facial image of the second user is acquired through the camera. The acquisition of the second facial image can refer to the above, where the second user moves to the shooting range of the camera, or the position and / or angle of the camera can be adjusted so that the camera can capture the second facial image of the second user, which will not be repeated here. The method of acquiring the first facial image based on the application authorization information can be understood as the inverse step of generating the application authorization information based on the first facial image. The matching process of the second facial image and the first facial image can be a matching process of the two images, or can be a matching process of the feature point sets in the two images. When the matching value is greater than the specified threshold, it is determined that the match is successful.
[0104] The present application does not limit the size of the specified threshold, which can be a fixed value, for example, 0.8, etc. Or it can be a dynamically set value, for example, the specified threshold is related to the safety value of the environment, the specified threshold is related to the authorization time length and the effective time length of the application authorization information, etc. Among them, the safety value of the environment can be determined based on whether the time of the second instruction belongs to working hours, whether the location of the paving equipment is an abnormal location, etc. It can be understood that the smaller the safety value, the more dangerous it is, and the specified threshold can be set to a larger value. The longer the authorization time length, the more proficient the authorized object is in the operation method of the paving equipment, and the specified threshold can be set to a smaller value. The faster the effective time length of the application authorization information expires, the closer the authorized object is to the time for re-verification, and the specified threshold can be set to a larger value.
[0105] It can be understood that after receiving the second instruction containing the application authorization information sent by the second user, the second face image of the second user is collected through the camera. First, the verified first face image is obtained based on the application authorization information, and the second face image is matched with the first face image. When the match is successful, it means that the second user is the first user and is an authorized user, and the second instruction sent by the second user can be executed. In this way, when the authorized application authorization information sends an instruction, the identity of the user sending the instruction is identified, which can further improve the safety of operating the paving equipment.
[0106] Furthermore, in addition to the effective duration of the application authorization information in the second instruction, a step of acquiring a second facial image of the second user through a camera is performed. Within the effective duration of the application authorization information, the step of the second instruction is performed. In this way, frequent identity authentication can be avoided, which can improve the convenience of operation.
[0107] In some feasible examples, the method also includes: the paving equipment collects environmental images through a camera; the paving equipment performs feature recognition on the environmental images to obtain an environmental feature set; the paving equipment determines the next instruction of the second instruction based on the second instruction and the environmental feature set; and the paving equipment sends the next instruction of the second instruction.
[0108] The environment image may refer to the above, which will not be described in detail here. The environment feature set may include the position parameter of the reference object. The next instruction of the second instruction may be the next task of the second task, or may be the next subtask of the subtask of the second task executed by the second instruction.
[0109] It can be understood that the position of the second task currently being performed by the paving equipment can be determined based on the environmental feature set, which can be specifically the position of the robot arm. The present application does not limit the method for determining the next instruction of the second instruction. The third instruction associated with the second instruction and the environmental feature set can be first obtained. When the number of third instructions is equal to 1, the third instruction is used as the second instruction; when the number of third instructions is greater than 1, the priority value of the execution of each third instruction is determined, and the third instruction with the largest priority value is selected as the second instruction. By sending the next instruction of the second instruction, the efficiency of operating the paving equipment can be improved.
[0110] The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is described below.
[0111] Please refer to Figure 3 , Figure 3 The schematic diagram of the structure of an information processing device provided in an embodiment of the present application is as follows. The information processing device 300 includes an acquisition unit 301, a generation unit 302, a transceiver unit 303, a determination unit 304 and an adjustment unit 305, wherein:
[0112] The acquisition unit 301 is used to respond to a first instruction of a first user, acquire simulation data generated by the first instruction, and collect a first face image of the first user through the camera, wherein the first instruction is used to simulate the paving equipment performing a first task or a subtask of the first task, and the first task belongs to an operation task set of the paving equipment; based on the simulation data, acquire an evaluation value of the first user performing the first task;
[0113] The generating unit 302 is configured to generate application authorization information of the first user based on the first facial image if the evaluation value is greater than a preset threshold.
[0114] Optionally, the determination unit 304 is used to determine the location information of the first user; and the adjustment unit 305 is used to adjust the position and / or angle of the camera based on the location information of the first user.
[0115] Optionally, the transceiver unit 303 is used to send a signal test instruction to the electronic device of the first user; receive a feedback signal from the electronic device; and the determination unit 304 is specifically used to determine the location information of the first user based on the signal strength and signal direction of the feedback signal.
[0116] Optionally, the first task is a photovoltaic panel installation task, the simulation data includes a simulated installation position of the photovoltaic panel and a simulated moving route of the photovoltaic panel, and the acquisition unit 301 is specifically used to obtain a reasonable value of the first instruction based on the simulated moving route of the photovoltaic panel; obtain the placement accuracy of the photovoltaic panel based on the simulated installation position of the photovoltaic panel; and obtain the evaluation value of the first user performing the first task based on the placement accuracy and the reasonable value.
[0117] Optionally, the transceiver unit 303 is used to obtain operation instruction information of the first task if the evaluation value is less than or equal to the preset threshold; and send a task test instruction of the first task based on the operation instruction information.
[0118] Optionally, the transceiver unit 303 is used to receive a second instruction from a second user, the second instruction is used to control the paving equipment to perform a second task or a subtask of the second task, the second task belongs to an operation task set of the paving equipment, and the second instruction includes the application authorization information; the acquisition unit 301 is also used to acquire a second facial image of the second user through the camera; acquire the first facial image based on the application authorization information; match the second facial image with the first facial image, and execute the second instruction when the match is successful.
[0119] Optionally, the acquisition unit 301 is also used to collect an environmental image through the camera; perform feature recognition on the environmental image to obtain an environmental feature set; the determination unit 304 is used to determine the next instruction of the second instruction based on the second instruction and the environmental feature set; the transceiver unit 303 is also used to send the next instruction of the second instruction.
[0120] It should be noted that the implementation of each unit can also refer to Figure 2 A corresponding description of at least one embodiment of the present invention.
[0121] Figure 4 This is a schematic diagram of the structure of a paving device provided in an embodiment of the present application. Figure 4 As shown, the paving device 10 includes a processor 401, a communication interface 402, a memory 403 and a camera 404. The processor 401, the communication interface 402, the memory 403 and the camera 404 can be connected to each other via a bus 405, or can be connected in other ways. Figure 3 The related functions implemented by the obtaining unit 301 , the generating unit 302 , the determining unit 304 and the adjusting unit 305 shown may be implemented by one or more processors 401 . Figure 3 The transceiver unit 303 shown may be implemented via one or more communication interfaces 402 .
[0122] The processor 401 includes one or more processors, for example, one or more central processing units. When the processor 401 is a central processing unit (CPU), the CPU may be a single-core CPU or a multi-core CPU. In the embodiment of the present application, the processor 401 is used to control the paving device 10 to implement Figure 2 The embodiment shown.
[0123] The communication interface 402 is used to realize communication with other devices. For example, if the paving device 10 is a terminal device, the communication interface 402 can realize communication between the terminal device and devices such as servers; if the paving device 10 is a server, the communication interface 402 can realize communication between the server and devices such as terminal devices.
[0124] The memory 403 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM), and is used to store relevant instructions and data.
[0125] In the embodiment of the present application, the memory 403 stores computer-readable instructions 406, and the processor 401 is used to execute the instructions stored in the memory 403. The above instructions are used to perform the following steps:
[0126] In response to a first instruction of a first user, obtaining simulation data generated by the first instruction, and obtaining a first face image of the first user through the camera, wherein the first instruction is used to simulate the paving equipment performing a first task or a subtask of the first task, and the first task belongs to an operation task set of the paving equipment;
[0127] Acquire, based on the simulation data, an evaluation value of the first user performing the first task;
[0128] If the evaluation value is greater than a preset threshold, application authorization information of the first user is generated based on the first facial image.
[0129] In a possible implementation, before acquiring the first facial image of the first user through the camera, the above instruction is further used to perform the following steps:
[0130] Determining location information of the first user;
[0131] The position and / or angle of the camera is adjusted based on the position information of the first user.
[0132] In a possible implementation manner, in determining the location information of the first user, the above instruction is specifically used to perform the following steps:
[0133] Sending a signal test instruction to the electronic device of the first user;
[0134] receiving a feedback signal from the electronic device;
[0135] The location information of the first user is determined based on the signal strength and signal direction of the feedback signal.
[0136] In a possible implementation, the first task is a photovoltaic panel installation task, and the simulation data includes a simulated installation position of the photovoltaic panel and a simulated moving route of the photovoltaic panel; in terms of obtaining the evaluation value of the first user performing the first task based on the simulation data, the above instructions are specifically used to perform the following steps:
[0137] Acquire a reasonable value of the first instruction based on the simulated moving route of the photovoltaic panel;
[0138] Obtaining a placement accuracy of the photovoltaic panel based on a simulated paving position of the photovoltaic panel;
[0139] An evaluation value of the first user performing the first task is obtained based on the placement accuracy and the reasonable value.
[0140] In a possible implementation, the above instruction is further used to execute the following steps:
[0141] If the evaluation value is less than or equal to the preset threshold, obtaining operation instruction information of the first task;
[0142] A task test instruction for the first task is sent based on the operation instruction information.
[0143] In a possible implementation, the above instruction is further used to execute the following steps:
[0144] receiving a second instruction from a second user, where the second instruction is used to control the paving device to perform a second task or a subtask of the second task, where the second task belongs to an operation task set of the paving device, and the second instruction includes the application authorization information;
[0145] Acquire a second facial image of the second user through the camera;
[0146] Acquire the first facial image based on the application authorization information;
[0147] The second facial image is matched with the first facial image, and when the match is successful, the second instruction is executed.
[0148] In a possible implementation, the above instruction is further used to execute the following steps:
[0149] Collecting environmental images through the camera;
[0150] Performing feature recognition on the environment image to obtain an environment feature set;
[0151] Determining a next instruction of the second instruction based on the second instruction and the set of environmental characteristics;
[0152] Send the next instruction of the second instruction.
[0153] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and the computer program includes program instructions. When the program instructions are executed by the processor, the processor executes part or all of the steps of any information processing method recorded in the above method embodiments.
[0154] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any information processing method recorded in the above method embodiments.
[0155] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.
[0156] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0157] In the several embodiments disclosed in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.
[0158] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.
[0159] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software program module.
[0160] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, RAM, ROM, mobile hard disks, magnetic disks or optical disks.
[0161] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of the present application. At the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. An information processing method, characterized in that: The method is based on a paving device, the paving device includes a camera, and the method includes: In response to a first instruction of a first user, obtaining simulation data generated by the first instruction, and obtaining a first face image of the first user through the camera, wherein the first instruction is used to simulate the paving equipment performing a first task or a subtask of the first task, the first task belongs to an operation task set of the paving equipment, and the first user is a user to be authorized to operate the paving equipment; Acquire, based on the simulation data, an evaluation value of the first user performing the first task; If the evaluation value is greater than a preset threshold, generating application authorization information of the first user based on the first face image, where the application authorization information of the first user is used to indicate that the first user is allowed to operate the paving equipment; If the evaluation value is less than or equal to the preset threshold, obtaining operation instruction information of the first task; A task test instruction for the first task is sent based on the operation instruction information, where the task test instruction for the first task is used to instruct the first user to perform the first task according to the operation instruction information.
2. The method according to claim 1, characterized in that Before acquiring the first facial image of the first user through the camera, the method further includes: Determining location information of the first user; The position and / or angle of the camera is adjusted based on the position information of the first user.
3. The method according to claim 2, characterized in that The determining the location information of the first user includes: Sending a signal test instruction to the electronic device of the first user; receiving a feedback signal from the electronic device; The location information of the first user is determined based on the signal strength and signal direction of the feedback signal.
4. The method according to any one of claims 1 to 3, characterized in that The first task is a photovoltaic panel installation task, and the simulation data includes a simulated installation position of the photovoltaic panel and a simulated moving route of the photovoltaic panel; The obtaining, based on the simulation data, an evaluation value of the first user performing the first task includes: Acquire a reasonable value of the first instruction based on the simulated moving route of the photovoltaic panel; Obtaining a placement accuracy of the photovoltaic panel based on a simulated paving position of the photovoltaic panel; An evaluation value of the first user performing the first task is obtained based on the placement accuracy and the reasonable value.
5. The method according to any one of claims 1 to 3, characterized in that The method further comprises: receiving a second instruction from a second user, where the second instruction is used to control the paving device to perform a second task or a subtask of the second task, where the second task belongs to an operation task set of the paving device, and the second instruction includes the application authorization information; Acquire a second facial image of the second user through the camera; Acquire the first facial image based on the application authorization information; The second facial image is matched with the first facial image, and when the match is successful, the second instruction is executed.
6. The method according to claim 5, characterized in that The method further comprises: Collecting environmental images through the camera; Performing feature recognition on the environment image to obtain an environment feature set; Determining a next instruction of the second instruction based on the second instruction and the set of environmental characteristics; Send the next instruction of the second instruction.
7. An information processing device, characterized in that: The device is based on a paving device, the paving device includes a camera, and the device includes a unit for executing a method according to any one of claims 1 to 6.
8. A paving device, characterized in that: The invention comprises a camera, a processor and a memory, wherein the memory is used to store computer-readable instructions, and the processor is used to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 6.
