Information processing device, method and related products

The blind assistance device enhances navigation for the blind by performing real-time scene recognition and path planning, addressing the limitations of existing tools and improving safety and convenience.

CN111832356BActive Publication Date: 2024-12-03CAMBRICON TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN201910316954.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-04-19
Publication Date
2024-12-03
Estimated Expiration
2040-05-02

AI Technical Summary

Technical Problem

Existing navigation tools for the blind, such as guide dogs and sticks, are inadequate for unknown routes, leading to low safety and navigation accuracy, and the blind often receive incorrect guidance due to visual impairment.

Method used

A blind assistance device with a processing unit and path planning unit that performs real-time scene recognition, path planning, and provides prompts to navigate around obstacles, utilizing interconnected computing devices for enhanced navigation capabilities.

Benefits of technology

Improves the safety and convenience of travel for the blind by enabling real-time obstacle avoidance and accurate navigation through scene recognition and path planning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111832356B_ABST
    Figure CN111832356B_ABST
Patent Text Reader

Abstract

The present application provides an information processing device, method and related products, wherein the related products include a neural network chip and a board, the board includes a storage device, an interface device and a control device as well as the neural network chip, the neural network chip is connected to the storage device, the control device and the interface device respectively; the storage device is used to store data; the interface device is used to realize data transmission between the neural network chip and an external device; the control device is used to monitor the status of the neural network chip.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and specifically to an information processing device, method and related products. Background Art

[0002] The blind are a vulnerable group in society. At present, the tools to assist the blind in their daily travel mainly rely on guide dogs and guide sticks. The training cost of guide dogs is high, the life span is limited, and they are not widely used. The guide stick consists of a pole and a handle to support the blind when walking. Although the blind have a guide stick to find the way, for navigation with known walking direction and walking route, only a guide stick or a navigation dog can find the way. However, for navigation with unknown walking direction and walking route, it is impossible to achieve it only with a guide stick or a blind dog. Even if the blind ask for directions, they cannot get correct guidance due to their visual impairment. Therefore, in the prior art, the safety of blind people's travel is low, and the accuracy of the navigation scheme that guides the blind to move forward is low. Summary of the invention

[0003] The embodiments of the present application provide a blind assistance device, a blind assistance method and related products, in order to identify objects in a scene, provide the best navigation route for the blind in real time, and facilitate the blind to travel.

[0004] In a first aspect, an embodiment of the present application provides a blind assistance device, the blind assistance device comprising a processing unit and a path planning unit, the processing unit comprising a computing unit and a control unit;

[0005] The control unit is used to read the first calculation instruction and send the first calculation instruction to the operation unit;

[0006] The computing unit is used to obtain target scene data and weights, and perform corresponding operations according to the target scene data, weights and the first computing instruction to obtain a scene recognition result;

[0007] The path planning unit is used to perform path planning according to the scene recognition result, obtain a target path, and perform a preset prompt operation on the target path.

[0008] In a second aspect, an embodiment of the present application provides a machine learning computing device, which includes one or more blind assistive devices described in the first aspect. The machine learning computing device is used to obtain data to be computed and control information from other processing devices, and perform specified machine learning operations, and transmit the execution results to other processing devices through an I / O interface;

[0009] When the machine learning computing device includes a plurality of the blind assistive devices, the plurality of computing devices may be linked and transmit data through a specific structure;

[0010] Among them, multiple blind assistance devices are interconnected and transmit data through a PCIE bus to support larger-scale machine learning operations; multiple blind assistance devices share the same control system or have their own control systems; multiple blind assistance devices share memory or have their own memory; the interconnection method of multiple blind assistance devices is any interconnection topology.

[0011] In a third aspect, an embodiment of the present application provides a combined processing device, which includes the machine learning processing device, the universal interconnection interface and other processing devices as described in the second aspect. The machine learning operation device interacts with the above-mentioned other processing devices to jointly complete the operation specified by the user. The combined processing device may also include a storage device, which is respectively connected to the machine learning operation device and the other processing device, and is used to store data of the machine learning operation device and the other processing device.

[0012] In a fourth aspect, an embodiment of the present application provides a neural network chip, which includes the blind assistance device described in the first aspect, the machine learning operation device described in the second aspect, or the combined processing device described in the third aspect.

[0013] In a fifth aspect, an embodiment of the present application provides a neural network chip packaging structure, wherein the neural network chip packaging structure includes the neural network chip described in the fourth aspect above;

[0014] In a sixth aspect, an embodiment of the present application provides a board card, which includes the neural network chip packaging structure described in the fifth aspect above.

[0015] In a seventh aspect, an embodiment of the present application provides an electronic device, which includes the neural network chip described in the sixth aspect or the board described in the sixth aspect.

[0016] In an eighth aspect, an embodiment of the present application provides a method for assisting the blind, the method being applied to a blind assistance device, the blind assistance device comprising a processing unit and a path planning unit, the processing unit comprising a computing unit and a control unit; the method comprising:

[0017] The control unit reads a first calculation instruction and sends the first calculation instruction to the operation unit;

[0018] The computing unit obtains target scene data and weights, performs corresponding operations according to the target scene data, weights and the first computing instruction, and obtains a scene recognition result;

[0019] The path planning unit performs path planning according to the scene recognition result to obtain a target path, and performs a preset prompt operation on the target path.

[0020] In some embodiments, the electronic device includes a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a server, a cloud server, a camera, a camcorder, a projector, a watch, a headset, a mobile storage, a wearable device, a vehicle, a household appliance, and / or a medical device.

[0021] In some embodiments, the means of transportation include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging, ultrasound equipment and / or electrocardiographs.

[0022] Implementing the embodiments of the present application has the following beneficial effects:

[0023] It can be seen that in the embodiment of the present application, through the blind assistance device, real-time scene recognition is performed on the environment in which the blind is located, the recognition result is obtained, the path is planned according to the recognition result, and the prompt operation is performed on the obtained target path to ensure that the blind avoids obstacles when moving, thereby improving the convenience of the blind's travel and ensuring the safety of the blind's travel. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 A structural block diagram of a blind assistive device provided in an embodiment of the present application;

[0026] Figure 2 A structural block diagram of a path planning unit provided in an embodiment of the present application;

[0027] Figure 3 A flowchart of a blind assistance method provided in an embodiment of the present application;

[0028] Figure 4 A structural block diagram of a combined processing device provided in an embodiment of the present application;

[0029] Figure 5 It is a structural diagram of another combined processing device provided in an embodiment of the present application.

[0030] Figure 6 It is a structural schematic diagram of a board provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0033] Reference to "embodiments" herein means that a particular feature, result, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0034] See also Figure 1 , Figure 1 An auxiliary device for the blind provided in an embodiment of the present application includes a processing unit 10 and a path planning unit 20. The processing unit 10 includes a computing unit 100 and a control unit 200, wherein:

[0035] The control unit 200 is used to read a first computing instruction and send the first computing instruction to the computing unit 100, wherein the first computing instruction is a computing instruction related to object recognition in a scene;

[0036] The computing unit 100 is used to obtain target scene data and weights, and perform corresponding operations according to the target scene data, weights and the first computing instruction to obtain a scene recognition result, wherein obtaining the target scene data includes: the computing unit 100 actively obtains the target scene data from the storage unit or passively receives the target scene data sent by the storage unit;

[0037] The path planning unit 20 is used to perform path planning according to the scene recognition result, obtain a target path, and perform a preset prompt operation on the target path.

[0038] Optionally, performing a preset prompt operation on the target path may be: broadcasting the target path by voice; or indicating the target path by vibration; or transmitting the target path to a blind user through brain wave stimulation, etc. This application does not make a sole limitation on the prompt operation.

[0039] In one possible example, Figure 2 As shown, the path planning unit 20 includes a planning unit 201 and a speech unit 202. The planning unit is used to: perform path planning according to the scene recognition result, obtain the target path, and send the target path to the speech unit 202; the speech unit 202 is used to voice broadcast the received target path.

[0040] It can be seen that in the embodiment of the present application, the computing unit performs scene recognition on the obtained target scene data to obtain a scene recognition result. The path planning unit determines the target path based on the scene recognition result, and performs a preset prompt operation on the target path to instruct the blind user to proceed along the target path, thereby avoiding obstacles in the way and improving the safety and convenience of the blind user's travel.

[0041] In a possible example, the computing unit 100 includes a master processing circuit 101 and at least one slave processing circuit 102, wherein:

[0042] The control unit 200 is further configured to parse the first calculation instruction to obtain a plurality of first sub-calculation instructions, and send the plurality of first sub-calculation instructions to the main processing circuit 101;

[0043] The main processing circuit 101 is used to perform pre-order processing on the target scene data, compose the target scene data into input data, split the input data into multiple input data blocks, and distribute the multiple input data blocks and the multiple first sub-computation instructions to the slave processing circuit 102, wherein the pre-order processing includes data type conversion processing, sparse processing or data filling processing, etc., and the data type conversion processing includes: converting fixed-point data to floating-point data or converting floating-point data to fixed-point data, etc.

[0044] The slave processing circuit 102 is configured to perform intermediate operations in parallel according to the received multiple input data blocks and the multiple first sub-computing instructions to obtain multiple intermediate results, and transmit the multiple intermediate results to the master processing circuit 101;

[0045] The main processing circuit 101 is used to perform subsequent processing on the received multiple intermediate results to obtain the object category corresponding to the target scene data, and use the object category as the scene recognition result, wherein the subsequent processing includes activation processing, normalization processing, pooling processing or bias processing, etc., and the activation processing includes: sigmoid, tanh, relu, softmax or linear activation.

[0046] The following is an actual example to illustrate the above-mentioned input data splitting method. Since the output result and the input data have the same data type, the splitting method is basically the same. Assuming that the data type is a matrix, the matrix is ​​H*W. If the value of H is small (less than the set threshold, such as 100), the splitting method can be to split the matrix H*W into H vectors along the H direction (each vector is a row of the matrix H*W), each vector is an input data block, and mark the position of the first element of the input data block, that is, mark it as an input data block (h, w), where h and w are the values ​​of the first element of the input data block h, w in the H direction and the W direction, respectively. For example, for the first input data block, h=1, w=1. After receiving the input data block (h, w) from the processing circuit, the input data block (h, w) is multiplied and accumulated one by one with each column element of the weight to obtain the input intermediate result (w, i). The w of the intermediate result is the w value of the input data block, and i is the column value of the column element calculated with the input data block. The main processing circuit determines that the position of the intermediate result in the hidden layer output result is w, i. For example, the input data block (1,1) and the input intermediate result (1,1) calculated by the first column of weights, the main processing circuit arranges the input intermediate result (1,1) in the first row and first column of the hidden layer output result.

[0047] In one possible example, Figure 1 As shown, the blind assistive device further includes an image acquisition unit 30, wherein:

[0048] An image acquisition unit 30 is used to acquire a scene image and obtain original scene data corresponding to the scene image;

[0049] The control unit 200 is further configured to obtain a second calculation instruction and send the second calculation instruction to the operation unit, wherein the second calculation instruction is a calculation instruction related to the target detection algorithm;

[0050] The operation unit 100 is also used to obtain the original scene data and weights, perform corresponding operations according to the original scene data, weights and second calculation instructions to obtain output results, determine the candidate area of ​​the scene image according to the output results, and mark the original scene data corresponding to the scene image contained in the candidate area as the target scene data, and the candidate area includes at least one object.

[0051] Of course, after receiving the second calculation instruction, the control unit needs to parse the second calculation instruction to obtain multiple second sub-calculation instructions, and send the split multiple second sub-calculation instructions to the operation unit 100, that is, perform a process similar to parsing the first calculation instruction, which will not be repeated here.

[0052] Optionally, the image acquisition unit 30 may be a camera. For example, when the camera may be a depth-of-field camera, the scene image may be acquired by: shooting with the camera to obtain the scene image; or shooting a video with the camera to obtain a scene video, and extracting a plurality of frames of images from the scene video at a preset sampling rate to obtain the scene image. For example, a frame of image may be sampled from the scene video every 10 frames, 15 frames, 20 frames or other values ​​to obtain the scene image.

[0053] In a possible example, when path planning is performed according to the scene recognition result to obtain the target path, the path planning unit 20 is specifically used to: plan several routes according to the scene recognition result; determine the object category on each of the several routes; determine the score of each route according to the object category, wherein there is a preset mapping relationship between the object category and the score; and use the route with the highest score as the target path.

[0054] In the above possible examples, when the image acquisition unit 30 is a depth of field camera, planning several routes according to the scene recognition result can be: acquiring the depth of field information of the scene image through the depth of field camera; determining the distance between the object corresponding to the scene recognition result and the depth of field camera according to the depth of field information; constructing a scene map according to the distance, and planning several routes from the scene map.

[0055] In one possible example, Figure 1 As shown, the processing unit also includes an instruction cache 300, a weight cache 400, an input neuron cache 500, and an output neuron cache 600;

[0056] The instruction cache 300 is used to cache the calculation instructions required in the calculation, and send the calculation instructions to the calculation unit 100 through the control unit 200, wherein the calculation instructions include the first calculation instructions and the second calculation instructions mentioned above, that is, when performing scene recognition and obtaining the scene recognition result, the first calculation instruction is cached, and when determining the target scene data from the original scene data, the second calculation instruction is cached;

[0057] The input neuron buffer 500 is used to buffer scene data and send the buffered scene data to the computing unit 100. The scene data includes original scene data and target scene data in the original scene data. That is, when scene recognition is performed and the scene recognition result is obtained, the target scene data is buffered. When the target scene data is determined from the original scene data, the original scene data is buffered.

[0058] The weight cache 400 is used to cache the weights required in the calculation and send the cached weights to the operation unit 100. The weights include a first weight associated with the first operation instruction and a second weight associated with the second operation instruction. That is, when performing scene recognition and obtaining a scene recognition result, the first weight is cached, and when determining the target scene data from the original scene data, the second weight is cached;

[0059] The output neuron cache 600 is used to cache the scene recognition result sent by the computing unit 100, and send the cached scene recognition result to the path planning unit 20; specifically, the scene recognition result is sent to the storage unit 40 through the direct memory access unit 50, and the path planning unit 20 reads the scene recognition result from the storage unit 40.

[0060] In one possible example, Figure 1 As shown, the blind assistive device further includes a storage unit 40 and a direct memory access unit 50. The storage unit 40 may include: one or any combination of a register and a cache. Specifically, the cache is used to store the calculation instruction (the first calculation instruction and the second calculation instruction); the register is used to store the scene data acquired by the image acquisition unit 30; and the cache is a high-speed temporary cache. The direct memory access unit 50 is used to read or store data and / or instructions from the storage unit 40. Specifically, the read scene data is sent to the input neuron cache 500, the read calculation instruction (the first calculation instruction or the second calculation instruction) is sent to the instruction cache 300, the read weight is sent to the weight cache 400, and the scene recognition result is read from the output neuron cache 600, and the scene recognition result is stored in the storage unit 40.

[0061] See also Figure 3 , Figure 3 A blind assistance method is provided in an embodiment of the present application. The method is applied to the above-mentioned blind assistance device. The blind assistance device includes a processing unit and a path planning unit. The processing unit includes a computing unit, a control unit and a cache unit. The method includes the following steps:

[0062] Step S301: The control unit reads a first calculation instruction and sends the first calculation instruction to the operation unit.

[0063] Step S302: The computing unit obtains target scene data and weights, performs corresponding operations according to the target scene data, weights and the first computing instruction, and obtains a scene recognition result.

[0064] Step S303: The path planning unit performs path planning according to the scene recognition result to obtain a target path, and performs a preset prompt operation on the target path.

[0065] In a possible example, the computing unit includes a master processing circuit and at least one slave processing circuit;

[0066] The control unit parses the first calculation instruction to obtain a plurality of first sub-calculation instructions, and sends the plurality of first sub-calculation instructions to the main processing circuit;

[0067] The master processing circuit performs pre-order processing on the target scene data, composes the target scene data into input data, splits the input data into a plurality of input data blocks, and distributes the plurality of input data blocks and the plurality of first sub-computing instructions to the slave processing circuit;

[0068] The slave processing circuit performs intermediate operations in parallel according to the received multiple input data blocks and the multiple first sub-computing instructions to obtain multiple intermediate results, and transmits the multiple intermediate results to the master processing circuit;

[0069] The main processing circuit performs subsequent processing on the received multiple intermediate results to obtain the object category corresponding to the target scene data, and uses the object category as the scene recognition result.

[0070] In a possible example, performing path planning according to the scene recognition result to obtain the target path specifically includes:

[0071] Planning a number of travel routes according to the scene recognition results;

[0072] Determining a category of an object on each of the plurality of travel routes;

[0073] Scoring of each route according to object category, wherein there is a preset mapping relationship between object category and score;

[0074] The highest scoring route is selected as the target route.

[0075] In a possible example, the device further includes an image acquisition unit;

[0076] The image acquisition unit is used to acquire a scene image and obtain original scene data corresponding to the scene image;

[0077] The control unit is used to obtain a second calculation instruction and send the second calculation instruction to the operation unit;

[0078] The operation unit is used to obtain the original scene data and weights, perform corresponding operations according to the original scene data, weights and second calculation instructions to obtain output results, determine the candidate area of ​​the scene image according to the output results, and mark the original scene data corresponding to the scene image contained in the candidate area as the target scene data, and the candidate area includes at least one object.

[0079] In a possible example, the processing unit further includes an instruction cache, an input neuron cache, a weight cache, and an output neuron cache;

[0080] The instruction cache caches the computational instructions required in the operation;

[0081] The input neuron buffer caches the scene data and sends the cached scene data to the computing unit;

[0082] The weight cache caches weights and sends the cached weights to the computing unit;

[0083] The output neuron cache caches the scene recognition result sent by the computing unit, and sends the cached scene recognition result to the path planning unit.

[0084] The present application also discloses a GRU device, which includes one or more computing devices mentioned in the present application, which are used to obtain data to be calculated and control information from other processing devices, perform specified GRU operations, and transmit the execution results to peripheral devices through I / O interfaces. Peripheral devices include cameras, displays, mice, keyboards, network cards, wifi interfaces, and servers. When more than one computing device is included, the computing devices can be linked and data can be transmitted through a specific structure, for example, interconnected and data can be transmitted through a PCIE bus to support larger-scale convolutional neural network training operations. At this time, the same control system can be shared, or each independent control system can be provided; memory can be shared, or each accelerator can have its own memory. In addition, the interconnection method can be any interconnection topology.

[0085] The GRU device has high compatibility and can be connected to various types of servers through a PCIE interface.

[0086] The present application also discloses a combined processing device, which includes the above-mentioned GRU device, a universal interconnection interface, and other processing devices. The GRU operation device interacts with other processing devices to jointly complete the operation specified by the user. Figure 4 A schematic diagram of a combined processing device.

[0087] Other processing devices include one or more types of general / special processors such as central processing unit (CPU), graphics processing unit (GPU), neural network processor, etc. There is no limit on the number of processors included in other processing devices. Other processing devices serve as interfaces between the GRU computing device and external data and control, including data handling, and complete basic control of the start and stop of the GRU computing device; other processing devices can also collaborate with the GRU computing device to complete computing tasks.

[0088] The universal interconnection interface is used to transmit data and control instructions between the GRU device and other processing devices. The GRU device obtains the required input data from other processing devices and writes it into the storage device on the GRU device chip; it can obtain control instructions from other processing devices and write them into the control cache on the GRU device chip; it can also read data in the storage module of the GRU device and transmit it to other processing devices.

[0089] Optionally, the structure is Figure 5 As shown, a storage device may also be included, and the storage device is connected to the GRU device and the other processing device respectively. The storage device is used to store data in the GRU device and the other processing device, and is particularly suitable for data that cannot be fully stored in the internal storage of the GRU device or other processing devices.

[0090] The combined processing device can be used as a SOC chip system for mobile phones, robots, drones, video surveillance equipment and other devices, effectively reducing the core area of ​​the control part, improving the processing speed, and reducing the overall power consumption. In this case, the universal interconnection interface of the combined processing device is connected to certain components of the device. Certain components include cameras, displays, mice, keyboards, network cards, and wifi interfaces.

[0091] In some embodiments, a chip is also applied for, which includes the above-mentioned GRU device or combined processing device.

[0092] In some embodiments, a chip packaging structure is applied for, which includes the above-mentioned chip.

[0093] In some embodiments, a board is applied for, which includes the above chip packaging structure. Figure 6 , Figure 6 A board is provided. In addition to the chip 389, the board may also include other supporting components, including but not limited to: a storage device 390, an interface device 391 and a control device 392;

[0094] The memory device 390 is connected to the chip in the chip package structure through a bus for storing data. The memory device may include multiple groups of storage units 393. Each group of storage units is connected to the chip through a bus. It is understood that each group of storage units may be DDR SDRAM (English: Double Data Rate SDRAM, double rate synchronous dynamic random access memory).

[0095] DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read out on the rising and falling edges of the clock pulse. The speed of DDR is twice that of standard SDRAM. In one embodiment, the storage device may include 4 groups of storage units. Each group of storage units may include multiple DDR4 particles (chips). In one embodiment, the chip may include 4 72-bit DDR4 controllers, 64 bits of the above 72-bit DDR4 controllers are used to transmit data, and 8 bits are used for ECC verification. It can be understood that when DDR4-3200 particles are used in each group of storage units, the theoretical bandwidth of data transmission can reach 25600MB / s.

[0096] In one embodiment, each group of the storage units includes a plurality of double rate synchronous dynamic random access memories arranged in parallel. DDR can transmit data twice in one clock cycle. A controller for controlling DDR is arranged in the chip to control the data transmission and data storage of each of the storage units.

[0097] The interface device is electrically connected to the chip in the chip packaging structure. The interface device is used to realize data transmission between the chip and an external device (such as a server or a computer). For example, in one embodiment, the interface device can be a standard PCIE interface. For example, the data to be processed is transmitted to the chip by the server through the standard PCIE interface to realize data transfer. Preferably, when the PCIE 3.0X 16 interface is used for transmission, the theoretical bandwidth can reach 16000MB / s. In another embodiment, the interface device can also be other interfaces. This application does not limit the specific forms of expression of the above-mentioned other interfaces. The interface unit can realize the switching function. In addition, the calculation results of the chip are still transmitted back to the external device (such as a server) by the interface device.

[0098] The control device is electrically connected to the chip. The control device is used to monitor the state of the chip. Specifically, the chip and the control device can be electrically connected via an SPI interface. The control device may include a single-chip microcomputer (Micro Controller Unit, MCU). For example, the chip may include multiple processing chips, multiple processing cores or multiple processing circuits, which can drive multiple loads. Therefore, the chip can be in different working states such as multi-load and light load. The control device can realize the regulation of the working states of multiple processing chips, multiple processing and / or multiple processing circuits in the chip.

[0099] In some embodiments, an electronic device is applied for, which includes the above-mentioned board.

[0100] Electronic devices include data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, mobile phones, driving recorders, navigators, sensors, cameras, servers, cloud servers, cameras, camcorders, projectors, watches, headphones, mobile storage, wearable devices, vehicles, household appliances, and / or medical devices.

[0101] The transportation means include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging, ultrasound machines and / or electrocardiographs.

[0102] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0103] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0104] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0105] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0106] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software program module.

[0107] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0108] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0109] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A blind assistive device, It is characterized in that The blind assist device comprises a processing unit and a path planning unit, wherein the processing unit comprises a computing unit and a control unit; The control unit is used to read the first calculation instruction and send the first calculation instruction to the operation unit; The computing unit is used to obtain target scene data and weights, and perform corresponding operations according to the target scene data, weights and the first computing instruction to obtain a scene recognition result; including: The computing unit includes a master processing circuit and at least one slave processing circuit; The control unit is further configured to parse the first calculation instruction to obtain a plurality of first sub-calculation instructions, and send the plurality of first sub-calculation instructions to the main processing circuit; The master processing circuit is used to perform pre-order processing on the target scene data, compose the target scene data into input data, split the input data into multiple input data blocks, and distribute the multiple input data blocks and the multiple first sub-computing instructions to the slave processing circuit; The step of splitting the input data into a plurality of input data blocks comprises: Splitting the input data into multiple vectors in the row direction, each vector being an input data block, and adding a mark to the input data block corresponding to each vector according to the position of the first element in each vector in the input data; The slave processing circuit is used to perform intermediate operations in parallel according to the received multiple input data blocks and the multiple first sub-computing instructions to obtain multiple intermediate results, and transmit the multiple intermediate results to the master processing circuit; The step of performing intermediate operations in parallel according to the received multiple input data blocks and the multiple first sub-computing instructions to obtain multiple intermediate results includes: Based on the multiple first sub-computation instructions, perform product and accumulation operations on each input data block and each column element in the weight in parallel to obtain an intermediate result corresponding to each input data block, and add a mark to the intermediate result according to the mark of each input data block and the column value of each column element, so that the main processing circuit arranges the position of each intermediate result in the output result based on the mark; The main processing circuit is used to perform subsequent processing on the received multiple intermediate results to obtain the object category corresponding to the target scene data, and use the object category as the scene recognition result; The path planning unit is used to perform path planning according to the scene recognition result, obtain a target path, and perform a preset prompt operation on the target path.

2. The device according to claim 1, It is characterized in that When performing path planning according to the scene recognition result to obtain the target path, the path planning unit is specifically used to: Planning a number of travel routes according to the scene recognition results; Determining a category of an object on each of the plurality of travel routes; The score of each route is determined according to the object category on each route, and the route with the highest score is used as the target path, wherein there is a preset mapping relationship between the object category and the score.

3. The device according to claim 1 or 2, It is characterized in that The device also includes an image acquisition unit; The image acquisition unit is used to acquire a scene image and obtain original scene data corresponding to the scene image; The control unit is used to read the second calculation instruction and send the second calculation instruction to the operation unit; The operation unit is used to obtain the original scene data and weights, perform corresponding operations according to the original scene data, weights and second calculation instructions to obtain output results, determine the candidate area of ​​the scene image according to the output results, and mark the original scene data corresponding to the scene image contained in the candidate area as target scene data, wherein the candidate area includes at least one object.

4. The device according to any one of claims 1 to 3, It is characterized in that The processing unit also includes an instruction cache, an input neuron cache, a weight cache, and an output neuron cache; The instruction cache is used to cache the calculation instructions required in the operation; The input neuron buffer is used to buffer scene data and send the buffered scene data to the computing unit; The weight cache is used to cache weights and send the cached weights to the computing unit; The output neuron cache is used to cache the scene recognition result and send the cached scene recognition result to the path planning unit.

5. A neural network chip, It is characterized in that The neural network chip includes the device as described in any one of claims 1-4.

6. A board, It is characterized in that The board includes: a storage device, an interface device and a control device and a neural network chip as claimed in claim 5; Wherein, the neural network chip is connected to the storage device, the control device and the interface device respectively; The storage device is used to store data; The interface device is used to realize data transmission between the neural network chip and external equipment; The control device is used to monitor the state of the neural network chip.

7. A method for assisting the blind. It is characterized in that The method is applied to a blind assistance device, the blind assistance device comprises a processing unit and a path planning unit, the processing unit comprises a computing unit and a control unit; the method comprises: The control unit reads a first calculation instruction and sends the first calculation instruction to the operation unit; The computing unit obtains target scene data and weights, performs corresponding operations according to the target scene data, weights and the first computing instruction, and obtains a scene recognition result; including: The computing unit includes a master processing circuit and at least one slave processing circuit; The control unit parses the first calculation instruction to obtain a plurality of first sub-calculation instructions, and sends the plurality of first sub-calculation instructions to the main processing circuit; The master processing circuit performs pre-order processing on the target scene data, composes the target scene data into input data, splits the input data into a plurality of input data blocks, and distributes the plurality of input data blocks and the plurality of first sub-computation commands to the slave processing circuit; The step of splitting the input data into a plurality of input data blocks comprises: Splitting the input data into multiple vectors in the row direction, each vector being an input data block, and adding a mark to the input data block corresponding to each vector according to the position of the first element in each vector in the input data; The slave processing circuit performs intermediate operations in parallel according to the received multiple input data blocks and the multiple first sub-computation instructions to obtain multiple intermediate results, and transmits the multiple intermediate results to the master processing circuit; The step of performing intermediate operations in parallel according to the received multiple input data blocks and the multiple first sub-computing instructions to obtain multiple intermediate results includes: Based on the multiple first sub-computation instructions, perform product and accumulation operations on each input data block and each column element in the weight in parallel to obtain an intermediate result corresponding to each input data block, and add a mark to the intermediate result according to the mark of each input data block and the column value of each column element, so that the main processing circuit arranges the position of each intermediate result in the output result based on the mark; The main processing circuit performs subsequent processing on the received multiple intermediate results to obtain the object category corresponding to the target scene data, and uses the object category as the scene recognition result; The path planning unit performs path planning according to the scene recognition result to obtain a target path, and performs a preset prompt operation on the target path.

8. The method according to claim 7, It is characterized in that The performing path planning according to the scene recognition result to obtain the target path specifically includes: Planning a number of travel routes according to the scene recognition results; Determining a category of an object on each of the plurality of travel routes; The score of each route is determined according to the object category on each route, and the route with the highest score is used as the target path, wherein there is a preset mapping relationship between the object category and the score.

Citation Information

Patent Citations

  • Auxiliary travel method and auxiliary travel system based on image recognition

    CN108844545A

  • Conversion method, apparatus, computer apparatus and storage medium

    CN109492241A